Oct 31 17:24:32.410429 ip-10-0-129-127 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Oct 31 17:24:32.410439 ip-10-0-129-127 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Oct 31 17:24:32.410448 ip-10-0-129-127 systemd[1]: kubelet.service: Failed with result 'resources'.
Oct 31 17:24:32.410770 ip-10-0-129-127 systemd[1]: Failed to start Kubernetes Kubelet.
Oct 31 17:24:42.659740 ip-10-0-129-127 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Oct 31 17:24:42.659761 ip-10-0-129-127 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot c6e4931c99a249b9abc47c8389a6675d --
Oct 31 17:26:37.718714 ip-10-0-129-127 systemd[1]: Starting Kubernetes Kubelet...
Oct 31 17:26:38.155129 ip-10-0-129-127 kubenswrapper[2578]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 31 17:26:38.155129 ip-10-0-129-127 kubenswrapper[2578]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Oct 31 17:26:38.155129 ip-10-0-129-127 kubenswrapper[2578]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 31 17:26:38.155129 ip-10-0-129-127 kubenswrapper[2578]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 31 17:26:38.158688 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.158611    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 31 17:26:38.158688 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.158687    2578 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 31 17:26:38.158688 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.158691    2578 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 31 17:26:38.158774 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.158694    2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 31 17:26:38.158774 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.158697    2578 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 31 17:26:38.158774 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.158700    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 31 17:26:38.158774 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.158702    2578 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 31 17:26:38.158774 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.158705    2578 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 31 17:26:38.158774 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.158707    2578 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 31 17:26:38.158774 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.158711    2578 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 31 17:26:38.158774 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.158713    2578 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 31 17:26:38.158774 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.158716    2578 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 31 17:26:38.158774 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.158718    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 31 17:26:38.158774 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.158721    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 31 17:26:38.158774 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.158724    2578 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 31 17:26:38.158774 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.158726    2578 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 31 17:26:38.158774 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.158729    2578 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 31 17:26:38.158774 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.158732    2578 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 31 17:26:38.158774 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.158734    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 31 17:26:38.158774 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.158737    2578 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 31 17:26:38.158774 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.158739    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 31 17:26:38.158774 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.158742    2578 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 31 17:26:38.159245 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.158744    2578 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 31 17:26:38.159245 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.158747    2578 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 31 17:26:38.159245 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.158749    2578 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 31 17:26:38.159245 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.158752    2578 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 31 17:26:38.159245 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.158754    2578 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 31 17:26:38.159245 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.158764    2578 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 31 17:26:38.159245 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.158767    2578 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 31 17:26:38.159245 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.158770    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 31 17:26:38.159245 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.158774    2578 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 31 17:26:38.159245 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.158779    2578 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 31 17:26:38.159245 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.158782    2578 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 31 17:26:38.159245 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.158785    2578 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 31 17:26:38.159245 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.158788    2578 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 31 17:26:38.159245 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.158791    2578 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 31 17:26:38.159245 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.158793    2578 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 31 17:26:38.159245 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.158796    2578 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 31 17:26:38.159245 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.158798    2578 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 31 17:26:38.159245 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.158801    2578 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 31 17:26:38.159245 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.158803    2578 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 31 17:26:38.159729 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.158806    2578 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 31 17:26:38.159729 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.158808    2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 31 17:26:38.159729 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.158811    2578 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 31 17:26:38.159729 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.158813    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 31 17:26:38.159729 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.158816    2578 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 31 17:26:38.159729 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.158818    2578 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 31 17:26:38.159729 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.158821    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 31 17:26:38.159729 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.158823    2578 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 31 17:26:38.159729 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.158826    2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 31 17:26:38.159729 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.158828    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 31 17:26:38.159729 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.158831    2578 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 31 17:26:38.159729 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.158833    2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 31 17:26:38.159729 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.158836    2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 31 17:26:38.159729 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.158839    2578 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 31 17:26:38.159729 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.158842    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 31 17:26:38.159729 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.158845    2578 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 31 17:26:38.159729 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.158848    2578 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 31 17:26:38.159729 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.158850    2578 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 31 17:26:38.159729 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.158854    2578 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 31 17:26:38.160184 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.158857    2578 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 31 17:26:38.160184 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.158859    2578 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 31 17:26:38.160184 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.158862    2578 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 31 17:26:38.160184 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.158864    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 31 17:26:38.160184 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.158866    2578 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 31 17:26:38.160184 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.158869    2578 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 31 17:26:38.160184 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.158872    2578 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 31 17:26:38.160184 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.158874    2578 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 31 17:26:38.160184 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.158877    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 31 17:26:38.160184 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.158880    2578 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 31 17:26:38.160184 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.158883    2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 31 17:26:38.160184 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.158885    2578 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 31 17:26:38.160184 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.158888    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 31 17:26:38.160184 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.158890    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 31 17:26:38.160184 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.158893    2578 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 31 17:26:38.160184 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.158895    2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 31 17:26:38.160184 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.158898    2578 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 31 17:26:38.160184 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.158901    2578 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 31 17:26:38.160184 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.158903    2578 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 31 17:26:38.160184 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.158906    2578 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 31 17:26:38.160688 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.158908    2578 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 31 17:26:38.160688 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.158911    2578 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 31 17:26:38.160688 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.158913    2578 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 31 17:26:38.160688 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.158916    2578 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 31 17:26:38.160688 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.158918    2578 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 31 17:26:38.160688 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.158921    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 31 17:26:38.160688 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.158923    2578 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 31 17:26:38.160688 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.158926    2578 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 31 17:26:38.160688 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.158929    2578 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 31 17:26:38.160688 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.158932    2578 feature_gate.go:386] unrecognized feature gate: Example2
Oct 31 17:26:38.160688 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.158934    2578 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 31 17:26:38.160688 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.158937    2578 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 31 17:26:38.160688 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.158940    2578 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 31 17:26:38.160688 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.158943    2578 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 31 17:26:38.160688 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.158945    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 31 17:26:38.160688 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.158948    2578 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 31 17:26:38.160688 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.158950    2578 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 31 17:26:38.160688 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.158953    2578 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 31 17:26:38.160688 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.158956    2578 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 31 17:26:38.160688 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.158958    2578 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 31 17:26:38.161213 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.158961    2578 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 31 17:26:38.161213 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.158963    2578 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 31 17:26:38.161213 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.158968    2578 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 31 17:26:38.161213 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.158971    2578 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 31 17:26:38.161213 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.158974    2578 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 31 17:26:38.161213 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.158978    2578 feature_gate.go:386] unrecognized feature gate: Example
Oct 31 17:26:38.161213 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.159736    2578 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 31 17:26:38.161213 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.159742    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 31 17:26:38.161213 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.159745    2578 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 31 17:26:38.161213 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.159748    2578 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 31 17:26:38.161213 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.159750    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 31 17:26:38.161213 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.159753    2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 31 17:26:38.161213 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.159756    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 31 17:26:38.161213 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.159759    2578 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 31 17:26:38.161213 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.159762    2578 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 31 17:26:38.161213 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.159764    2578 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 31 17:26:38.161213 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.159767    2578 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 31 17:26:38.161213 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.159770    2578 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 31 17:26:38.161213 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.159772    2578 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 31 17:26:38.161213 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.159777    2578 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 31 17:26:38.161710 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.159780    2578 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 31 17:26:38.161710 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.159783    2578 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 31 17:26:38.161710 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.159785    2578 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 31 17:26:38.161710 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.159788    2578 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 31 17:26:38.161710 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.159790    2578 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 31 17:26:38.161710 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.159793    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 31 17:26:38.161710 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.159796    2578 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 31 17:26:38.161710 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.159799    2578 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 31 17:26:38.161710 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.159801    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 31 17:26:38.161710 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.159804    2578 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 31 17:26:38.161710 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.159806    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 31 17:26:38.161710 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.159809    2578 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 31 17:26:38.161710 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.159811    2578 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 31 17:26:38.161710 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.159814    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 31 17:26:38.161710 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.159816    2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 31 17:26:38.161710 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.159819    2578 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 31 17:26:38.161710 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.159821    2578 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 31 17:26:38.161710 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.159824    2578 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 31 17:26:38.161710 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.159826    2578 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 31 17:26:38.162157 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.159829    2578 feature_gate.go:386] unrecognized feature gate: Example
Oct 31 17:26:38.162157 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.159832    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 31 17:26:38.162157 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.159834    2578 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 31 17:26:38.162157 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.159837    2578 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 31 17:26:38.162157 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.159840    2578 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 31 17:26:38.162157 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.159843    2578 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 31 17:26:38.162157 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.159845    2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 31 17:26:38.162157 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.159847    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 31 17:26:38.162157 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.159851    2578 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 31 17:26:38.162157 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.159855    2578 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 31 17:26:38.162157 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.159857    2578 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 31 17:26:38.162157 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.159860    2578 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 31 17:26:38.162157 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.159863    2578 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 31 17:26:38.162157 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.159866    2578 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 31 17:26:38.162157 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.159868    2578 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 31 17:26:38.162157 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.159871    2578 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 31 17:26:38.162157 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.159873    2578 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 31 17:26:38.162157 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.159876    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 31 17:26:38.162157 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.159878    2578 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 31 17:26:38.162621 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.159881    2578 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 31 17:26:38.162621 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.159884    2578 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 31 17:26:38.162621 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.159887    2578 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 31 17:26:38.162621 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.159889    2578 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 31 17:26:38.162621 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.159892    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 31 17:26:38.162621 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.159896    2578 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 31 17:26:38.162621 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.159899    2578 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 31 17:26:38.162621 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.159902    2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 31 17:26:38.162621 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.159905    2578 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 31 17:26:38.162621 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.159907    2578 feature_gate.go:386] unrecognized feature gate: Example2
Oct 31 17:26:38.162621 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.159911    2578 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 31 17:26:38.162621 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.159913    2578 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 31 17:26:38.162621 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.159916    2578 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 31 17:26:38.162621 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.159919    2578 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 31 17:26:38.162621 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.159921    2578 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 31 17:26:38.162621 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.159924    2578 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 31 17:26:38.162621 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.159926    2578 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 31 17:26:38.162621 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.159929    2578 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 31 17:26:38.162621 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.159932    2578 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 31 17:26:38.163079 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.159935    2578 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 31 17:26:38.163079 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.159937    2578 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 31 17:26:38.163079 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.159940    2578 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 31 17:26:38.163079 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.159942    2578 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 31 17:26:38.163079 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.159945    2578 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 31 17:26:38.163079 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.159948    2578 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 31 17:26:38.163079 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.159951    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 31 17:26:38.163079 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.159954    2578 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 31 17:26:38.163079 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.159957    2578 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 31 17:26:38.163079 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.159959    2578 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 31 17:26:38.163079 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.159962    2578 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 31 17:26:38.163079 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.159964    2578 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 31 17:26:38.163079 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.159967    2578 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 31 17:26:38.163079 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.159969    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 31 17:26:38.163079 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.159973    2578 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 31 17:26:38.163079 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.159975    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 31 17:26:38.163079 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.159978    2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 31 17:26:38.163079 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.159980    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 31 17:26:38.163079 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.159983    2578 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 31 17:26:38.163079 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.159985    2578 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 31 17:26:38.163580 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.159988    2578 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 31 17:26:38.163580 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.159990    2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 31 17:26:38.163580 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.159993    2578 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 31 17:26:38.163580 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.159996    2578 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 31 17:26:38.163580 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.159999    2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 31 17:26:38.163580 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.160002    2578 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 31 17:26:38.163580 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.160004    2578 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 31 17:26:38.163580 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.160007    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 31 17:26:38.163580 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.160009    2578 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 31 17:26:38.163580 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.160012    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 31 17:26:38.163580 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.160014    2578 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 31 17:26:38.163580 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.160017    2578 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 31 17:26:38.163580 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.160019    2578 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 31 17:26:38.163580 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.160022    2578 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 31 17:26:38.163580 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.160025    2578 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 31 17:26:38.163580 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.160517    2578 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 31 17:26:38.163580 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.160522    2578 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 31 17:26:38.163580 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.160526    2578 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 31 17:26:38.163580 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.160529    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 31 17:26:38.163580 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.160532    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 31 17:26:38.164067 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.160535    2578 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 31 17:26:38.164067 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.160538    2578 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 31 17:26:38.164067 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.160540    2578 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 31 17:26:38.164067 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.160543    2578 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 31 17:26:38.164067 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.160545    2578 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 31 17:26:38.164067 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.160548    2578 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 31 17:26:38.164067 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.160553    2578 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 31 17:26:38.164067 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.160557    2578 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 31 17:26:38.164067 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.160559    2578 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 31 17:26:38.164067 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.160562    2578 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 31 17:26:38.164067 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.160565    2578 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 31 17:26:38.164067 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.160567    2578 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 31 17:26:38.164067 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.160570    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 31 17:26:38.164067 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.160572    2578 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 31 17:26:38.164067 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.160575    2578 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 31 17:26:38.164067 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.160578    2578 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 31 17:26:38.164067 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.160581    2578 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 31 17:26:38.164067 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.160584    2578 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 31 17:26:38.164067 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.160586    2578 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 31 17:26:38.164538 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.160589    2578 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 31 17:26:38.164538 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.160591    2578 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 31 17:26:38.164538 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.160594    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 31 17:26:38.164538 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.160597    2578 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 31 17:26:38.164538 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.160599    2578 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 31 17:26:38.164538 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.160602    2578 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 31 17:26:38.164538 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.160604    2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 31 17:26:38.164538 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.160607    2578 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 31 17:26:38.164538 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.160609    2578 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 31 17:26:38.164538 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.160611    2578 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 31 17:26:38.164538 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.160614    2578 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 31 17:26:38.164538 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.160616    2578 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 31 17:26:38.164538 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.160619    2578 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 31 17:26:38.164538 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.160622    2578 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 31 17:26:38.164538 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.160624    2578 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 31 17:26:38.164538 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.160626    2578 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 31 17:26:38.164538 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.160629    2578 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 31 17:26:38.164538 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.160631    2578 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 31 17:26:38.164538 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.160634    2578 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 31 17:26:38.164538 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.160636    2578 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 31 17:26:38.165014 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.160639    2578 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 31 17:26:38.165014 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.160642    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 31 17:26:38.165014 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.160644    2578 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 31 17:26:38.165014 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.160647    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 31 17:26:38.165014 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.160649    2578 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 31 17:26:38.165014 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.160652    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 31 17:26:38.165014 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.160654    2578 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 31 17:26:38.165014 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.160657    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 31 17:26:38.165014 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.160660    2578 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 31 17:26:38.165014 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.160662    2578 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 31 17:26:38.165014 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.160665    2578 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 31 17:26:38.165014 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.160667    2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 31 17:26:38.165014 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.160670    2578 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 31 17:26:38.165014 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.160673    2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 31 17:26:38.165014 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.160675    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 31 17:26:38.165014 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.160680    2578 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 31 17:26:38.165014 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.160682    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 31 17:26:38.165014 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.160684    2578 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 31 17:26:38.165014 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.160687    2578 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 31 17:26:38.165486 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.160689    2578 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 31 17:26:38.165486 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.160692    2578 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 31 17:26:38.165486 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.160695    2578 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 31 17:26:38.165486 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.160697    2578 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 31 17:26:38.165486 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.160700    2578 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 31 17:26:38.165486 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.160703    2578 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 31 17:26:38.165486 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.160706    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 31 17:26:38.165486 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.160708    2578 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 31 17:26:38.165486 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.160711    2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 31 17:26:38.165486 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.160713    2578 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 31 17:26:38.165486 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.160716    2578 feature_gate.go:386] unrecognized feature gate: Example2
Oct 31 17:26:38.165486 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.160718    2578 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 31 17:26:38.165486 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.160721    2578 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 31 17:26:38.165486 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.160723    2578 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 31 17:26:38.165486 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.160726    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 31 17:26:38.165486 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.160729    2578 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 31 17:26:38.165486 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.160731    2578 feature_gate.go:386] unrecognized feature gate: Example
Oct 31 17:26:38.165486 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.160734    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 31 17:26:38.165486 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.160737    2578 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 31 17:26:38.165486 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.160739    2578 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 31 17:26:38.165977 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.160742    2578 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 31 17:26:38.165977 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.160744    2578 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 31 17:26:38.165977 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.160747    2578 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 31 17:26:38.165977 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.160749    2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 31 17:26:38.165977 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.160752    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 31 17:26:38.165977 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.160756    2578 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 31 17:26:38.165977 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.160760    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 31 17:26:38.165977 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.160762    2578 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 31 17:26:38.165977 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.160766    2578 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 31 17:26:38.165977 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.160770    2578 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 31 17:26:38.165977 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.160773    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 31 17:26:38.165977 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.160776    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 31 17:26:38.165977 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.160778    2578 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 31 17:26:38.165977 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.160781    2578 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 31 17:26:38.165977 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.160783    2578 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 31 17:26:38.165977 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.160786    2578 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 31 17:26:38.165977 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.160788    2578 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 31 17:26:38.165977 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.160791    2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 31 17:26:38.165977 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.160793    2578 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 31 17:26:38.166403 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.160796    2578 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 31 17:26:38.166403 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.160798    2578 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 31 17:26:38.166403 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.160801    2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 31 17:26:38.166403 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.160803    2578 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 31 17:26:38.166403 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.161950    2578 flags.go:64] FLAG: --address="0.0.0.0"
Oct 31 17:26:38.166403 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.161960    2578 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Oct 31 17:26:38.166403 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.161966    2578 flags.go:64] FLAG: --anonymous-auth="true"
Oct 31 17:26:38.166403 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.161971    2578 flags.go:64] FLAG: --application-metrics-count-limit="100"
Oct 31 17:26:38.166403 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.161977    2578 flags.go:64] FLAG: --authentication-token-webhook="false"
Oct 31 17:26:38.166403 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.161980    2578 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Oct 31 17:26:38.166403 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.161984    2578 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Oct 31 17:26:38.166403 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.161988    2578 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Oct 31 17:26:38.166403 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.161992    2578 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Oct 31 17:26:38.166403 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.161995    2578 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Oct 31 17:26:38.166403 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.161998    2578 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Oct 31 17:26:38.166403 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.162001    2578 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Oct 31 17:26:38.166403 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.162004    2578 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Oct 31 17:26:38.166403 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.162007    2578 flags.go:64] FLAG: --cgroup-root=""
Oct 31 17:26:38.166403 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.162010    2578 flags.go:64] FLAG: --cgroups-per-qos="true"
Oct 31 17:26:38.166403 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.162013    2578 flags.go:64] FLAG: --client-ca-file=""
Oct 31 17:26:38.166403 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.162016    2578 flags.go:64] FLAG: --cloud-provider="external"
Oct 31 17:26:38.166403 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.162021    2578 flags.go:64] FLAG: --cluster-dns="[]"
Oct 31 17:26:38.166403 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.162025    2578 flags.go:64] FLAG: --cluster-domain=""
Oct 31 17:26:38.166968 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.162028    2578 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Oct 31 17:26:38.166968 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.162031    2578 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Oct 31 17:26:38.166968 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.162034    2578 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Oct 31 17:26:38.166968 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.162038    2578 flags.go:64] FLAG: --container-log-max-files="5"
Oct 31 17:26:38.166968 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.162042    2578 flags.go:64] FLAG: --container-log-max-size="10Mi"
Oct 31 17:26:38.166968 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.162045    2578 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Oct 31 17:26:38.166968 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.162048    2578 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Oct 31 17:26:38.166968 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.162051    2578 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Oct 31 17:26:38.166968 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.162054    2578 flags.go:64] FLAG: --contention-profiling="false"
Oct 31 17:26:38.166968 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.162057    2578 flags.go:64] FLAG: --cpu-cfs-quota="true"
Oct 31 17:26:38.166968 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.162060    2578 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Oct 31 17:26:38.166968 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.162063    2578 flags.go:64] FLAG: --cpu-manager-policy="none"
Oct 31 17:26:38.166968 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.162066    2578 flags.go:64] FLAG: --cpu-manager-policy-options=""
Oct 31 17:26:38.166968 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.162072    2578 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Oct 31 17:26:38.166968 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.162075    2578 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Oct 31 17:26:38.166968 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.162078    2578 flags.go:64] FLAG: --enable-debugging-handlers="true"
Oct 31 17:26:38.166968 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.162081    2578 flags.go:64] FLAG: --enable-load-reader="false"
Oct 31 17:26:38.166968 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.162084    2578 flags.go:64] FLAG: --enable-server="true"
Oct 31 17:26:38.166968 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.162087    2578 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Oct 31 17:26:38.166968 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.162091    2578 flags.go:64] FLAG: --event-burst="100"
Oct 31 17:26:38.166968 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.162094    2578 flags.go:64] FLAG: --event-qps="50"
Oct 31 17:26:38.166968 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.162097    2578 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Oct 31 17:26:38.166968 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.162101    2578 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Oct 31 17:26:38.166968 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.162103    2578 flags.go:64] FLAG: --eviction-hard=""
Oct 31 17:26:38.167561 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.162108    2578 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Oct 31 17:26:38.167561 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.162111    2578 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Oct 31 17:26:38.167561 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.162114    2578 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Oct 31 17:26:38.167561 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.162117    2578 flags.go:64] FLAG: --eviction-soft=""
Oct 31 17:26:38.167561 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.162120    2578 flags.go:64] FLAG: --eviction-soft-grace-period=""
Oct 31 17:26:38.167561 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.162123    2578 flags.go:64] FLAG: --exit-on-lock-contention="false"
Oct 31 17:26:38.167561 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.162127    2578 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Oct 31 17:26:38.167561 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.162131    2578 flags.go:64] FLAG: --experimental-mounter-path=""
Oct 31 17:26:38.167561 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.162134    2578 flags.go:64] FLAG: --fail-cgroupv1="true"
Oct 31 17:26:38.167561 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.162137    2578 flags.go:64] FLAG: --fail-swap-on="true"
Oct 31 17:26:38.167561 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.162140    2578 flags.go:64] FLAG: --feature-gates=""
Oct 31 17:26:38.167561 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.162144    2578 flags.go:64] FLAG: --file-check-frequency="20s"
Oct 31 17:26:38.167561 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.162147    2578 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Oct 31 17:26:38.167561 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.162150    2578 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Oct 31 17:26:38.167561 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.162153    2578 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Oct 31 17:26:38.167561 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.162157    2578 flags.go:64] FLAG: --healthz-port="10248"
Oct 31 17:26:38.167561 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.162160    2578 flags.go:64] FLAG: --help="false"
Oct 31 17:26:38.167561 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.162163    2578 flags.go:64] FLAG: --hostname-override="ip-10-0-129-127.ec2.internal"
Oct 31 17:26:38.167561 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.162166    2578 flags.go:64] FLAG: --housekeeping-interval="10s"
Oct 31 17:26:38.167561 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.162169    2578 flags.go:64] FLAG: --http-check-frequency="20s"
Oct 31 17:26:38.167561 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.162172    2578 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Oct 31 17:26:38.167561 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.162176    2578 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Oct 31 17:26:38.167561 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.162179    2578 flags.go:64] FLAG: --image-gc-high-threshold="85"
Oct 31 17:26:38.168105 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.162182    2578 flags.go:64] FLAG: --image-gc-low-threshold="80"
Oct 31 17:26:38.168105 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.162185    2578 flags.go:64] FLAG: --image-service-endpoint=""
Oct 31 17:26:38.168105 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.162188    2578 flags.go:64] FLAG: --kernel-memcg-notification="false"
Oct 31 17:26:38.168105 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.162191    2578 flags.go:64] FLAG: --kube-api-burst="100"
Oct 31 17:26:38.168105 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.162194    2578 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Oct 31 17:26:38.168105 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.162198    2578 flags.go:64] FLAG: --kube-api-qps="50"
Oct 31 17:26:38.168105 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.162200    2578 flags.go:64] FLAG: --kube-reserved=""
Oct 31 17:26:38.168105 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.162203    2578 flags.go:64] FLAG: --kube-reserved-cgroup=""
Oct 31 17:26:38.168105 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.162206    2578 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Oct 31 17:26:38.168105 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.162209    2578 flags.go:64] FLAG: --kubelet-cgroups=""
Oct 31 17:26:38.168105 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.162212    2578 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Oct 31 17:26:38.168105 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.162215    2578 flags.go:64] FLAG: --lock-file=""
Oct 31 17:26:38.168105 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.162218    2578 flags.go:64] FLAG: --log-cadvisor-usage="false"
Oct 31 17:26:38.168105 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.162220    2578 flags.go:64] FLAG: --log-flush-frequency="5s"
Oct 31 17:26:38.168105 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.162223    2578 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Oct 31 17:26:38.168105 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.162230    2578 flags.go:64] FLAG: --log-json-split-stream="false"
Oct 31 17:26:38.168105 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.162233    2578 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Oct 31 17:26:38.168105 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.162236    2578 flags.go:64] FLAG: --log-text-split-stream="false"
Oct 31 17:26:38.168105 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.162238    2578 flags.go:64] FLAG: --logging-format="text"
Oct 31 17:26:38.168105 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.162241    2578 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Oct 31 17:26:38.168105 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.162245    2578 flags.go:64] FLAG: --make-iptables-util-chains="true"
Oct 31 17:26:38.168105 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.162248    2578 flags.go:64] FLAG: --manifest-url=""
Oct 31 17:26:38.168105 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.162251    2578 flags.go:64] FLAG: --manifest-url-header=""
Oct 31 17:26:38.168105 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.162255    2578 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Oct 31 17:26:38.168105 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.162258    2578 flags.go:64] FLAG: --max-open-files="1000000"
Oct 31 17:26:38.168729 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.162262    2578 flags.go:64] FLAG: --max-pods="110"
Oct 31 17:26:38.168729 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.162265    2578 flags.go:64] FLAG: --maximum-dead-containers="-1"
Oct 31 17:26:38.168729 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.162268    2578 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Oct 31 17:26:38.168729 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.162271    2578 flags.go:64] FLAG: --memory-manager-policy="None"
Oct 31 17:26:38.168729 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.162274    2578 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Oct 31 17:26:38.168729 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.162277    2578 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Oct 31 17:26:38.168729 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.162280    2578 flags.go:64] FLAG: --node-ip="0.0.0.0"
Oct 31 17:26:38.168729 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.162283    2578 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Oct 31 17:26:38.168729 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.162289    2578 flags.go:64] FLAG: --node-status-max-images="50"
Oct 31 17:26:38.168729 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.162292    2578 flags.go:64] FLAG: --node-status-update-frequency="10s"
Oct 31 17:26:38.168729 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.162295    2578 flags.go:64] FLAG: --oom-score-adj="-999"
Oct 31 17:26:38.168729 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.162298    2578 flags.go:64] FLAG: --pod-cidr=""
Oct 31 17:26:38.168729 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.162302    2578 flags.go:64] FLAG: --pod-manifest-path=""
Oct 31 17:26:38.168729 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.162304    2578 flags.go:64] FLAG: --pod-max-pids="-1"
Oct 31 17:26:38.168729 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.162307    2578 flags.go:64] FLAG: --pods-per-core="0"
Oct 31 17:26:38.168729 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.162310    2578 flags.go:64] FLAG: --port="10250"
Oct 31 17:26:38.168729 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.162313    2578 flags.go:64] FLAG: --protect-kernel-defaults="false"
Oct 31 17:26:38.168729 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.162316    2578 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-017ba90e9c135b582"
Oct 31 17:26:38.168729 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.162320    2578 flags.go:64] FLAG: --qos-reserved=""
Oct 31 17:26:38.168729 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.162322    2578 flags.go:64] FLAG: --read-only-port="10255"
Oct 31 17:26:38.168729 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.162325    2578 flags.go:64] FLAG: --register-node="true"
Oct 31 17:26:38.168729 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.162328    2578 flags.go:64] FLAG: --register-with-taints=""
Oct 31 17:26:38.168729 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.162337    2578 flags.go:64] FLAG: --registry-burst="10"
Oct 31 17:26:38.168729 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.162340    2578 flags.go:64] FLAG: --registry-qps="5"
Oct 31 17:26:38.168729 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.162342    2578 flags.go:64] FLAG: --reserved-cpus=""
Oct 31 17:26:38.168729 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.162345    2578 flags.go:64] FLAG: --reserved-memory=""
Oct 31 17:26:38.169865 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.162349    2578 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Oct 31 17:26:38.169865 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.162352    2578 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Oct 31 17:26:38.169865 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.162355    2578 flags.go:64] FLAG: --rotate-certificates="false"
Oct 31 17:26:38.169865 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.162358    2578 flags.go:64] FLAG: --rotate-server-certificates="false"
Oct 31 17:26:38.169865 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.162360    2578 flags.go:64] FLAG: --runonce="false"
Oct 31 17:26:38.169865 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.162363    2578 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Oct 31 17:26:38.169865 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.162366    2578 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Oct 31 17:26:38.169865 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.162370    2578 flags.go:64] FLAG: --seccomp-default="false"
Oct 31 17:26:38.169865 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.162373    2578 flags.go:64] FLAG: --serialize-image-pulls="true"
Oct 31 17:26:38.169865 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.162376    2578 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Oct 31 17:26:38.169865 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.162378    2578 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Oct 31 17:26:38.169865 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.162382    2578 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Oct 31 17:26:38.169865 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.162385    2578 flags.go:64] FLAG: --storage-driver-password="root"
Oct 31 17:26:38.169865 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.162388    2578 flags.go:64] FLAG: --storage-driver-secure="false"
Oct 31 17:26:38.169865 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.162391    2578 flags.go:64] FLAG: --storage-driver-table="stats"
Oct 31 17:26:38.169865 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.162393    2578 flags.go:64] FLAG: --storage-driver-user="root"
Oct 31 17:26:38.169865 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.162396    2578 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Oct 31 17:26:38.169865 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.162399    2578 flags.go:64] FLAG: --sync-frequency="1m0s"
Oct 31 17:26:38.169865 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.162402    2578 flags.go:64] FLAG: --system-cgroups=""
Oct 31 17:26:38.169865 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.162405    2578 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Oct 31 17:26:38.169865 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.162411    2578 flags.go:64] FLAG: --system-reserved-cgroup=""
Oct 31 17:26:38.169865 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.162413    2578 flags.go:64] FLAG: --tls-cert-file=""
Oct 31 17:26:38.169865 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.162416    2578 flags.go:64] FLAG: --tls-cipher-suites="[]"
Oct 31 17:26:38.169865 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.162420    2578 flags.go:64] FLAG: --tls-min-version=""
Oct 31 17:26:38.169865 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.162423    2578 flags.go:64] FLAG: --tls-private-key-file=""
Oct 31 17:26:38.171013 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.162425    2578 flags.go:64] FLAG: --topology-manager-policy="none"
Oct 31 17:26:38.171013 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.162428    2578 flags.go:64] FLAG: --topology-manager-policy-options=""
Oct 31 17:26:38.171013 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.162431    2578 flags.go:64] FLAG: --topology-manager-scope="container"
Oct 31 17:26:38.171013 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.162435    2578 flags.go:64] FLAG: --v="2"
Oct 31 17:26:38.171013 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.162449    2578 flags.go:64] FLAG: --version="false"
Oct 31 17:26:38.171013 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.162453    2578 flags.go:64] FLAG: --vmodule=""
Oct 31 17:26:38.171013 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.162457    2578 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Oct 31 17:26:38.171013 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.162461    2578 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Oct 31 17:26:38.171013 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.162591    2578 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 31 17:26:38.171013 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.162595    2578 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 31 17:26:38.171013 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.162598    2578 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 31 17:26:38.171013 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.162601    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 31 17:26:38.171013 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.162604    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 31 17:26:38.171013 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.162607    2578 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 31 17:26:38.171013 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.162610    2578 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 31 17:26:38.171013 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.162612    2578 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 31 17:26:38.171013 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.162615    2578 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 31 17:26:38.171013 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.162617    2578 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 31 17:26:38.171013 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.162620    2578 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 31 17:26:38.171013 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.162623    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 31 17:26:38.171013 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.162626    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 31 17:26:38.171830 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.162629    2578 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 31 17:26:38.171830 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.162631    2578 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 31 17:26:38.171830 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.162634    2578 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 31 17:26:38.171830 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.162637    2578 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 31 17:26:38.171830 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.162640    2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 31 17:26:38.171830 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.162643    2578 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 31 17:26:38.171830 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.162646    2578 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 31 17:26:38.171830 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.162649    2578 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 31 17:26:38.171830 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.162651    2578 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 31 17:26:38.171830 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.162654    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 31 17:26:38.171830 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.162656    2578 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 31 17:26:38.171830 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.162659    2578 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 31 17:26:38.171830 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.162662    2578 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 31 17:26:38.171830 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.162664    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 31 17:26:38.171830 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.162668    2578 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 31 17:26:38.171830 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.162670    2578 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 31 17:26:38.171830 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.162673    2578 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 31 17:26:38.171830 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.162675    2578 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 31 17:26:38.171830 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.162678    2578 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 31 17:26:38.171830 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.162680    2578 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 31 17:26:38.172335 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.162683    2578 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 31 17:26:38.172335 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.162686    2578 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 31 17:26:38.172335 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.162688    2578 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 31 17:26:38.172335 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.162691    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 31 17:26:38.172335 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.162693    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 31 17:26:38.172335 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.162696    2578 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 31 17:26:38.172335 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.162698    2578 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 31 17:26:38.172335 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.162701    2578 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 31 17:26:38.172335 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.162703    2578 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 31 17:26:38.172335 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.162707    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 31 17:26:38.172335 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.162710    2578 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 31 17:26:38.172335 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.162712    2578 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 31 17:26:38.172335 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.162715    2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 31 17:26:38.172335 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.162718    2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 31 17:26:38.172335 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.162720    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 31 17:26:38.172335 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.162722    2578 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 31 17:26:38.172335 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.162725    2578 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 31 17:26:38.172335 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.162728    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 31 17:26:38.172335 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.162731    2578 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 31 17:26:38.172335 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.162734    2578 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 31 17:26:38.172840 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.162736    2578 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 31 17:26:38.172840 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.162739    2578 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 31 17:26:38.172840 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.162741    2578 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 31 17:26:38.172840 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.162744    2578 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 31 17:26:38.172840 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.162746    2578 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 31 17:26:38.172840 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.162749    2578 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 31 17:26:38.172840 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.162751    2578 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 31 17:26:38.172840 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.162754    2578 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 31 17:26:38.172840 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.162757    2578 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 31 17:26:38.172840 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.162761    2578 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 31 17:26:38.172840 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.162764    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 31 17:26:38.172840 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.162767    2578 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 31 17:26:38.172840 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.162770    2578 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 31 17:26:38.172840 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.162773    2578 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 31 17:26:38.172840 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.162775    2578 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 31 17:26:38.172840 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.162778    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 31 17:26:38.172840 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.162780    2578 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 31 17:26:38.172840 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.162783    2578 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 31 17:26:38.172840 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.162785    2578 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 31 17:26:38.173312 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.162788    2578 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 31 17:26:38.173312 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.162790    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 31 17:26:38.173312 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.162795    2578 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 31 17:26:38.173312 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.162799    2578 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 31 17:26:38.173312 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.162802    2578 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 31 17:26:38.173312 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.162804    2578 feature_gate.go:386] unrecognized feature gate: Example
Oct 31 17:26:38.173312 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.162807    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 31 17:26:38.173312 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.162810    2578 feature_gate.go:386] unrecognized feature gate: Example2
Oct 31 17:26:38.173312 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.162812    2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 31 17:26:38.173312 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.162815    2578 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 31 17:26:38.173312 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.162817    2578 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 31 17:26:38.173312 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.162820    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 31 17:26:38.173312 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.162823    2578 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 31 17:26:38.173312 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.162826    2578 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 31 17:26:38.173312 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.162829    2578 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 31 17:26:38.173312 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.162831    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 31 17:26:38.173312 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.162834    2578 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 31 17:26:38.173312 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.162837    2578 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 31 17:26:38.173312 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.162839    2578 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 31 17:26:38.173882 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.162842    2578 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 31 17:26:38.173882 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.162844    2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 31 17:26:38.173882 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.162847    2578 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 31 17:26:38.173882 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.162850    2578 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 31 17:26:38.173882 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.162852    2578 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 31 17:26:38.173882 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.162855    2578 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 31 17:26:38.173882 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.162857    2578 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 31 17:26:38.173882 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.162860    2578 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 31 17:26:38.173882 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.162862    2578 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 31 17:26:38.173882 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.162865    2578 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 31 17:26:38.173882 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.162867    2578 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 31 17:26:38.173882 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.162870    2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 31 17:26:38.173882 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.162873    2578 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 31 17:26:38.173882 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.162875    2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 31 17:26:38.173882 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.162878    2578 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 31 17:26:38.173882 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.162906    2578 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 31 17:26:38.174254 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.162958    2578 feature_gate.go:386] unrecognized feature gate: Example
Oct 31 17:26:38.174254 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.162963    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 31 17:26:38.174254 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.162966    2578 feature_gate.go:386] unrecognized feature gate: Example2
Oct 31 17:26:38.174254 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.162969    2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 31 17:26:38.174254 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.162972    2578 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 31 17:26:38.174254 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.162974    2578 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 31 17:26:38.174254 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.162977    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 31 17:26:38.174254 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.162980    2578 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 31 17:26:38.174254 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.162982    2578 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 31 17:26:38.174254 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.162985    2578 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 31 17:26:38.174254 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.162988    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 31 17:26:38.174254 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.162991    2578 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 31 17:26:38.174254 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.162994    2578 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 31 17:26:38.174254 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.162996    2578 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 31 17:26:38.174254 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.162999    2578 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 31 17:26:38.174254 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.163002    2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 31 17:26:38.174254 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.163004    2578 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 31 17:26:38.174254 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.163007    2578 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 31 17:26:38.174254 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.163009    2578 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 31 17:26:38.174709 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.163012    2578 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 31 17:26:38.174709 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.163014    2578 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 31 17:26:38.174709 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.163017    2578 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 31 17:26:38.174709 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.163019    2578 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 31 17:26:38.174709 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.163022    2578 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 31 17:26:38.174709 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.163024    2578 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 31 17:26:38.174709 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.163027    2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 31 17:26:38.174709 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.163030    2578 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 31 17:26:38.174709 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.163032    2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 31 17:26:38.174709 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.163035    2578 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 31 17:26:38.174709 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.163037    2578 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 31 17:26:38.174709 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.163040    2578 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 31 17:26:38.174709 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.163042    2578 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 31 17:26:38.174709 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.163046    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 31 17:26:38.174709 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.163048    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 31 17:26:38.174709 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.163051    2578 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 31 17:26:38.174709 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.163054    2578 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 31 17:26:38.174709 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.163056    2578 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 31 17:26:38.174709 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.163059    2578 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 31 17:26:38.174709 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.163061    2578 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 31 17:26:38.175184 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.163064    2578 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 31 17:26:38.175184 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.163069    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 31 17:26:38.175184 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.163071    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 31 17:26:38.175184 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.163074    2578 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 31 17:26:38.175184 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.163076    2578 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 31 17:26:38.175184 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.163079    2578 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 31 17:26:38.175184 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.163082    2578 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 31 17:26:38.175184 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.163085    2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 31 17:26:38.175184 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.163088    2578 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 31 17:26:38.175184 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.163091    2578 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 31 17:26:38.175184 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.163093    2578 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 31 17:26:38.175184 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.163096    2578 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 31 17:26:38.175184 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.163098    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 31 17:26:38.175184 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.163101    2578 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 31 17:26:38.175184 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.163104    2578 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 31 17:26:38.175184 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.163106    2578 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 31 17:26:38.175184 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.163108    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 31 17:26:38.175184 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.163111    2578 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 31 17:26:38.175184 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.163113    2578 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 31 17:26:38.175184 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.163116    2578 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 31 17:26:38.175654 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.163118    2578 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 31 17:26:38.175654 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.163121    2578 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 31 17:26:38.175654 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.163123    2578 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 31 17:26:38.175654 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.163126    2578 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 31 17:26:38.175654 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.163129    2578 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 31 17:26:38.175654 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.163133    2578 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 31 17:26:38.175654 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.163135    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 31 17:26:38.175654 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.163138    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 31 17:26:38.175654 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.163140    2578 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 31 17:26:38.175654 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.163143    2578 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 31 17:26:38.175654 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.163146    2578 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 31 17:26:38.175654 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.163148    2578 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 31 17:26:38.175654 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.163151    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 31 17:26:38.175654 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.163155    2578 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 31 17:26:38.175654 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.163157    2578 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 31 17:26:38.175654 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.163160    2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 31 17:26:38.175654 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.163162    2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 31 17:26:38.175654 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.163165    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 31 17:26:38.175654 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.163168    2578 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 31 17:26:38.175654 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.163171    2578 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 31 17:26:38.176149 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.163174    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 31 17:26:38.176149 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.163176    2578 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 31 17:26:38.176149 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.163179    2578 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 31 17:26:38.176149 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.163181    2578 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 31 17:26:38.176149 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.163183    2578 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 31 17:26:38.176149 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.163186    2578 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 31 17:26:38.176149 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.163188    2578 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 31 17:26:38.176149 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.163191    2578 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 31 17:26:38.176149 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.163193    2578 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 31 17:26:38.176149 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.163196    2578 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 31 17:26:38.176149 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.163198    2578 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 31 17:26:38.176149 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.163202    2578 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 31 17:26:38.176149 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.163205    2578 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 31 17:26:38.176149 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.163208    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 31 17:26:38.176149 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.163211    2578 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 31 17:26:38.176149 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.163214    2578 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 31 17:26:38.176149 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.163217    2578 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 31 17:26:38.176149 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.163220    2578 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 31 17:26:38.176149 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.163223    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 31 17:26:38.176605 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.163225    2578 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 31 17:26:38.176605 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.163228    2578 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 31 17:26:38.176605 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.163230    2578 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 31 17:26:38.176605 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.163233    2578 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 31 17:26:38.176605 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.163236    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 31 17:26:38.176605 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.163240    2578 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 31 17:26:38.176605 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.163245    2578 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 31 17:26:38.176605 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.163248    2578 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 31 17:26:38.176605 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.172589    2578 server.go:525] "Kubelet version" kubeletVersion="v1.35.9"
Oct 31 17:26:38.176605 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.172602    2578 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Oct 31 17:26:38.176605 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.172620    2578 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Oct 31 17:26:38.176605 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.172625    2578 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Oct 31 17:26:38.176605 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.172682    2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 31 17:26:38.176605 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.172687    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 31 17:26:38.176605 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.172691    2578 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 31 17:26:38.176605 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.172694    2578 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 31 17:26:38.176605 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.172697    2578 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 31 17:26:38.176605 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.172699    2578 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 31 17:26:38.176605 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.172702    2578 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 31 17:26:38.177078 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.172705    2578 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 31 17:26:38.177078 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.172708    2578 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 31 17:26:38.177078 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.172711    2578 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 31 17:26:38.177078 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.172713    2578 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 31 17:26:38.177078 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.172716    2578 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 31 17:26:38.177078 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.172719    2578 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 31 17:26:38.177078 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.172733    2578 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 31 17:26:38.177078 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.172736    2578 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 31 17:26:38.177078 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.172742    2578 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 31 17:26:38.177078 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.172745    2578 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 31 17:26:38.177078 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.172748    2578 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 31 17:26:38.177078 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.172750    2578 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 31 17:26:38.177078 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.172754    2578 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 31 17:26:38.177078 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.172756    2578 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 31 17:26:38.177078 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.172759    2578 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 31 17:26:38.177078 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.172763    2578 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 31 17:26:38.177078 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.172767    2578 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 31 17:26:38.177078 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.172770    2578 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 31 17:26:38.177078 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.172773    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 31 17:26:38.177584 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.172776    2578 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 31 17:26:38.177584 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.172779    2578 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 31 17:26:38.177584 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.172782    2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 31 17:26:38.177584 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.172785    2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 31 17:26:38.177584 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.172787    2578 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 31 17:26:38.177584 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.172790    2578 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 31 17:26:38.177584 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.172793    2578 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 31 17:26:38.177584 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.172796    2578 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 31 17:26:38.177584 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.172798    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 31 17:26:38.177584 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.172801    2578 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 31 17:26:38.177584 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.172803    2578 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 31 17:26:38.177584 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.172806    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 31 17:26:38.177584 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.172809    2578 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 31 17:26:38.177584 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.172811    2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 31 17:26:38.177584 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.172814    2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 31 17:26:38.177584 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.172816    2578 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 31 17:26:38.177584 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.172819    2578 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 31 17:26:38.177584 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.172822    2578 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 31 17:26:38.177584 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.172824    2578 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 31 17:26:38.178038 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.172827    2578 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 31 17:26:38.178038 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.172829    2578 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 31 17:26:38.178038 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.172832    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 31 17:26:38.178038 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.172835    2578 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 31 17:26:38.178038 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.172837    2578 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 31 17:26:38.178038 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.172840    2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 31 17:26:38.178038 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.172842    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 31 17:26:38.178038 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.172845    2578 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 31 17:26:38.178038 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.172848    2578 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 31 17:26:38.178038 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.172851    2578 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 31 17:26:38.178038 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.172854    2578 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 31 17:26:38.178038 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.172857    2578 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 31 17:26:38.178038 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.172859    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 31 17:26:38.178038 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.172863    2578 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 31 17:26:38.178038 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.172868    2578 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 31 17:26:38.178038 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.172871    2578 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 31 17:26:38.178038 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.172874    2578 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 31 17:26:38.178038 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.172876    2578 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 31 17:26:38.178038 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.172879    2578 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 31 17:26:38.178499 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.172881    2578 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 31 17:26:38.178499 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.172884    2578 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 31 17:26:38.178499 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.172887    2578 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 31 17:26:38.178499 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.172889    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 31 17:26:38.178499 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.172892    2578 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 31 17:26:38.178499 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.172894    2578 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 31 17:26:38.178499 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.172897    2578 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 31 17:26:38.178499 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.172900    2578 feature_gate.go:386] unrecognized feature gate: Example2
Oct 31 17:26:38.178499 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.172902    2578 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 31 17:26:38.178499 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.172905    2578 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 31 17:26:38.178499 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.172907    2578 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 31 17:26:38.178499 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.172910    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 31 17:26:38.178499 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.172913    2578 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 31 17:26:38.178499 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.172915    2578 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 31 17:26:38.178499 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.172917    2578 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 31 17:26:38.178499 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.172920    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 31 17:26:38.178499 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.172923    2578 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 31 17:26:38.178499 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.172926    2578 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 31 17:26:38.178499 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.172928    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 31 17:26:38.178499 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.172931    2578 feature_gate.go:386] unrecognized feature gate: Example
Oct 31 17:26:38.178967 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.172933    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 31 17:26:38.178967 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.172936    2578 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 31 17:26:38.178967 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.172938    2578 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 31 17:26:38.178967 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.172941    2578 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 31 17:26:38.178967 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.172943    2578 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 31 17:26:38.178967 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.172946    2578 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 31 17:26:38.178967 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.172949    2578 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 31 17:26:38.178967 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.172952    2578 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 31 17:26:38.178967 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.172954    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 31 17:26:38.178967 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.172957    2578 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 31 17:26:38.178967 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.172959    2578 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 31 17:26:38.178967 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.172962    2578 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 31 17:26:38.178967 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.172965    2578 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 31 17:26:38.178967 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.172967    2578 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 31 17:26:38.178967 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.172970    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 31 17:26:38.178967 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.172980    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 31 17:26:38.178967 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.172984    2578 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 31 17:26:38.178967 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.172986    2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 31 17:26:38.178967 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.172989    2578 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 31 17:26:38.179405 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.172992    2578 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 31 17:26:38.179405 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.172994    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 31 17:26:38.179405 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.172997    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 31 17:26:38.179405 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.173025    2578 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 31 17:26:38.179405 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173172    2578 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 31 17:26:38.179405 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173178    2578 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 31 17:26:38.179405 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173181    2578 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 31 17:26:38.179405 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173183    2578 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 31 17:26:38.179405 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173186    2578 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 31 17:26:38.179405 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173189    2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 31 17:26:38.179405 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173192    2578 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 31 17:26:38.179405 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173195    2578 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 31 17:26:38.179405 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173197    2578 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 31 17:26:38.179405 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173200    2578 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 31 17:26:38.179405 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173203    2578 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 31 17:26:38.179405 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173205    2578 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 31 17:26:38.179825 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173208    2578 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 31 17:26:38.179825 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173210    2578 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 31 17:26:38.179825 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173213    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 31 17:26:38.179825 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173215    2578 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 31 17:26:38.179825 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173218    2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 31 17:26:38.179825 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173221    2578 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 31 17:26:38.179825 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173224    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 31 17:26:38.179825 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173226    2578 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 31 17:26:38.179825 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173229    2578 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 31 17:26:38.179825 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173231    2578 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 31 17:26:38.179825 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173234    2578 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 31 17:26:38.179825 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173237    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 31 17:26:38.179825 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173239    2578 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 31 17:26:38.179825 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173242    2578 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 31 17:26:38.179825 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173244    2578 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 31 17:26:38.179825 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173247    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 31 17:26:38.179825 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173249    2578 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 31 17:26:38.179825 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173252    2578 feature_gate.go:386] unrecognized feature gate: Example
Oct 31 17:26:38.179825 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173254    2578 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 31 17:26:38.180274 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173257    2578 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 31 17:26:38.180274 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173259    2578 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 31 17:26:38.180274 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173261    2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 31 17:26:38.180274 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173264    2578 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 31 17:26:38.180274 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173266    2578 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 31 17:26:38.180274 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173269    2578 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 31 17:26:38.180274 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173272    2578 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 31 17:26:38.180274 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173274    2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 31 17:26:38.180274 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173278    2578 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 31 17:26:38.180274 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173283    2578 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 31 17:26:38.180274 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173286    2578 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 31 17:26:38.180274 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173289    2578 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 31 17:26:38.180274 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173292    2578 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 31 17:26:38.180274 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173294    2578 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 31 17:26:38.180274 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173297    2578 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 31 17:26:38.180274 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173299    2578 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 31 17:26:38.180274 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173302    2578 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 31 17:26:38.180274 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173304    2578 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 31 17:26:38.180274 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173307    2578 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 31 17:26:38.180730 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173309    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 31 17:26:38.180730 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173312    2578 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 31 17:26:38.180730 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173315    2578 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 31 17:26:38.180730 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173317    2578 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 31 17:26:38.180730 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173320    2578 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 31 17:26:38.180730 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173323    2578 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 31 17:26:38.180730 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173325    2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 31 17:26:38.180730 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173328    2578 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 31 17:26:38.180730 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173330    2578 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 31 17:26:38.180730 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173333    2578 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 31 17:26:38.180730 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173335    2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 31 17:26:38.180730 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173338    2578 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 31 17:26:38.180730 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173340    2578 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 31 17:26:38.180730 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173343    2578 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 31 17:26:38.180730 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173345    2578 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 31 17:26:38.180730 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173348    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 31 17:26:38.180730 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173350    2578 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 31 17:26:38.180730 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173353    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 31 17:26:38.180730 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173355    2578 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 31 17:26:38.180730 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173358    2578 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 31 17:26:38.181202 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173361    2578 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 31 17:26:38.181202 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173366    2578 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 31 17:26:38.181202 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173369    2578 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 31 17:26:38.181202 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173372    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 31 17:26:38.181202 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173375    2578 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 31 17:26:38.181202 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173377    2578 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 31 17:26:38.181202 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173380    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 31 17:26:38.181202 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173382    2578 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 31 17:26:38.181202 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173385    2578 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 31 17:26:38.181202 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173388    2578 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 31 17:26:38.181202 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173390    2578 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 31 17:26:38.181202 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173393    2578 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 31 17:26:38.181202 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173395    2578 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 31 17:26:38.181202 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173398    2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 31 17:26:38.181202 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173401    2578 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 31 17:26:38.181202 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173403    2578 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 31 17:26:38.181202 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173406    2578 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 31 17:26:38.181202 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173408    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 31 17:26:38.181626 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173411    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 31 17:26:38.181626 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173413    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 31 17:26:38.181626 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173416    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 31 17:26:38.181626 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173418    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 31 17:26:38.181626 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173421    2578 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 31 17:26:38.181626 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173423    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 31 17:26:38.181626 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173426    2578 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 31 17:26:38.181626 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173428    2578 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 31 17:26:38.181626 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173431    2578 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 31 17:26:38.181626 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173433    2578 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 31 17:26:38.181626 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173436    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 31 17:26:38.181626 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173439    2578 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 31 17:26:38.181626 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173456    2578 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 31 17:26:38.181626 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173459    2578 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 31 17:26:38.181626 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173461    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 31 17:26:38.181626 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173464    2578 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 31 17:26:38.181626 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173468    2578 feature_gate.go:386] unrecognized feature gate: Example2
Oct 31 17:26:38.181626 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173471    2578 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 31 17:26:38.182042 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.173495    2578 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 31 17:26:38.182042 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173527    2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 31 17:26:38.182042 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173531    2578 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 31 17:26:38.182042 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173533    2578 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 31 17:26:38.182042 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173536    2578 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 31 17:26:38.182042 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173538    2578 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 31 17:26:38.182042 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173541    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 31 17:26:38.182042 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173544    2578 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 31 17:26:38.182042 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173546    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 31 17:26:38.182042 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173549    2578 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 31 17:26:38.182042 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173551    2578 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 31 17:26:38.182042 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173555    2578 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 31 17:26:38.182042 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173559    2578 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 31 17:26:38.182042 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173562    2578 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 31 17:26:38.182042 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173565    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 31 17:26:38.182402 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173568    2578 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 31 17:26:38.182402 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173571    2578 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 31 17:26:38.182402 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173573    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 31 17:26:38.182402 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173576    2578 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 31 17:26:38.182402 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173580    2578 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 31 17:26:38.182402 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173582    2578 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 31 17:26:38.182402 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173585    2578 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 31 17:26:38.182402 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173587    2578 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 31 17:26:38.182402 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173589    2578 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 31 17:26:38.182402 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173592    2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 31 17:26:38.182402 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173595    2578 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 31 17:26:38.182402 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173597    2578 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 31 17:26:38.182402 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173600    2578 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 31 17:26:38.182402 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173602    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 31 17:26:38.182402 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173605    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 31 17:26:38.182402 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173607    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 31 17:26:38.182402 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173610    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 31 17:26:38.182402 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173613    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 31 17:26:38.182402 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173616    2578 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 31 17:26:38.182876 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173618    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 31 17:26:38.182876 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173621    2578 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 31 17:26:38.182876 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173624    2578 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 31 17:26:38.182876 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173626    2578 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 31 17:26:38.182876 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173629    2578 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 31 17:26:38.182876 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173631    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 31 17:26:38.182876 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173634    2578 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 31 17:26:38.182876 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173636    2578 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 31 17:26:38.182876 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173639    2578 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 31 17:26:38.182876 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173641    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 31 17:26:38.182876 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173643    2578 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 31 17:26:38.182876 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173646    2578 feature_gate.go:386] unrecognized feature gate: Example2
Oct 31 17:26:38.182876 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173649    2578 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 31 17:26:38.182876 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173651    2578 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 31 17:26:38.182876 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173653    2578 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 31 17:26:38.182876 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173656    2578 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 31 17:26:38.182876 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173658    2578 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 31 17:26:38.182876 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173661    2578 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 31 17:26:38.182876 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173663    2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 31 17:26:38.182876 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173666    2578 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 31 17:26:38.183342 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173669    2578 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 31 17:26:38.183342 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173672    2578 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 31 17:26:38.183342 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173675    2578 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 31 17:26:38.183342 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173677    2578 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 31 17:26:38.183342 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173679    2578 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 31 17:26:38.183342 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173682    2578 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 31 17:26:38.183342 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173684    2578 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 31 17:26:38.183342 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173687    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 31 17:26:38.183342 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173690    2578 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 31 17:26:38.183342 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173692    2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 31 17:26:38.183342 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173696    2578 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 31 17:26:38.183342 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173699    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 31 17:26:38.183342 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173701    2578 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 31 17:26:38.183342 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173704    2578 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 31 17:26:38.183342 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173706    2578 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 31 17:26:38.183342 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173708    2578 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 31 17:26:38.183342 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173711    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 31 17:26:38.183342 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173713    2578 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 31 17:26:38.183342 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173715    2578 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 31 17:26:38.183342 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173718    2578 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 31 17:26:38.183821 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173720    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 31 17:26:38.183821 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173723    2578 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 31 17:26:38.183821 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173725    2578 feature_gate.go:386] unrecognized feature gate: Example
Oct 31 17:26:38.183821 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173728    2578 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 31 17:26:38.183821 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173730    2578 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 31 17:26:38.183821 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173735    2578 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 31 17:26:38.183821 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173738    2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 31 17:26:38.183821 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173741    2578 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 31 17:26:38.183821 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173743    2578 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 31 17:26:38.183821 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173746    2578 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 31 17:26:38.183821 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173748    2578 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 31 17:26:38.183821 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173751    2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 31 17:26:38.183821 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173755    2578 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 31 17:26:38.183821 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173759    2578 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 31 17:26:38.183821 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173762    2578 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 31 17:26:38.183821 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173765    2578 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 31 17:26:38.183821 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173768    2578 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 31 17:26:38.183821 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173771    2578 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 31 17:26:38.183821 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173774    2578 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 31 17:26:38.184262 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173776    2578 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 31 17:26:38.184262 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173779    2578 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 31 17:26:38.184262 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173782    2578 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 31 17:26:38.184262 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173785    2578 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 31 17:26:38.184262 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173788    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 31 17:26:38.184262 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173791    2578 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 31 17:26:38.184262 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173793    2578 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 31 17:26:38.184262 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173796    2578 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 31 17:26:38.184262 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173798    2578 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 31 17:26:38.184262 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173801    2578 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 31 17:26:38.184262 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173803    2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 31 17:26:38.184262 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173806    2578 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 31 17:26:38.184262 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173808    2578 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 31 17:26:38.184262 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:38.173811    2578 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 31 17:26:38.184262 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.173861    2578 server.go:957] "Client rotation is on, will bootstrap in background"
Oct 31 17:26:38.184262 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.180481    2578 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Oct 31 17:26:38.184262 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.181552    2578 server.go:1014] "Starting client certificate rotation"
Oct 31 17:26:38.184262 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.181651    2578 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Oct 31 17:26:38.184262 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.181680    2578 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Oct 31 17:26:38.184714 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.183775    2578 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 31 17:26:38.185220 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.184986    2578 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 31 17:26:38.194638 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.194623    2578 log.go:25] "Validated CRI v1 runtime API"
Oct 31 17:26:38.198524 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.198509    2578 log.go:25] "Validated CRI v1 image API"
Oct 31 17:26:38.199356 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.199331    2578 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Oct 31 17:26:38.203136 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.203108    2578 fs.go:135] Filesystem UUIDs: map[3b3f032c-14aa-4995-a2c0-3ee30a204a2e:/dev/nvme0n1p4 502e58b0-16c8-4f02-bcf2-2c4ee8f17d33:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2]
Oct 31 17:26:38.203192 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.203136    2578 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0}]
Oct 31 17:26:38.208526 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.208381    2578 manager.go:217] Machine: {Timestamp:2026-10-31 17:26:38.207902515 +0000 UTC m=+0.373420892 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099066 MemoryCapacity:33164333056 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec22644a1a04d4d435bd958c94ea5c24 SystemUUID:ec22644a-1a04-d4d4-35bd-958c94ea5c24 BootID:c6e4931c-99a2-49b9-abc4-7c8389a6675d Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16582164480 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16582168576 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:ed:5c:21:57 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:ed:5c:21:57 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:be:bf:2f:5b:05:54 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164333056 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Oct 31 17:26:38.208526 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.208520    2578 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Oct 31 17:26:38.208655 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.208623    2578 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Oct 31 17:26:38.212135 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.212115    2578 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Oct 31 17:26:38.212287 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.212136    2578 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-129-127.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Oct 31 17:26:38.212337 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.212290    2578 topology_manager.go:143] "Creating topology manager with none policy"
Oct 31 17:26:38.212337 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.212298    2578 container_manager_linux.go:311] "Creating device plugin manager"
Oct 31 17:26:38.212337 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.212319    2578 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 31 17:26:38.212337 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.212335    2578 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 31 17:26:38.213617 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.213607    2578 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager"
Oct 31 17:26:38.218366 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.218354    2578 state_mem.go:41] "Initialized" logger="CPUManager state memory"
Oct 31 17:26:38.218513 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.218503    2578 server.go:1235] "Using root directory" path="/var/lib/kubelet"
Oct 31 17:26:38.219351 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.219341    2578 kubelet.go:493] "Attempting to sync node with API server"
Oct 31 17:26:38.219407 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.219361    2578 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests"
Oct 31 17:26:38.219407 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.219374    2578 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Oct 31 17:26:38.219407 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.219386    2578 kubelet.go:405] "Adding apiserver pod source"
Oct 31 17:26:38.219407 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.219397    2578 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Oct 31 17:26:38.219803 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.219790    2578 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 31 17:26:38.223532 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.223422    2578 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1"
Oct 31 17:26:38.230681 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.230665    2578 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 31 17:26:38.231709 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.231693    2578 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Oct 31 17:26:38.231777 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.231735    2578 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Oct 31 17:26:38.231929 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.231916    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Oct 31 17:26:38.231986 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.231935    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Oct 31 17:26:38.231986 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.231944    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Oct 31 17:26:38.231986 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.231953    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Oct 31 17:26:38.231986 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.231963    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Oct 31 17:26:38.231986 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.231972    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Oct 31 17:26:38.232604 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.232593    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Oct 31 17:26:38.232664 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.232606    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Oct 31 17:26:38.232664 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.232617    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Oct 31 17:26:38.232664 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.232627    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Oct 31 17:26:38.232664 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.232641    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Oct 31 17:26:38.232664 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.232654    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Oct 31 17:26:38.233505 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.233493    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Oct 31 17:26:38.233566 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.233509    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Oct 31 17:26:38.237309 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.237294    2578 server.go:1263] "Started kubelet"
Oct 31 17:26:38.237482 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.237390    2578 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Oct 31 17:26:38.237543 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.237506    2578 server_v1.go:49] "podresources" method="list" useActivePods=true
Oct 31 17:26:38.237791 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.237364    2578 server.go:182] "Starting to listen" address="0.0.0.0" port=10250
Oct 31 17:26:38.237967 ip-10-0-129-127 systemd[1]: Started Kubernetes Kubelet.
Oct 31 17:26:38.239032 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.239003    2578 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Oct 31 17:26:38.239899 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.239883    2578 server.go:317] "Adding debug handlers to kubelet server"
Oct 31 17:26:38.243857 ip-10-0-129-127 kubenswrapper[2578]: E1031 17:26:38.243833    2578 kubelet.go:1679] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Oct 31 17:26:38.246327 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.246311    2578 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Oct 31 17:26:38.247009 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.246993    2578 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer"
Oct 31 17:26:38.247355 ip-10-0-129-127 kubenswrapper[2578]: E1031 17:26:38.247315    2578 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-129-127.ec2.internal\" not found"
Oct 31 17:26:38.247872 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.247854    2578 volume_manager.go:309] "The desired_state_of_world populator starts"
Oct 31 17:26:38.247956 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.247874    2578 volume_manager.go:311] "Starting Kubelet Volume Manager"
Oct 31 17:26:38.247956 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.247855    2578 factory.go:55] Registering systemd factory
Oct 31 17:26:38.247956 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.247898    2578 factory.go:223] Registration of the systemd container factory successfully
Oct 31 17:26:38.247956 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.247855    2578 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Oct 31 17:26:38.248139 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.247978    2578 reconstruct.go:97] "Volume reconstruction finished"
Oct 31 17:26:38.248139 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.247988    2578 reconciler.go:29] "Reconciler: start to sync state"
Oct 31 17:26:38.248139 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.248094    2578 factory.go:153] Registering CRI-O factory
Oct 31 17:26:38.248139 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.248110    2578 factory.go:223] Registration of the crio container factory successfully
Oct 31 17:26:38.248298 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.248158    2578 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Oct 31 17:26:38.248298 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.248176    2578 factory.go:103] Registering Raw factory
Oct 31 17:26:38.248298 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.248190    2578 manager.go:1199] Started watching for new ooms in manager
Oct 31 17:26:38.249239 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.249222    2578 manager.go:319] Starting recovery of all containers
Oct 31 17:26:38.260333 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.260318    2578 manager.go:324] Recovery completed
Oct 31 17:26:38.264365 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.264349    2578 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 31 17:26:38.266158 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.266140    2578 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-127.ec2.internal" event="NodeHasSufficientMemory"
Oct 31 17:26:38.266239 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.266171    2578 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-127.ec2.internal" event="NodeHasNoDiskPressure"
Oct 31 17:26:38.266239 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.266188    2578 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-127.ec2.internal" event="NodeHasSufficientPID"
Oct 31 17:26:38.266819 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.266807    2578 cpu_manager.go:226] "Starting" policy="none"
Oct 31 17:26:38.266819 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.266817    2578 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 31 17:26:38.266900 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.266817    2578 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s"
Oct 31 17:26:38.266900 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.266886    2578 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory"
Oct 31 17:26:38.267434 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.267419    2578 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-127.ec2.internal" event="NodeHasSufficientMemory"
Oct 31 17:26:38.267504 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.267457    2578 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-127.ec2.internal" event="NodeHasNoDiskPressure"
Oct 31 17:26:38.267504 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.267473    2578 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-127.ec2.internal" event="NodeHasSufficientPID"
Oct 31 17:26:38.269130 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.269118    2578 policy_none.go:50] "Start"
Oct 31 17:26:38.269167 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.269134    2578 memory_manager.go:187] "Starting memorymanager" policy="None"
Oct 31 17:26:38.269167 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.269143    2578 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Oct 31 17:26:38.269471 ip-10-0-129-127 kubenswrapper[2578]: E1031 17:26:38.269415    2578 controller.go:201] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-129-127.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Oct 31 17:26:38.269471 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.269416    2578 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-129-127.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Oct 31 17:26:38.270559 ip-10-0-129-127 kubenswrapper[2578]: E1031 17:26:38.269421    2578 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-127.ec2.internal.18e3ad47451afbf4  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-127.ec2.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-129-127.ec2.internal,},FirstTimestamp:2026-10-31 17:26:38.23726898 +0000 UTC m=+0.402787356,LastTimestamp:2026-10-31 17:26:38.23726898 +0000 UTC m=+0.402787356,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-127.ec2.internal,}"
Oct 31 17:26:38.271407 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.271391    2578 policy_none.go:44] "Start"
Oct 31 17:26:38.279483 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.279466    2578 reflector.go:446] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Oct 31 17:26:38.279556 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.279523    2578 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-2zqzg"
Oct 31 17:26:38.283288 ip-10-0-129-127 kubenswrapper[2578]: E1031 17:26:38.283198    2578 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-127.ec2.internal.18e3ad4746d3cb05  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-127.ec2.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-129-127.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-129-127.ec2.internal,},FirstTimestamp:2026-10-31 17:26:38.266157829 +0000 UTC m=+0.431676207,LastTimestamp:2026-10-31 17:26:38.266157829 +0000 UTC m=+0.431676207,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-127.ec2.internal,}"
Oct 31 17:26:38.283986 ip-10-0-129-127 kubenswrapper[2578]: E1031 17:26:38.283958    2578 reflector.go:204] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Service"
Oct 31 17:26:38.283986 ip-10-0-129-127 kubenswrapper[2578]: E1031 17:26:38.283959    2578 reflector.go:204] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.CSIDriver"
Oct 31 17:26:38.303095 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.303063    2578 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"}
Oct 31 17:26:38.305503 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.305490    2578 manager.go:341] "Starting Device Plugin manager"
Oct 31 17:26:38.305591 ip-10-0-129-127 kubenswrapper[2578]: E1031 17:26:38.305537    2578 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Oct 31 17:26:38.305591 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.305549    2578 server.go:87] "Starting device plugin registration server"
Oct 31 17:26:38.305737 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.305726    2578 eviction_manager.go:194] "Eviction manager: starting control loop"
Oct 31 17:26:38.305789 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.305737    2578 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Oct 31 17:26:38.305862 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.305847    2578 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Oct 31 17:26:38.305932 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.305916    2578 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts"
Oct 31 17:26:38.305932 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.305928    2578 plugin_manager.go:121] "Starting Kubelet Plugin Manager"
Oct 31 17:26:38.306538 ip-10-0-129-127 kubenswrapper[2578]: E1031 17:26:38.306458    2578 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Oct 31 17:26:38.306538 ip-10-0-129-127 kubenswrapper[2578]: E1031 17:26:38.306507    2578 eviction_manager.go:297] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-129-127.ec2.internal\" not found"
Oct 31 17:26:38.313914 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.313898    2578 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 31 17:26:38.373838 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.373815    2578 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Oct 31 17:26:38.375778 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.374893    2578 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Oct 31 17:26:38.375778 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.374918    2578 status_manager.go:249] "Starting to sync pod status with apiserver"
Oct 31 17:26:38.375778 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.374943    2578 kubelet.go:2521] "Starting kubelet main sync loop"
Oct 31 17:26:38.375778 ip-10-0-129-127 kubenswrapper[2578]: E1031 17:26:38.374983    2578 kubelet.go:2545] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Oct 31 17:26:38.408557 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.408512    2578 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 31 17:26:38.409327 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.409308    2578 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-127.ec2.internal" event="NodeHasSufficientMemory"
Oct 31 17:26:38.409410 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.409340    2578 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-127.ec2.internal" event="NodeHasNoDiskPressure"
Oct 31 17:26:38.409410 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.409352    2578 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-127.ec2.internal" event="NodeHasSufficientPID"
Oct 31 17:26:38.409410 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.409377    2578 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-129-127.ec2.internal"
Oct 31 17:26:38.427250 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.427228    2578 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-129-127.ec2.internal"
Oct 31 17:26:38.430045 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.430026    2578 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 31 17:26:38.476094 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.476068    2578 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-127.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-129-127.ec2.internal"]
Oct 31 17:26:38.478499 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.478484    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-127.ec2.internal"
Oct 31 17:26:38.480599 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.480582    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-127.ec2.internal"
Oct 31 17:26:38.510032 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.510014    2578 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-127.ec2.internal"
Oct 31 17:26:38.512629 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.512611    2578 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-127.ec2.internal"
Oct 31 17:26:38.533790 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.533774    2578 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 31 17:26:38.533862 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.533844    2578 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 31 17:26:38.550185 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.550155    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/5136faca2de3de7a06b0d414d9c6fca7-config\") pod \"kube-apiserver-proxy-ip-10-0-129-127.ec2.internal\" (UID: \"5136faca2de3de7a06b0d414d9c6fca7\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-127.ec2.internal"
Oct 31 17:26:38.550260 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.550202    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7881d2520ea389377a82c2df6a5dd230-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-127.ec2.internal\" (UID: \"7881d2520ea389377a82c2df6a5dd230\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-127.ec2.internal"
Oct 31 17:26:38.550260 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.550231    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7881d2520ea389377a82c2df6a5dd230-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-127.ec2.internal\" (UID: \"7881d2520ea389377a82c2df6a5dd230\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-127.ec2.internal"
Oct 31 17:26:38.651343 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.651324    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7881d2520ea389377a82c2df6a5dd230-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-127.ec2.internal\" (UID: \"7881d2520ea389377a82c2df6a5dd230\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-127.ec2.internal"
Oct 31 17:26:38.651421 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.651352    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7881d2520ea389377a82c2df6a5dd230-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-127.ec2.internal\" (UID: \"7881d2520ea389377a82c2df6a5dd230\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-127.ec2.internal"
Oct 31 17:26:38.651421 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.651369    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/5136faca2de3de7a06b0d414d9c6fca7-config\") pod \"kube-apiserver-proxy-ip-10-0-129-127.ec2.internal\" (UID: \"5136faca2de3de7a06b0d414d9c6fca7\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-127.ec2.internal"
Oct 31 17:26:38.651509 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.651417    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7881d2520ea389377a82c2df6a5dd230-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-127.ec2.internal\" (UID: \"7881d2520ea389377a82c2df6a5dd230\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-127.ec2.internal"
Oct 31 17:26:38.651509 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.651468    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7881d2520ea389377a82c2df6a5dd230-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-127.ec2.internal\" (UID: \"7881d2520ea389377a82c2df6a5dd230\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-127.ec2.internal"
Oct 31 17:26:38.651509 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.651475    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/5136faca2de3de7a06b0d414d9c6fca7-config\") pod \"kube-apiserver-proxy-ip-10-0-129-127.ec2.internal\" (UID: \"5136faca2de3de7a06b0d414d9c6fca7\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-127.ec2.internal"
Oct 31 17:26:38.837042 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.837014    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-127.ec2.internal"
Oct 31 17:26:38.838069 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:38.838049    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-127.ec2.internal"
Oct 31 17:26:39.181791 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.181560    2578 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Oct 31 17:26:39.182300 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.181916    2578 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received"
Oct 31 17:26:39.220421 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.220394    2578 apiserver.go:52] "Watching apiserver"
Oct 31 17:26:39.246866 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.246839    2578 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Oct 31 17:26:39.251793 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.251771    2578 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 31 17:26:39.282674 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.282650    2578 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Oct 31 17:26:39.282957 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.282939    2578 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-multus/network-metrics-daemon-pnjdm","kube-system/global-pull-secret-syncer-kmkdh","openshift-image-registry/node-ca-qxnvq","openshift-network-diagnostics/network-check-target-tskzw","openshift-dns/node-resolver-7pfr4","openshift-network-operator/iptables-alerter-jzcgs","openshift-ovn-kubernetes/ovnkube-node-zpf4d","kube-system/konnectivity-agent-mnbmt","kube-system/kube-apiserver-proxy-ip-10-0-129-127.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2v874","openshift-cluster-node-tuning-operator/tuned-rvsdp","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-127.ec2.internal","openshift-multus/multus-additional-cni-plugins-fhgqw","openshift-multus/multus-rl4hl"]
Oct 31 17:26:39.285543 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.285524    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pnjdm"
Oct 31 17:26:39.285616 ip-10-0-129-127 kubenswrapper[2578]: E1031 17:26:39.285595    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pnjdm" podUID="138bfcc8-f199-4ffd-b5d2-8d83ade6199a"
Oct 31 17:26:39.285666 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.285494    2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-10-30 17:21:39 +0000 UTC" deadline="2028-04-10 15:40:38.63016604 +0000 UTC"
Oct 31 17:26:39.285666 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.285638    2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12646h13m59.34453522s"
Oct 31 17:26:39.287738 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.287722    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kmkdh"
Oct 31 17:26:39.287792 ip-10-0-129-127 kubenswrapper[2578]: E1031 17:26:39.287775    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kmkdh" podUID="6d7781d6-8f73-4f1d-bb9a-0212e2eb6e4b"
Oct 31 17:26:39.289890 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.289868    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-qxnvq"
Oct 31 17:26:39.292042 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.292022    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tskzw"
Oct 31 17:26:39.292135 ip-10-0-129-127 kubenswrapper[2578]: E1031 17:26:39.292087    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tskzw" podUID="fb9c203a-af67-4df4-88dc-74cf9eab3623"
Oct 31 17:26:39.294232 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.294213    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-7pfr4"
Oct 31 17:26:39.296425 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.296409    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-jzcgs"
Oct 31 17:26:39.298828 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.298810    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-zpf4d"
Oct 31 17:26:39.301145 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.301128    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-mnbmt"
Oct 31 17:26:39.303464 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.303437    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2v874"
Oct 31 17:26:39.306404 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.306387    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-rvsdp"
Oct 31 17:26:39.307065 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.307047    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Oct 31 17:26:39.307181 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.307163    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Oct 31 17:26:39.307236 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.307166    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Oct 31 17:26:39.307236 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.307222    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-zv4z8\""
Oct 31 17:26:39.308937 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.308913    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-fhgqw"
Oct 31 17:26:39.311150 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.311133    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Oct 31 17:26:39.311234 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.311207    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Oct 31 17:26:39.311349 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.311332    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-vvnlg\""
Oct 31 17:26:39.311396 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.311364    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-rl4hl"
Oct 31 17:26:39.313376 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.313358    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-xbzzg\""
Oct 31 17:26:39.313376 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.313369    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Oct 31 17:26:39.313515 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.313463    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Oct 31 17:26:39.313515 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.313472    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Oct 31 17:26:39.315612 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.315596    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Oct 31 17:26:39.316104 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.316087    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Oct 31 17:26:39.317732 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.317706    2578 reflector.go:446] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Oct 31 17:26:39.317732 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.317720    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Oct 31 17:26:39.317910 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.317749    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Oct 31 17:26:39.317910 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.317770    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Oct 31 17:26:39.317910 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.317780    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-8d7mp\""
Oct 31 17:26:39.317910 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.317786    2578 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-5xqbh"
Oct 31 17:26:39.317910 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.317882    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Oct 31 17:26:39.319310 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.319292    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-k6pvq\""
Oct 31 17:26:39.319394 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.319322    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Oct 31 17:26:39.319394 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.319366    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Oct 31 17:26:39.320199 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.320172    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Oct 31 17:26:39.320314 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.320300    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Oct 31 17:26:39.324375 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.324355    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Oct 31 17:26:39.324475 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.324422    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-9gvm6\""
Oct 31 17:26:39.324475 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.324441    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Oct 31 17:26:39.349712 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.349690    2578 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Oct 31 17:26:39.354482 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.354459    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/72516605-a894-4a62-84c7-834122a01fb8-system-cni-dir\") pod \"multus-additional-cni-plugins-fhgqw\" (UID: \"72516605-a894-4a62-84c7-834122a01fb8\") " pod="openshift-multus/multus-additional-cni-plugins-fhgqw"
Oct 31 17:26:39.354585 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.354484    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e-cni-binary-copy\") pod \"multus-rl4hl\" (UID: \"47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e\") " pod="openshift-multus/multus-rl4hl"
Oct 31 17:26:39.354585 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.354503    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/0f353547-2feb-42f4-aae4-b0f1a30efa34-socket-dir\") pod \"aws-ebs-csi-driver-node-2v874\" (UID: \"0f353547-2feb-42f4-aae4-b0f1a30efa34\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2v874"
Oct 31 17:26:39.354585 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.354565    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/422b9ca9-3e0e-4fe0-860d-7c7d24b179fa-lib-modules\") pod \"tuned-rvsdp\" (UID: \"422b9ca9-3e0e-4fe0-860d-7c7d24b179fa\") " pod="openshift-cluster-node-tuning-operator/tuned-rvsdp"
Oct 31 17:26:39.354760 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.354615    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/87e61dcd-75f6-4d3f-8ed9-7d35b06a9bb3-serviceca\") pod \"node-ca-qxnvq\" (UID: \"87e61dcd-75f6-4d3f-8ed9-7d35b06a9bb3\") " pod="openshift-image-registry/node-ca-qxnvq"
Oct 31 17:26:39.354760 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.354686    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vv9nf\" (UniqueName: \"kubernetes.io/projected/87e61dcd-75f6-4d3f-8ed9-7d35b06a9bb3-kube-api-access-vv9nf\") pod \"node-ca-qxnvq\" (UID: \"87e61dcd-75f6-4d3f-8ed9-7d35b06a9bb3\") " pod="openshift-image-registry/node-ca-qxnvq"
Oct 31 17:26:39.354760 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.354728    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5819fd64-9ac0-4436-b8fe-dcd3a27827ac-var-lib-openvswitch\") pod \"ovnkube-node-zpf4d\" (UID: \"5819fd64-9ac0-4436-b8fe-dcd3a27827ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpf4d"
Oct 31 17:26:39.354911 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.354758    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/5819fd64-9ac0-4436-b8fe-dcd3a27827ac-ovnkube-script-lib\") pod \"ovnkube-node-zpf4d\" (UID: \"5819fd64-9ac0-4436-b8fe-dcd3a27827ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpf4d"
Oct 31 17:26:39.354911 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.354784    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-28ft5\" (UniqueName: \"kubernetes.io/projected/47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e-kube-api-access-28ft5\") pod \"multus-rl4hl\" (UID: \"47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e\") " pod="openshift-multus/multus-rl4hl"
Oct 31 17:26:39.354911 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.354808    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/422b9ca9-3e0e-4fe0-860d-7c7d24b179fa-tmp\") pod \"tuned-rvsdp\" (UID: \"422b9ca9-3e0e-4fe0-860d-7c7d24b179fa\") " pod="openshift-cluster-node-tuning-operator/tuned-rvsdp"
Oct 31 17:26:39.355067 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.354907    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/5819fd64-9ac0-4436-b8fe-dcd3a27827ac-ovn-node-metrics-cert\") pod \"ovnkube-node-zpf4d\" (UID: \"5819fd64-9ac0-4436-b8fe-dcd3a27827ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpf4d"
Oct 31 17:26:39.355067 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.354952    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/422b9ca9-3e0e-4fe0-860d-7c7d24b179fa-sys\") pod \"tuned-rvsdp\" (UID: \"422b9ca9-3e0e-4fe0-860d-7c7d24b179fa\") " pod="openshift-cluster-node-tuning-operator/tuned-rvsdp"
Oct 31 17:26:39.355067 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.354997    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/72516605-a894-4a62-84c7-834122a01fb8-cni-binary-copy\") pod \"multus-additional-cni-plugins-fhgqw\" (UID: \"72516605-a894-4a62-84c7-834122a01fb8\") " pod="openshift-multus/multus-additional-cni-plugins-fhgqw"
Oct 31 17:26:39.355067 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.355028    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5819fd64-9ac0-4436-b8fe-dcd3a27827ac-host-slash\") pod \"ovnkube-node-zpf4d\" (UID: \"5819fd64-9ac0-4436-b8fe-dcd3a27827ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpf4d"
Oct 31 17:26:39.355067 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.355057    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-59c8z\" (UniqueName: \"kubernetes.io/projected/422b9ca9-3e0e-4fe0-860d-7c7d24b179fa-kube-api-access-59c8z\") pod \"tuned-rvsdp\" (UID: \"422b9ca9-3e0e-4fe0-860d-7c7d24b179fa\") " pod="openshift-cluster-node-tuning-operator/tuned-rvsdp"
Oct 31 17:26:39.355248 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.355108    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e-multus-daemon-config\") pod \"multus-rl4hl\" (UID: \"47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e\") " pod="openshift-multus/multus-rl4hl"
Oct 31 17:26:39.355248 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.355173    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6d7781d6-8f73-4f1d-bb9a-0212e2eb6e4b-kubelet-config\") pod \"global-pull-secret-syncer-kmkdh\" (UID: \"6d7781d6-8f73-4f1d-bb9a-0212e2eb6e4b\") " pod="kube-system/global-pull-secret-syncer-kmkdh"
Oct 31 17:26:39.355248 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.355199    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5819fd64-9ac0-4436-b8fe-dcd3a27827ac-run-openvswitch\") pod \"ovnkube-node-zpf4d\" (UID: \"5819fd64-9ac0-4436-b8fe-dcd3a27827ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpf4d"
Oct 31 17:26:39.355248 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.355228    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/5819fd64-9ac0-4436-b8fe-dcd3a27827ac-node-log\") pod \"ovnkube-node-zpf4d\" (UID: \"5819fd64-9ac0-4436-b8fe-dcd3a27827ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpf4d"
Oct 31 17:26:39.355369 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.355250    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/2f646206-1186-4fcf-a9ca-fe92371d4d2b-konnectivity-ca\") pod \"konnectivity-agent-mnbmt\" (UID: \"2f646206-1186-4fcf-a9ca-fe92371d4d2b\") " pod="kube-system/konnectivity-agent-mnbmt"
Oct 31 17:26:39.355369 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.355273    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-grrfv\" (UniqueName: \"kubernetes.io/projected/b8bbaf33-7cd9-4a30-86f7-6e20ac45b0a0-kube-api-access-grrfv\") pod \"node-resolver-7pfr4\" (UID: \"b8bbaf33-7cd9-4a30-86f7-6e20ac45b0a0\") " pod="openshift-dns/node-resolver-7pfr4"
Oct 31 17:26:39.355369 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.355304    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/72516605-a894-4a62-84c7-834122a01fb8-tuning-conf-dir\") pod \"multus-additional-cni-plugins-fhgqw\" (UID: \"72516605-a894-4a62-84c7-834122a01fb8\") " pod="openshift-multus/multus-additional-cni-plugins-fhgqw"
Oct 31 17:26:39.355369 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.355329    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e-multus-cni-dir\") pod \"multus-rl4hl\" (UID: \"47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e\") " pod="openshift-multus/multus-rl4hl"
Oct 31 17:26:39.355503 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.355374    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e-multus-socket-dir-parent\") pod \"multus-rl4hl\" (UID: \"47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e\") " pod="openshift-multus/multus-rl4hl"
Oct 31 17:26:39.355503 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.355411    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e-host-run-netns\") pod \"multus-rl4hl\" (UID: \"47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e\") " pod="openshift-multus/multus-rl4hl"
Oct 31 17:26:39.355503 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.355436    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/5819fd64-9ac0-4436-b8fe-dcd3a27827ac-run-ovn\") pod \"ovnkube-node-zpf4d\" (UID: \"5819fd64-9ac0-4436-b8fe-dcd3a27827ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpf4d"
Oct 31 17:26:39.355503 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.355485    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5819fd64-9ac0-4436-b8fe-dcd3a27827ac-host-run-ovn-kubernetes\") pod \"ovnkube-node-zpf4d\" (UID: \"5819fd64-9ac0-4436-b8fe-dcd3a27827ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpf4d"
Oct 31 17:26:39.355688 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.355507    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/0f353547-2feb-42f4-aae4-b0f1a30efa34-etc-selinux\") pod \"aws-ebs-csi-driver-node-2v874\" (UID: \"0f353547-2feb-42f4-aae4-b0f1a30efa34\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2v874"
Oct 31 17:26:39.355688 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.355522    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/422b9ca9-3e0e-4fe0-860d-7c7d24b179fa-etc-sysctl-d\") pod \"tuned-rvsdp\" (UID: \"422b9ca9-3e0e-4fe0-860d-7c7d24b179fa\") " pod="openshift-cluster-node-tuning-operator/tuned-rvsdp"
Oct 31 17:26:39.355688 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.355536    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/138bfcc8-f199-4ffd-b5d2-8d83ade6199a-metrics-certs\") pod \"network-metrics-daemon-pnjdm\" (UID: \"138bfcc8-f199-4ffd-b5d2-8d83ade6199a\") " pod="openshift-multus/network-metrics-daemon-pnjdm"
Oct 31 17:26:39.355688 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.355563    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6d7781d6-8f73-4f1d-bb9a-0212e2eb6e4b-dbus\") pod \"global-pull-secret-syncer-kmkdh\" (UID: \"6d7781d6-8f73-4f1d-bb9a-0212e2eb6e4b\") " pod="kube-system/global-pull-secret-syncer-kmkdh"
Oct 31 17:26:39.355688 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.355611    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6d7781d6-8f73-4f1d-bb9a-0212e2eb6e4b-original-pull-secret\") pod \"global-pull-secret-syncer-kmkdh\" (UID: \"6d7781d6-8f73-4f1d-bb9a-0212e2eb6e4b\") " pod="kube-system/global-pull-secret-syncer-kmkdh"
Oct 31 17:26:39.355688 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.355655    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/422b9ca9-3e0e-4fe0-860d-7c7d24b179fa-host\") pod \"tuned-rvsdp\" (UID: \"422b9ca9-3e0e-4fe0-860d-7c7d24b179fa\") " pod="openshift-cluster-node-tuning-operator/tuned-rvsdp"
Oct 31 17:26:39.355688 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.355683    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c6k68\" (UniqueName: \"kubernetes.io/projected/fb9c203a-af67-4df4-88dc-74cf9eab3623-kube-api-access-c6k68\") pod \"network-check-target-tskzw\" (UID: \"fb9c203a-af67-4df4-88dc-74cf9eab3623\") " pod="openshift-network-diagnostics/network-check-target-tskzw"
Oct 31 17:26:39.355985 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.355733    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5819fd64-9ac0-4436-b8fe-dcd3a27827ac-host-run-netns\") pod \"ovnkube-node-zpf4d\" (UID: \"5819fd64-9ac0-4436-b8fe-dcd3a27827ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpf4d"
Oct 31 17:26:39.355985 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.355770    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b8bbaf33-7cd9-4a30-86f7-6e20ac45b0a0-tmp-dir\") pod \"node-resolver-7pfr4\" (UID: \"b8bbaf33-7cd9-4a30-86f7-6e20ac45b0a0\") " pod="openshift-dns/node-resolver-7pfr4"
Oct 31 17:26:39.355985 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.355793    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e-cnibin\") pod \"multus-rl4hl\" (UID: \"47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e\") " pod="openshift-multus/multus-rl4hl"
Oct 31 17:26:39.355985 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.355829    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e-host-var-lib-cni-multus\") pod \"multus-rl4hl\" (UID: \"47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e\") " pod="openshift-multus/multus-rl4hl"
Oct 31 17:26:39.355985 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.355876    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d6ck5\" (UniqueName: \"kubernetes.io/projected/138bfcc8-f199-4ffd-b5d2-8d83ade6199a-kube-api-access-d6ck5\") pod \"network-metrics-daemon-pnjdm\" (UID: \"138bfcc8-f199-4ffd-b5d2-8d83ade6199a\") " pod="openshift-multus/network-metrics-daemon-pnjdm"
Oct 31 17:26:39.355985 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.355907    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/5819fd64-9ac0-4436-b8fe-dcd3a27827ac-systemd-units\") pod \"ovnkube-node-zpf4d\" (UID: \"5819fd64-9ac0-4436-b8fe-dcd3a27827ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpf4d"
Oct 31 17:26:39.355985 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.355936    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5819fd64-9ac0-4436-b8fe-dcd3a27827ac-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-zpf4d\" (UID: \"5819fd64-9ac0-4436-b8fe-dcd3a27827ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpf4d"
Oct 31 17:26:39.355985 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.355983    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/422b9ca9-3e0e-4fe0-860d-7c7d24b179fa-etc-sysconfig\") pod \"tuned-rvsdp\" (UID: \"422b9ca9-3e0e-4fe0-860d-7c7d24b179fa\") " pod="openshift-cluster-node-tuning-operator/tuned-rvsdp"
Oct 31 17:26:39.356330 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.356017    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/422b9ca9-3e0e-4fe0-860d-7c7d24b179fa-etc-sysctl-conf\") pod \"tuned-rvsdp\" (UID: \"422b9ca9-3e0e-4fe0-860d-7c7d24b179fa\") " pod="openshift-cluster-node-tuning-operator/tuned-rvsdp"
Oct 31 17:26:39.356330 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.356041    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/5819fd64-9ac0-4436-b8fe-dcd3a27827ac-run-systemd\") pod \"ovnkube-node-zpf4d\" (UID: \"5819fd64-9ac0-4436-b8fe-dcd3a27827ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpf4d"
Oct 31 17:26:39.356330 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.356075    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5819fd64-9ac0-4436-b8fe-dcd3a27827ac-host-cni-bin\") pod \"ovnkube-node-zpf4d\" (UID: \"5819fd64-9ac0-4436-b8fe-dcd3a27827ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpf4d"
Oct 31 17:26:39.356330 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.356109    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/422b9ca9-3e0e-4fe0-860d-7c7d24b179fa-run\") pod \"tuned-rvsdp\" (UID: \"422b9ca9-3e0e-4fe0-860d-7c7d24b179fa\") " pod="openshift-cluster-node-tuning-operator/tuned-rvsdp"
Oct 31 17:26:39.356330 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.356194    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xxrgs\" (UniqueName: \"kubernetes.io/projected/72516605-a894-4a62-84c7-834122a01fb8-kube-api-access-xxrgs\") pod \"multus-additional-cni-plugins-fhgqw\" (UID: \"72516605-a894-4a62-84c7-834122a01fb8\") " pod="openshift-multus/multus-additional-cni-plugins-fhgqw"
Oct 31 17:26:39.356330 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.356224    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e-host-var-lib-cni-bin\") pod \"multus-rl4hl\" (UID: \"47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e\") " pod="openshift-multus/multus-rl4hl"
Oct 31 17:26:39.356330 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.356258    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e-hostroot\") pod \"multus-rl4hl\" (UID: \"47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e\") " pod="openshift-multus/multus-rl4hl"
Oct 31 17:26:39.356330 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.356302    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/03a15842-25a2-4fbe-b418-8867070c4482-iptables-alerter-script\") pod \"iptables-alerter-jzcgs\" (UID: \"03a15842-25a2-4fbe-b418-8867070c4482\") " pod="openshift-network-operator/iptables-alerter-jzcgs"
Oct 31 17:26:39.356330 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.356324    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/5819fd64-9ac0-4436-b8fe-dcd3a27827ac-host-kubelet\") pod \"ovnkube-node-zpf4d\" (UID: \"5819fd64-9ac0-4436-b8fe-dcd3a27827ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpf4d"
Oct 31 17:26:39.356714 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.356340    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/5819fd64-9ac0-4436-b8fe-dcd3a27827ac-ovnkube-config\") pod \"ovnkube-node-zpf4d\" (UID: \"5819fd64-9ac0-4436-b8fe-dcd3a27827ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpf4d"
Oct 31 17:26:39.356714 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.356356    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/2f646206-1186-4fcf-a9ca-fe92371d4d2b-agent-certs\") pod \"konnectivity-agent-mnbmt\" (UID: \"2f646206-1186-4fcf-a9ca-fe92371d4d2b\") " pod="kube-system/konnectivity-agent-mnbmt"
Oct 31 17:26:39.356714 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.356380    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/0f353547-2feb-42f4-aae4-b0f1a30efa34-sys-fs\") pod \"aws-ebs-csi-driver-node-2v874\" (UID: \"0f353547-2feb-42f4-aae4-b0f1a30efa34\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2v874"
Oct 31 17:26:39.356714 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.356443    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qfvft\" (UniqueName: \"kubernetes.io/projected/0f353547-2feb-42f4-aae4-b0f1a30efa34-kube-api-access-qfvft\") pod \"aws-ebs-csi-driver-node-2v874\" (UID: \"0f353547-2feb-42f4-aae4-b0f1a30efa34\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2v874"
Oct 31 17:26:39.356714 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.356479    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/422b9ca9-3e0e-4fe0-860d-7c7d24b179fa-etc-modprobe-d\") pod \"tuned-rvsdp\" (UID: \"422b9ca9-3e0e-4fe0-860d-7c7d24b179fa\") " pod="openshift-cluster-node-tuning-operator/tuned-rvsdp"
Oct 31 17:26:39.356714 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.356494    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e-host-run-k8s-cni-cncf-io\") pod \"multus-rl4hl\" (UID: \"47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e\") " pod="openshift-multus/multus-rl4hl"
Oct 31 17:26:39.356714 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.356509    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g2n2p\" (UniqueName: \"kubernetes.io/projected/5819fd64-9ac0-4436-b8fe-dcd3a27827ac-kube-api-access-g2n2p\") pod \"ovnkube-node-zpf4d\" (UID: \"5819fd64-9ac0-4436-b8fe-dcd3a27827ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpf4d"
Oct 31 17:26:39.356714 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.356524    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/422b9ca9-3e0e-4fe0-860d-7c7d24b179fa-var-lib-kubelet\") pod \"tuned-rvsdp\" (UID: \"422b9ca9-3e0e-4fe0-860d-7c7d24b179fa\") " pod="openshift-cluster-node-tuning-operator/tuned-rvsdp"
Oct 31 17:26:39.356714 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.356552    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b8bbaf33-7cd9-4a30-86f7-6e20ac45b0a0-hosts-file\") pod \"node-resolver-7pfr4\" (UID: \"b8bbaf33-7cd9-4a30-86f7-6e20ac45b0a0\") " pod="openshift-dns/node-resolver-7pfr4"
Oct 31 17:26:39.356714 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.356579    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e-host-var-lib-kubelet\") pod \"multus-rl4hl\" (UID: \"47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e\") " pod="openshift-multus/multus-rl4hl"
Oct 31 17:26:39.356714 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.356603    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e-multus-conf-dir\") pod \"multus-rl4hl\" (UID: \"47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e\") " pod="openshift-multus/multus-rl4hl"
Oct 31 17:26:39.356714 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.356619    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e-host-run-multus-certs\") pod \"multus-rl4hl\" (UID: \"47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e\") " pod="openshift-multus/multus-rl4hl"
Oct 31 17:26:39.356714 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.356680    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5819fd64-9ac0-4436-b8fe-dcd3a27827ac-etc-openvswitch\") pod \"ovnkube-node-zpf4d\" (UID: \"5819fd64-9ac0-4436-b8fe-dcd3a27827ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpf4d"
Oct 31 17:26:39.356714 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.356715    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/5819fd64-9ac0-4436-b8fe-dcd3a27827ac-host-cni-netd\") pod \"ovnkube-node-zpf4d\" (UID: \"5819fd64-9ac0-4436-b8fe-dcd3a27827ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpf4d"
Oct 31 17:26:39.357329 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.356735    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/0f353547-2feb-42f4-aae4-b0f1a30efa34-registration-dir\") pod \"aws-ebs-csi-driver-node-2v874\" (UID: \"0f353547-2feb-42f4-aae4-b0f1a30efa34\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2v874"
Oct 31 17:26:39.357329 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.356789    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/422b9ca9-3e0e-4fe0-860d-7c7d24b179fa-etc-kubernetes\") pod \"tuned-rvsdp\" (UID: \"422b9ca9-3e0e-4fe0-860d-7c7d24b179fa\") " pod="openshift-cluster-node-tuning-operator/tuned-rvsdp"
Oct 31 17:26:39.357329 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.356833    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/03a15842-25a2-4fbe-b418-8867070c4482-host-slash\") pod \"iptables-alerter-jzcgs\" (UID: \"03a15842-25a2-4fbe-b418-8867070c4482\") " pod="openshift-network-operator/iptables-alerter-jzcgs"
Oct 31 17:26:39.357329 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.356851    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/72516605-a894-4a62-84c7-834122a01fb8-cnibin\") pod \"multus-additional-cni-plugins-fhgqw\" (UID: \"72516605-a894-4a62-84c7-834122a01fb8\") " pod="openshift-multus/multus-additional-cni-plugins-fhgqw"
Oct 31 17:26:39.357329 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.356899    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e-system-cni-dir\") pod \"multus-rl4hl\" (UID: \"47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e\") " pod="openshift-multus/multus-rl4hl"
Oct 31 17:26:39.357329 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.356930    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/5819fd64-9ac0-4436-b8fe-dcd3a27827ac-log-socket\") pod \"ovnkube-node-zpf4d\" (UID: \"5819fd64-9ac0-4436-b8fe-dcd3a27827ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpf4d"
Oct 31 17:26:39.357329 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.356970    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/422b9ca9-3e0e-4fe0-860d-7c7d24b179fa-etc-systemd\") pod \"tuned-rvsdp\" (UID: \"422b9ca9-3e0e-4fe0-860d-7c7d24b179fa\") " pod="openshift-cluster-node-tuning-operator/tuned-rvsdp"
Oct 31 17:26:39.357329 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.357006    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/72516605-a894-4a62-84c7-834122a01fb8-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-fhgqw\" (UID: \"72516605-a894-4a62-84c7-834122a01fb8\") " pod="openshift-multus/multus-additional-cni-plugins-fhgqw"
Oct 31 17:26:39.357329 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.357044    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/72516605-a894-4a62-84c7-834122a01fb8-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-fhgqw\" (UID: \"72516605-a894-4a62-84c7-834122a01fb8\") " pod="openshift-multus/multus-additional-cni-plugins-fhgqw"
Oct 31 17:26:39.357329 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.357075    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e-etc-kubernetes\") pod \"multus-rl4hl\" (UID: \"47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e\") " pod="openshift-multus/multus-rl4hl"
Oct 31 17:26:39.357329 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.357119    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/87e61dcd-75f6-4d3f-8ed9-7d35b06a9bb3-host\") pod \"node-ca-qxnvq\" (UID: \"87e61dcd-75f6-4d3f-8ed9-7d35b06a9bb3\") " pod="openshift-image-registry/node-ca-qxnvq"
Oct 31 17:26:39.357329 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.357171    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/5819fd64-9ac0-4436-b8fe-dcd3a27827ac-env-overrides\") pod \"ovnkube-node-zpf4d\" (UID: \"5819fd64-9ac0-4436-b8fe-dcd3a27827ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpf4d"
Oct 31 17:26:39.357329 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.357198    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/0f353547-2feb-42f4-aae4-b0f1a30efa34-kubelet-dir\") pod \"aws-ebs-csi-driver-node-2v874\" (UID: \"0f353547-2feb-42f4-aae4-b0f1a30efa34\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2v874"
Oct 31 17:26:39.357329 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.357225    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/0f353547-2feb-42f4-aae4-b0f1a30efa34-device-dir\") pod \"aws-ebs-csi-driver-node-2v874\" (UID: \"0f353547-2feb-42f4-aae4-b0f1a30efa34\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2v874"
Oct 31 17:26:39.357329 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.357250    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/422b9ca9-3e0e-4fe0-860d-7c7d24b179fa-etc-tuned\") pod \"tuned-rvsdp\" (UID: \"422b9ca9-3e0e-4fe0-860d-7c7d24b179fa\") " pod="openshift-cluster-node-tuning-operator/tuned-rvsdp"
Oct 31 17:26:39.357329 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.357277    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lc2r4\" (UniqueName: \"kubernetes.io/projected/03a15842-25a2-4fbe-b418-8867070c4482-kube-api-access-lc2r4\") pod \"iptables-alerter-jzcgs\" (UID: \"03a15842-25a2-4fbe-b418-8867070c4482\") " pod="openshift-network-operator/iptables-alerter-jzcgs"
Oct 31 17:26:39.417051 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.417012    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Oct 31 17:26:39.417051 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.417035    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Oct 31 17:26:39.417285 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.417264    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Oct 31 17:26:39.417368 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.417353    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-x4c5q\""
Oct 31 17:26:39.417499 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.417468    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-5bhk8\""
Oct 31 17:26:39.417598 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.417533    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-g2t4b\""
Oct 31 17:26:39.417598 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.417585    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Oct 31 17:26:39.417717 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.417626    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Oct 31 17:26:39.417717 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.417664    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Oct 31 17:26:39.417825 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.417729    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Oct 31 17:26:39.457528 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.457471    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/422b9ca9-3e0e-4fe0-860d-7c7d24b179fa-run\") pod \"tuned-rvsdp\" (UID: \"422b9ca9-3e0e-4fe0-860d-7c7d24b179fa\") " pod="openshift-cluster-node-tuning-operator/tuned-rvsdp"
Oct 31 17:26:39.457528 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.457501    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xxrgs\" (UniqueName: \"kubernetes.io/projected/72516605-a894-4a62-84c7-834122a01fb8-kube-api-access-xxrgs\") pod \"multus-additional-cni-plugins-fhgqw\" (UID: \"72516605-a894-4a62-84c7-834122a01fb8\") " pod="openshift-multus/multus-additional-cni-plugins-fhgqw"
Oct 31 17:26:39.457660 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.457531    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e-host-var-lib-cni-bin\") pod \"multus-rl4hl\" (UID: \"47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e\") " pod="openshift-multus/multus-rl4hl"
Oct 31 17:26:39.457660 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.457548    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e-hostroot\") pod \"multus-rl4hl\" (UID: \"47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e\") " pod="openshift-multus/multus-rl4hl"
Oct 31 17:26:39.457660 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.457564    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/03a15842-25a2-4fbe-b418-8867070c4482-iptables-alerter-script\") pod \"iptables-alerter-jzcgs\" (UID: \"03a15842-25a2-4fbe-b418-8867070c4482\") " pod="openshift-network-operator/iptables-alerter-jzcgs"
Oct 31 17:26:39.457660 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.457610    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/5819fd64-9ac0-4436-b8fe-dcd3a27827ac-host-kubelet\") pod \"ovnkube-node-zpf4d\" (UID: \"5819fd64-9ac0-4436-b8fe-dcd3a27827ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpf4d"
Oct 31 17:26:39.457660 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.457625    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e-host-var-lib-cni-bin\") pod \"multus-rl4hl\" (UID: \"47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e\") " pod="openshift-multus/multus-rl4hl"
Oct 31 17:26:39.457660 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.457638    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e-hostroot\") pod \"multus-rl4hl\" (UID: \"47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e\") " pod="openshift-multus/multus-rl4hl"
Oct 31 17:26:39.457660 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.457642    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/422b9ca9-3e0e-4fe0-860d-7c7d24b179fa-run\") pod \"tuned-rvsdp\" (UID: \"422b9ca9-3e0e-4fe0-860d-7c7d24b179fa\") " pod="openshift-cluster-node-tuning-operator/tuned-rvsdp"
Oct 31 17:26:39.457660 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.457652    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/5819fd64-9ac0-4436-b8fe-dcd3a27827ac-ovnkube-config\") pod \"ovnkube-node-zpf4d\" (UID: \"5819fd64-9ac0-4436-b8fe-dcd3a27827ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpf4d"
Oct 31 17:26:39.458006 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.457690    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/5819fd64-9ac0-4436-b8fe-dcd3a27827ac-host-kubelet\") pod \"ovnkube-node-zpf4d\" (UID: \"5819fd64-9ac0-4436-b8fe-dcd3a27827ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpf4d"
Oct 31 17:26:39.458006 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.457697    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/2f646206-1186-4fcf-a9ca-fe92371d4d2b-agent-certs\") pod \"konnectivity-agent-mnbmt\" (UID: \"2f646206-1186-4fcf-a9ca-fe92371d4d2b\") " pod="kube-system/konnectivity-agent-mnbmt"
Oct 31 17:26:39.458006 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.457720    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/0f353547-2feb-42f4-aae4-b0f1a30efa34-sys-fs\") pod \"aws-ebs-csi-driver-node-2v874\" (UID: \"0f353547-2feb-42f4-aae4-b0f1a30efa34\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2v874"
Oct 31 17:26:39.458006 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.457736    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qfvft\" (UniqueName: \"kubernetes.io/projected/0f353547-2feb-42f4-aae4-b0f1a30efa34-kube-api-access-qfvft\") pod \"aws-ebs-csi-driver-node-2v874\" (UID: \"0f353547-2feb-42f4-aae4-b0f1a30efa34\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2v874"
Oct 31 17:26:39.458006 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.457759    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/422b9ca9-3e0e-4fe0-860d-7c7d24b179fa-etc-modprobe-d\") pod \"tuned-rvsdp\" (UID: \"422b9ca9-3e0e-4fe0-860d-7c7d24b179fa\") " pod="openshift-cluster-node-tuning-operator/tuned-rvsdp"
Oct 31 17:26:39.458006 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.457784    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e-host-run-k8s-cni-cncf-io\") pod \"multus-rl4hl\" (UID: \"47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e\") " pod="openshift-multus/multus-rl4hl"
Oct 31 17:26:39.458006 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.457804    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/0f353547-2feb-42f4-aae4-b0f1a30efa34-sys-fs\") pod \"aws-ebs-csi-driver-node-2v874\" (UID: \"0f353547-2feb-42f4-aae4-b0f1a30efa34\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2v874"
Oct 31 17:26:39.458006 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.457814    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g2n2p\" (UniqueName: \"kubernetes.io/projected/5819fd64-9ac0-4436-b8fe-dcd3a27827ac-kube-api-access-g2n2p\") pod \"ovnkube-node-zpf4d\" (UID: \"5819fd64-9ac0-4436-b8fe-dcd3a27827ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpf4d"
Oct 31 17:26:39.458006 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.457843    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/422b9ca9-3e0e-4fe0-860d-7c7d24b179fa-var-lib-kubelet\") pod \"tuned-rvsdp\" (UID: \"422b9ca9-3e0e-4fe0-860d-7c7d24b179fa\") " pod="openshift-cluster-node-tuning-operator/tuned-rvsdp"
Oct 31 17:26:39.458006 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.457883    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b8bbaf33-7cd9-4a30-86f7-6e20ac45b0a0-hosts-file\") pod \"node-resolver-7pfr4\" (UID: \"b8bbaf33-7cd9-4a30-86f7-6e20ac45b0a0\") " pod="openshift-dns/node-resolver-7pfr4"
Oct 31 17:26:39.458006 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.457908    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e-host-var-lib-kubelet\") pod \"multus-rl4hl\" (UID: \"47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e\") " pod="openshift-multus/multus-rl4hl"
Oct 31 17:26:39.458006 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.457915    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/422b9ca9-3e0e-4fe0-860d-7c7d24b179fa-etc-modprobe-d\") pod \"tuned-rvsdp\" (UID: \"422b9ca9-3e0e-4fe0-860d-7c7d24b179fa\") " pod="openshift-cluster-node-tuning-operator/tuned-rvsdp"
Oct 31 17:26:39.458006 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.457939    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e-multus-conf-dir\") pod \"multus-rl4hl\" (UID: \"47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e\") " pod="openshift-multus/multus-rl4hl"
Oct 31 17:26:39.458006 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.457981    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e-multus-conf-dir\") pod \"multus-rl4hl\" (UID: \"47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e\") " pod="openshift-multus/multus-rl4hl"
Oct 31 17:26:39.458006 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.458002    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e-host-run-multus-certs\") pod \"multus-rl4hl\" (UID: \"47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e\") " pod="openshift-multus/multus-rl4hl"
Oct 31 17:26:39.458731 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.458029    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/422b9ca9-3e0e-4fe0-860d-7c7d24b179fa-var-lib-kubelet\") pod \"tuned-rvsdp\" (UID: \"422b9ca9-3e0e-4fe0-860d-7c7d24b179fa\") " pod="openshift-cluster-node-tuning-operator/tuned-rvsdp"
Oct 31 17:26:39.458731 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.458036    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5819fd64-9ac0-4436-b8fe-dcd3a27827ac-etc-openvswitch\") pod \"ovnkube-node-zpf4d\" (UID: \"5819fd64-9ac0-4436-b8fe-dcd3a27827ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpf4d"
Oct 31 17:26:39.458731 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.458071    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/5819fd64-9ac0-4436-b8fe-dcd3a27827ac-host-cni-netd\") pod \"ovnkube-node-zpf4d\" (UID: \"5819fd64-9ac0-4436-b8fe-dcd3a27827ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpf4d"
Oct 31 17:26:39.458731 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.458069    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e-host-run-k8s-cni-cncf-io\") pod \"multus-rl4hl\" (UID: \"47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e\") " pod="openshift-multus/multus-rl4hl"
Oct 31 17:26:39.458731 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.458097    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/0f353547-2feb-42f4-aae4-b0f1a30efa34-registration-dir\") pod \"aws-ebs-csi-driver-node-2v874\" (UID: \"0f353547-2feb-42f4-aae4-b0f1a30efa34\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2v874"
Oct 31 17:26:39.458731 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.458116    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e-host-var-lib-kubelet\") pod \"multus-rl4hl\" (UID: \"47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e\") " pod="openshift-multus/multus-rl4hl"
Oct 31 17:26:39.458731 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.458131    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/422b9ca9-3e0e-4fe0-860d-7c7d24b179fa-etc-kubernetes\") pod \"tuned-rvsdp\" (UID: \"422b9ca9-3e0e-4fe0-860d-7c7d24b179fa\") " pod="openshift-cluster-node-tuning-operator/tuned-rvsdp"
Oct 31 17:26:39.458731 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.458134    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e-host-run-multus-certs\") pod \"multus-rl4hl\" (UID: \"47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e\") " pod="openshift-multus/multus-rl4hl"
Oct 31 17:26:39.458731 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.458144    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5819fd64-9ac0-4436-b8fe-dcd3a27827ac-etc-openvswitch\") pod \"ovnkube-node-zpf4d\" (UID: \"5819fd64-9ac0-4436-b8fe-dcd3a27827ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpf4d"
Oct 31 17:26:39.458731 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.458158    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/03a15842-25a2-4fbe-b418-8867070c4482-host-slash\") pod \"iptables-alerter-jzcgs\" (UID: \"03a15842-25a2-4fbe-b418-8867070c4482\") " pod="openshift-network-operator/iptables-alerter-jzcgs"
Oct 31 17:26:39.458731 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.458168    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/0f353547-2feb-42f4-aae4-b0f1a30efa34-registration-dir\") pod \"aws-ebs-csi-driver-node-2v874\" (UID: \"0f353547-2feb-42f4-aae4-b0f1a30efa34\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2v874"
Oct 31 17:26:39.458731 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.458173    2578 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Oct 31 17:26:39.458731 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.458185    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/72516605-a894-4a62-84c7-834122a01fb8-cnibin\") pod \"multus-additional-cni-plugins-fhgqw\" (UID: \"72516605-a894-4a62-84c7-834122a01fb8\") " pod="openshift-multus/multus-additional-cni-plugins-fhgqw"
Oct 31 17:26:39.458731 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.458195    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/5819fd64-9ac0-4436-b8fe-dcd3a27827ac-host-cni-netd\") pod \"ovnkube-node-zpf4d\" (UID: \"5819fd64-9ac0-4436-b8fe-dcd3a27827ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpf4d"
Oct 31 17:26:39.458731 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.458198    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/03a15842-25a2-4fbe-b418-8867070c4482-host-slash\") pod \"iptables-alerter-jzcgs\" (UID: \"03a15842-25a2-4fbe-b418-8867070c4482\") " pod="openshift-network-operator/iptables-alerter-jzcgs"
Oct 31 17:26:39.458731 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.458083    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b8bbaf33-7cd9-4a30-86f7-6e20ac45b0a0-hosts-file\") pod \"node-resolver-7pfr4\" (UID: \"b8bbaf33-7cd9-4a30-86f7-6e20ac45b0a0\") " pod="openshift-dns/node-resolver-7pfr4"
Oct 31 17:26:39.458731 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.458210    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e-system-cni-dir\") pod \"multus-rl4hl\" (UID: \"47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e\") " pod="openshift-multus/multus-rl4hl"
Oct 31 17:26:39.458731 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.458214    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/5819fd64-9ac0-4436-b8fe-dcd3a27827ac-ovnkube-config\") pod \"ovnkube-node-zpf4d\" (UID: \"5819fd64-9ac0-4436-b8fe-dcd3a27827ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpf4d"
Oct 31 17:26:39.459477 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.458235    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/5819fd64-9ac0-4436-b8fe-dcd3a27827ac-log-socket\") pod \"ovnkube-node-zpf4d\" (UID: \"5819fd64-9ac0-4436-b8fe-dcd3a27827ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpf4d"
Oct 31 17:26:39.459477 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.458240    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/422b9ca9-3e0e-4fe0-860d-7c7d24b179fa-etc-kubernetes\") pod \"tuned-rvsdp\" (UID: \"422b9ca9-3e0e-4fe0-860d-7c7d24b179fa\") " pod="openshift-cluster-node-tuning-operator/tuned-rvsdp"
Oct 31 17:26:39.459477 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.458262    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/422b9ca9-3e0e-4fe0-860d-7c7d24b179fa-etc-systemd\") pod \"tuned-rvsdp\" (UID: \"422b9ca9-3e0e-4fe0-860d-7c7d24b179fa\") " pod="openshift-cluster-node-tuning-operator/tuned-rvsdp"
Oct 31 17:26:39.459477 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.458262    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/72516605-a894-4a62-84c7-834122a01fb8-cnibin\") pod \"multus-additional-cni-plugins-fhgqw\" (UID: \"72516605-a894-4a62-84c7-834122a01fb8\") " pod="openshift-multus/multus-additional-cni-plugins-fhgqw"
Oct 31 17:26:39.459477 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.458147    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/03a15842-25a2-4fbe-b418-8867070c4482-iptables-alerter-script\") pod \"iptables-alerter-jzcgs\" (UID: \"03a15842-25a2-4fbe-b418-8867070c4482\") " pod="openshift-network-operator/iptables-alerter-jzcgs"
Oct 31 17:26:39.459477 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.458288    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/72516605-a894-4a62-84c7-834122a01fb8-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-fhgqw\" (UID: \"72516605-a894-4a62-84c7-834122a01fb8\") " pod="openshift-multus/multus-additional-cni-plugins-fhgqw"
Oct 31 17:26:39.459477 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.458294    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/5819fd64-9ac0-4436-b8fe-dcd3a27827ac-log-socket\") pod \"ovnkube-node-zpf4d\" (UID: \"5819fd64-9ac0-4436-b8fe-dcd3a27827ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpf4d"
Oct 31 17:26:39.459477 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.458315    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/422b9ca9-3e0e-4fe0-860d-7c7d24b179fa-etc-systemd\") pod \"tuned-rvsdp\" (UID: \"422b9ca9-3e0e-4fe0-860d-7c7d24b179fa\") " pod="openshift-cluster-node-tuning-operator/tuned-rvsdp"
Oct 31 17:26:39.459477 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.458319    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e-system-cni-dir\") pod \"multus-rl4hl\" (UID: \"47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e\") " pod="openshift-multus/multus-rl4hl"
Oct 31 17:26:39.459477 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.458317    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/72516605-a894-4a62-84c7-834122a01fb8-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-fhgqw\" (UID: \"72516605-a894-4a62-84c7-834122a01fb8\") " pod="openshift-multus/multus-additional-cni-plugins-fhgqw"
Oct 31 17:26:39.459477 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.458370    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e-etc-kubernetes\") pod \"multus-rl4hl\" (UID: \"47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e\") " pod="openshift-multus/multus-rl4hl"
Oct 31 17:26:39.459477 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.458396    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/87e61dcd-75f6-4d3f-8ed9-7d35b06a9bb3-host\") pod \"node-ca-qxnvq\" (UID: \"87e61dcd-75f6-4d3f-8ed9-7d35b06a9bb3\") " pod="openshift-image-registry/node-ca-qxnvq"
Oct 31 17:26:39.459477 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.458417    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/5819fd64-9ac0-4436-b8fe-dcd3a27827ac-env-overrides\") pod \"ovnkube-node-zpf4d\" (UID: \"5819fd64-9ac0-4436-b8fe-dcd3a27827ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpf4d"
Oct 31 17:26:39.459477 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.458431    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e-etc-kubernetes\") pod \"multus-rl4hl\" (UID: \"47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e\") " pod="openshift-multus/multus-rl4hl"
Oct 31 17:26:39.459477 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.458433    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/0f353547-2feb-42f4-aae4-b0f1a30efa34-kubelet-dir\") pod \"aws-ebs-csi-driver-node-2v874\" (UID: \"0f353547-2feb-42f4-aae4-b0f1a30efa34\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2v874"
Oct 31 17:26:39.459477 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.458482    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/0f353547-2feb-42f4-aae4-b0f1a30efa34-kubelet-dir\") pod \"aws-ebs-csi-driver-node-2v874\" (UID: \"0f353547-2feb-42f4-aae4-b0f1a30efa34\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2v874"
Oct 31 17:26:39.459477 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.458486    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/0f353547-2feb-42f4-aae4-b0f1a30efa34-device-dir\") pod \"aws-ebs-csi-driver-node-2v874\" (UID: \"0f353547-2feb-42f4-aae4-b0f1a30efa34\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2v874"
Oct 31 17:26:39.460378 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.458522    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/422b9ca9-3e0e-4fe0-860d-7c7d24b179fa-etc-tuned\") pod \"tuned-rvsdp\" (UID: \"422b9ca9-3e0e-4fe0-860d-7c7d24b179fa\") " pod="openshift-cluster-node-tuning-operator/tuned-rvsdp"
Oct 31 17:26:39.460378 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.458538    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/0f353547-2feb-42f4-aae4-b0f1a30efa34-device-dir\") pod \"aws-ebs-csi-driver-node-2v874\" (UID: \"0f353547-2feb-42f4-aae4-b0f1a30efa34\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2v874"
Oct 31 17:26:39.460378 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.458547    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lc2r4\" (UniqueName: \"kubernetes.io/projected/03a15842-25a2-4fbe-b418-8867070c4482-kube-api-access-lc2r4\") pod \"iptables-alerter-jzcgs\" (UID: \"03a15842-25a2-4fbe-b418-8867070c4482\") " pod="openshift-network-operator/iptables-alerter-jzcgs"
Oct 31 17:26:39.460378 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.458495    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/87e61dcd-75f6-4d3f-8ed9-7d35b06a9bb3-host\") pod \"node-ca-qxnvq\" (UID: \"87e61dcd-75f6-4d3f-8ed9-7d35b06a9bb3\") " pod="openshift-image-registry/node-ca-qxnvq"
Oct 31 17:26:39.460378 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.458573    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/72516605-a894-4a62-84c7-834122a01fb8-system-cni-dir\") pod \"multus-additional-cni-plugins-fhgqw\" (UID: \"72516605-a894-4a62-84c7-834122a01fb8\") " pod="openshift-multus/multus-additional-cni-plugins-fhgqw"
Oct 31 17:26:39.460378 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.458597    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e-cni-binary-copy\") pod \"multus-rl4hl\" (UID: \"47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e\") " pod="openshift-multus/multus-rl4hl"
Oct 31 17:26:39.460378 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.458625    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/0f353547-2feb-42f4-aae4-b0f1a30efa34-socket-dir\") pod \"aws-ebs-csi-driver-node-2v874\" (UID: \"0f353547-2feb-42f4-aae4-b0f1a30efa34\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2v874"
Oct 31 17:26:39.460378 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.458652    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/422b9ca9-3e0e-4fe0-860d-7c7d24b179fa-lib-modules\") pod \"tuned-rvsdp\" (UID: \"422b9ca9-3e0e-4fe0-860d-7c7d24b179fa\") " pod="openshift-cluster-node-tuning-operator/tuned-rvsdp"
Oct 31 17:26:39.460378 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.458680    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/87e61dcd-75f6-4d3f-8ed9-7d35b06a9bb3-serviceca\") pod \"node-ca-qxnvq\" (UID: \"87e61dcd-75f6-4d3f-8ed9-7d35b06a9bb3\") " pod="openshift-image-registry/node-ca-qxnvq"
Oct 31 17:26:39.460378 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.458705    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vv9nf\" (UniqueName: \"kubernetes.io/projected/87e61dcd-75f6-4d3f-8ed9-7d35b06a9bb3-kube-api-access-vv9nf\") pod \"node-ca-qxnvq\" (UID: \"87e61dcd-75f6-4d3f-8ed9-7d35b06a9bb3\") " pod="openshift-image-registry/node-ca-qxnvq"
Oct 31 17:26:39.460378 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.458729    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5819fd64-9ac0-4436-b8fe-dcd3a27827ac-var-lib-openvswitch\") pod \"ovnkube-node-zpf4d\" (UID: \"5819fd64-9ac0-4436-b8fe-dcd3a27827ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpf4d"
Oct 31 17:26:39.460378 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.458780    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/5819fd64-9ac0-4436-b8fe-dcd3a27827ac-ovnkube-script-lib\") pod \"ovnkube-node-zpf4d\" (UID: \"5819fd64-9ac0-4436-b8fe-dcd3a27827ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpf4d"
Oct 31 17:26:39.460378 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.458809    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-28ft5\" (UniqueName: \"kubernetes.io/projected/47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e-kube-api-access-28ft5\") pod \"multus-rl4hl\" (UID: \"47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e\") " pod="openshift-multus/multus-rl4hl"
Oct 31 17:26:39.460378 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.458839    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/422b9ca9-3e0e-4fe0-860d-7c7d24b179fa-tmp\") pod \"tuned-rvsdp\" (UID: \"422b9ca9-3e0e-4fe0-860d-7c7d24b179fa\") " pod="openshift-cluster-node-tuning-operator/tuned-rvsdp"
Oct 31 17:26:39.460378 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.458864    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/5819fd64-9ac0-4436-b8fe-dcd3a27827ac-ovn-node-metrics-cert\") pod \"ovnkube-node-zpf4d\" (UID: \"5819fd64-9ac0-4436-b8fe-dcd3a27827ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpf4d"
Oct 31 17:26:39.460378 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.458893    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/422b9ca9-3e0e-4fe0-860d-7c7d24b179fa-sys\") pod \"tuned-rvsdp\" (UID: \"422b9ca9-3e0e-4fe0-860d-7c7d24b179fa\") " pod="openshift-cluster-node-tuning-operator/tuned-rvsdp"
Oct 31 17:26:39.460378 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.458919    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/72516605-a894-4a62-84c7-834122a01fb8-cni-binary-copy\") pod \"multus-additional-cni-plugins-fhgqw\" (UID: \"72516605-a894-4a62-84c7-834122a01fb8\") " pod="openshift-multus/multus-additional-cni-plugins-fhgqw"
Oct 31 17:26:39.461226 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.458937    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/72516605-a894-4a62-84c7-834122a01fb8-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-fhgqw\" (UID: \"72516605-a894-4a62-84c7-834122a01fb8\") " pod="openshift-multus/multus-additional-cni-plugins-fhgqw"
Oct 31 17:26:39.461226 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.458946    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5819fd64-9ac0-4436-b8fe-dcd3a27827ac-host-slash\") pod \"ovnkube-node-zpf4d\" (UID: \"5819fd64-9ac0-4436-b8fe-dcd3a27827ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpf4d"
Oct 31 17:26:39.461226 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.458917    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/72516605-a894-4a62-84c7-834122a01fb8-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-fhgqw\" (UID: \"72516605-a894-4a62-84c7-834122a01fb8\") " pod="openshift-multus/multus-additional-cni-plugins-fhgqw"
Oct 31 17:26:39.461226 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.458960    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/422b9ca9-3e0e-4fe0-860d-7c7d24b179fa-lib-modules\") pod \"tuned-rvsdp\" (UID: \"422b9ca9-3e0e-4fe0-860d-7c7d24b179fa\") " pod="openshift-cluster-node-tuning-operator/tuned-rvsdp"
Oct 31 17:26:39.461226 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.458974    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-59c8z\" (UniqueName: \"kubernetes.io/projected/422b9ca9-3e0e-4fe0-860d-7c7d24b179fa-kube-api-access-59c8z\") pod \"tuned-rvsdp\" (UID: \"422b9ca9-3e0e-4fe0-860d-7c7d24b179fa\") " pod="openshift-cluster-node-tuning-operator/tuned-rvsdp"
Oct 31 17:26:39.461226 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.458919    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/5819fd64-9ac0-4436-b8fe-dcd3a27827ac-env-overrides\") pod \"ovnkube-node-zpf4d\" (UID: \"5819fd64-9ac0-4436-b8fe-dcd3a27827ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpf4d"
Oct 31 17:26:39.461226 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.459044    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5819fd64-9ac0-4436-b8fe-dcd3a27827ac-host-slash\") pod \"ovnkube-node-zpf4d\" (UID: \"5819fd64-9ac0-4436-b8fe-dcd3a27827ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpf4d"
Oct 31 17:26:39.461226 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.459093    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5819fd64-9ac0-4436-b8fe-dcd3a27827ac-var-lib-openvswitch\") pod \"ovnkube-node-zpf4d\" (UID: \"5819fd64-9ac0-4436-b8fe-dcd3a27827ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpf4d"
Oct 31 17:26:39.461226 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.459157    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/422b9ca9-3e0e-4fe0-860d-7c7d24b179fa-sys\") pod \"tuned-rvsdp\" (UID: \"422b9ca9-3e0e-4fe0-860d-7c7d24b179fa\") " pod="openshift-cluster-node-tuning-operator/tuned-rvsdp"
Oct 31 17:26:39.461226 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.459218    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/72516605-a894-4a62-84c7-834122a01fb8-system-cni-dir\") pod \"multus-additional-cni-plugins-fhgqw\" (UID: \"72516605-a894-4a62-84c7-834122a01fb8\") " pod="openshift-multus/multus-additional-cni-plugins-fhgqw"
Oct 31 17:26:39.461226 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.459248    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e-multus-daemon-config\") pod \"multus-rl4hl\" (UID: \"47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e\") " pod="openshift-multus/multus-rl4hl"
Oct 31 17:26:39.461226 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.459275    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6d7781d6-8f73-4f1d-bb9a-0212e2eb6e4b-kubelet-config\") pod \"global-pull-secret-syncer-kmkdh\" (UID: \"6d7781d6-8f73-4f1d-bb9a-0212e2eb6e4b\") " pod="kube-system/global-pull-secret-syncer-kmkdh"
Oct 31 17:26:39.461226 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.459314    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5819fd64-9ac0-4436-b8fe-dcd3a27827ac-run-openvswitch\") pod \"ovnkube-node-zpf4d\" (UID: \"5819fd64-9ac0-4436-b8fe-dcd3a27827ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpf4d"
Oct 31 17:26:39.461226 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.459351    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/5819fd64-9ac0-4436-b8fe-dcd3a27827ac-node-log\") pod \"ovnkube-node-zpf4d\" (UID: \"5819fd64-9ac0-4436-b8fe-dcd3a27827ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpf4d"
Oct 31 17:26:39.461226 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.459375    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/2f646206-1186-4fcf-a9ca-fe92371d4d2b-konnectivity-ca\") pod \"konnectivity-agent-mnbmt\" (UID: \"2f646206-1186-4fcf-a9ca-fe92371d4d2b\") " pod="kube-system/konnectivity-agent-mnbmt"
Oct 31 17:26:39.461226 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.459397    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-grrfv\" (UniqueName: \"kubernetes.io/projected/b8bbaf33-7cd9-4a30-86f7-6e20ac45b0a0-kube-api-access-grrfv\") pod \"node-resolver-7pfr4\" (UID: \"b8bbaf33-7cd9-4a30-86f7-6e20ac45b0a0\") " pod="openshift-dns/node-resolver-7pfr4"
Oct 31 17:26:39.461226 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.459420    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/72516605-a894-4a62-84c7-834122a01fb8-tuning-conf-dir\") pod \"multus-additional-cni-plugins-fhgqw\" (UID: \"72516605-a894-4a62-84c7-834122a01fb8\") " pod="openshift-multus/multus-additional-cni-plugins-fhgqw"
Oct 31 17:26:39.462083 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.459443    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e-multus-cni-dir\") pod \"multus-rl4hl\" (UID: \"47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e\") " pod="openshift-multus/multus-rl4hl"
Oct 31 17:26:39.462083 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.459513    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e-multus-socket-dir-parent\") pod \"multus-rl4hl\" (UID: \"47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e\") " pod="openshift-multus/multus-rl4hl"
Oct 31 17:26:39.462083 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.459558    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e-host-run-netns\") pod \"multus-rl4hl\" (UID: \"47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e\") " pod="openshift-multus/multus-rl4hl"
Oct 31 17:26:39.462083 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.459582    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/5819fd64-9ac0-4436-b8fe-dcd3a27827ac-run-ovn\") pod \"ovnkube-node-zpf4d\" (UID: \"5819fd64-9ac0-4436-b8fe-dcd3a27827ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpf4d"
Oct 31 17:26:39.462083 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.459608    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5819fd64-9ac0-4436-b8fe-dcd3a27827ac-host-run-ovn-kubernetes\") pod \"ovnkube-node-zpf4d\" (UID: \"5819fd64-9ac0-4436-b8fe-dcd3a27827ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpf4d"
Oct 31 17:26:39.462083 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.459636    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/0f353547-2feb-42f4-aae4-b0f1a30efa34-etc-selinux\") pod \"aws-ebs-csi-driver-node-2v874\" (UID: \"0f353547-2feb-42f4-aae4-b0f1a30efa34\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2v874"
Oct 31 17:26:39.462083 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.459664    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/422b9ca9-3e0e-4fe0-860d-7c7d24b179fa-etc-sysctl-d\") pod \"tuned-rvsdp\" (UID: \"422b9ca9-3e0e-4fe0-860d-7c7d24b179fa\") " pod="openshift-cluster-node-tuning-operator/tuned-rvsdp"
Oct 31 17:26:39.462083 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.459693    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/138bfcc8-f199-4ffd-b5d2-8d83ade6199a-metrics-certs\") pod \"network-metrics-daemon-pnjdm\" (UID: \"138bfcc8-f199-4ffd-b5d2-8d83ade6199a\") " pod="openshift-multus/network-metrics-daemon-pnjdm"
Oct 31 17:26:39.462083 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.459718    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6d7781d6-8f73-4f1d-bb9a-0212e2eb6e4b-dbus\") pod \"global-pull-secret-syncer-kmkdh\" (UID: \"6d7781d6-8f73-4f1d-bb9a-0212e2eb6e4b\") " pod="kube-system/global-pull-secret-syncer-kmkdh"
Oct 31 17:26:39.462083 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.459742    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6d7781d6-8f73-4f1d-bb9a-0212e2eb6e4b-original-pull-secret\") pod \"global-pull-secret-syncer-kmkdh\" (UID: \"6d7781d6-8f73-4f1d-bb9a-0212e2eb6e4b\") " pod="kube-system/global-pull-secret-syncer-kmkdh"
Oct 31 17:26:39.462083 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.459761    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/422b9ca9-3e0e-4fe0-860d-7c7d24b179fa-host\") pod \"tuned-rvsdp\" (UID: \"422b9ca9-3e0e-4fe0-860d-7c7d24b179fa\") " pod="openshift-cluster-node-tuning-operator/tuned-rvsdp"
Oct 31 17:26:39.462083 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.459948    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c6k68\" (UniqueName: \"kubernetes.io/projected/fb9c203a-af67-4df4-88dc-74cf9eab3623-kube-api-access-c6k68\") pod \"network-check-target-tskzw\" (UID: \"fb9c203a-af67-4df4-88dc-74cf9eab3623\") " pod="openshift-network-diagnostics/network-check-target-tskzw"
Oct 31 17:26:39.462083 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.459961    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/5819fd64-9ac0-4436-b8fe-dcd3a27827ac-ovnkube-script-lib\") pod \"ovnkube-node-zpf4d\" (UID: \"5819fd64-9ac0-4436-b8fe-dcd3a27827ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpf4d"
Oct 31 17:26:39.462083 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.459979    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5819fd64-9ac0-4436-b8fe-dcd3a27827ac-host-run-netns\") pod \"ovnkube-node-zpf4d\" (UID: \"5819fd64-9ac0-4436-b8fe-dcd3a27827ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpf4d"
Oct 31 17:26:39.462083 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.460009    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b8bbaf33-7cd9-4a30-86f7-6e20ac45b0a0-tmp-dir\") pod \"node-resolver-7pfr4\" (UID: \"b8bbaf33-7cd9-4a30-86f7-6e20ac45b0a0\") " pod="openshift-dns/node-resolver-7pfr4"
Oct 31 17:26:39.462083 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.460046    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e-cnibin\") pod \"multus-rl4hl\" (UID: \"47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e\") " pod="openshift-multus/multus-rl4hl"
Oct 31 17:26:39.462083 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.460056    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6d7781d6-8f73-4f1d-bb9a-0212e2eb6e4b-dbus\") pod \"global-pull-secret-syncer-kmkdh\" (UID: \"6d7781d6-8f73-4f1d-bb9a-0212e2eb6e4b\") " pod="kube-system/global-pull-secret-syncer-kmkdh"
Oct 31 17:26:39.462083 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.460066    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e-multus-socket-dir-parent\") pod \"multus-rl4hl\" (UID: \"47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e\") " pod="openshift-multus/multus-rl4hl"
Oct 31 17:26:39.462766 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.460074    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e-host-var-lib-cni-multus\") pod \"multus-rl4hl\" (UID: \"47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e\") " pod="openshift-multus/multus-rl4hl"
Oct 31 17:26:39.462766 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.460096    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e-host-run-netns\") pod \"multus-rl4hl\" (UID: \"47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e\") " pod="openshift-multus/multus-rl4hl"
Oct 31 17:26:39.462766 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.460102    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d6ck5\" (UniqueName: \"kubernetes.io/projected/138bfcc8-f199-4ffd-b5d2-8d83ade6199a-kube-api-access-d6ck5\") pod \"network-metrics-daemon-pnjdm\" (UID: \"138bfcc8-f199-4ffd-b5d2-8d83ade6199a\") " pod="openshift-multus/network-metrics-daemon-pnjdm"
Oct 31 17:26:39.462766 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.460114    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/422b9ca9-3e0e-4fe0-860d-7c7d24b179fa-host\") pod \"tuned-rvsdp\" (UID: \"422b9ca9-3e0e-4fe0-860d-7c7d24b179fa\") " pod="openshift-cluster-node-tuning-operator/tuned-rvsdp"
Oct 31 17:26:39.462766 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.460129    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/5819fd64-9ac0-4436-b8fe-dcd3a27827ac-systemd-units\") pod \"ovnkube-node-zpf4d\" (UID: \"5819fd64-9ac0-4436-b8fe-dcd3a27827ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpf4d"
Oct 31 17:26:39.462766 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.460134    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/5819fd64-9ac0-4436-b8fe-dcd3a27827ac-run-ovn\") pod \"ovnkube-node-zpf4d\" (UID: \"5819fd64-9ac0-4436-b8fe-dcd3a27827ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpf4d"
Oct 31 17:26:39.462766 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.460160    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5819fd64-9ac0-4436-b8fe-dcd3a27827ac-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-zpf4d\" (UID: \"5819fd64-9ac0-4436-b8fe-dcd3a27827ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpf4d"
Oct 31 17:26:39.462766 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.460168    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e-multus-cni-dir\") pod \"multus-rl4hl\" (UID: \"47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e\") " pod="openshift-multus/multus-rl4hl"
Oct 31 17:26:39.462766 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.460174    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5819fd64-9ac0-4436-b8fe-dcd3a27827ac-host-run-ovn-kubernetes\") pod \"ovnkube-node-zpf4d\" (UID: \"5819fd64-9ac0-4436-b8fe-dcd3a27827ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpf4d"
Oct 31 17:26:39.462766 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.460191    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/422b9ca9-3e0e-4fe0-860d-7c7d24b179fa-etc-sysconfig\") pod \"tuned-rvsdp\" (UID: \"422b9ca9-3e0e-4fe0-860d-7c7d24b179fa\") " pod="openshift-cluster-node-tuning-operator/tuned-rvsdp"
Oct 31 17:26:39.462766 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.460261    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e-cnibin\") pod \"multus-rl4hl\" (UID: \"47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e\") " pod="openshift-multus/multus-rl4hl"
Oct 31 17:26:39.462766 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.460184    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/5819fd64-9ac0-4436-b8fe-dcd3a27827ac-node-log\") pod \"ovnkube-node-zpf4d\" (UID: \"5819fd64-9ac0-4436-b8fe-dcd3a27827ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpf4d"
Oct 31 17:26:39.462766 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.460306    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e-host-var-lib-cni-multus\") pod \"multus-rl4hl\" (UID: \"47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e\") " pod="openshift-multus/multus-rl4hl"
Oct 31 17:26:39.462766 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.460374    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6d7781d6-8f73-4f1d-bb9a-0212e2eb6e4b-kubelet-config\") pod \"global-pull-secret-syncer-kmkdh\" (UID: \"6d7781d6-8f73-4f1d-bb9a-0212e2eb6e4b\") " pod="kube-system/global-pull-secret-syncer-kmkdh"
Oct 31 17:26:39.462766 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.460424    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/422b9ca9-3e0e-4fe0-860d-7c7d24b179fa-etc-sysctl-conf\") pod \"tuned-rvsdp\" (UID: \"422b9ca9-3e0e-4fe0-860d-7c7d24b179fa\") " pod="openshift-cluster-node-tuning-operator/tuned-rvsdp"
Oct 31 17:26:39.462766 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.460461    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/5819fd64-9ac0-4436-b8fe-dcd3a27827ac-run-systemd\") pod \"ovnkube-node-zpf4d\" (UID: \"5819fd64-9ac0-4436-b8fe-dcd3a27827ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpf4d"
Oct 31 17:26:39.462766 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.460490    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5819fd64-9ac0-4436-b8fe-dcd3a27827ac-host-cni-bin\") pod \"ovnkube-node-zpf4d\" (UID: \"5819fd64-9ac0-4436-b8fe-dcd3a27827ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpf4d"
Oct 31 17:26:39.462766 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.460426    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5819fd64-9ac0-4436-b8fe-dcd3a27827ac-host-run-netns\") pod \"ovnkube-node-zpf4d\" (UID: \"5819fd64-9ac0-4436-b8fe-dcd3a27827ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpf4d"
Oct 31 17:26:39.463275 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.460552    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/5819fd64-9ac0-4436-b8fe-dcd3a27827ac-systemd-units\") pod \"ovnkube-node-zpf4d\" (UID: \"5819fd64-9ac0-4436-b8fe-dcd3a27827ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpf4d"
Oct 31 17:26:39.463275 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.460602    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/5819fd64-9ac0-4436-b8fe-dcd3a27827ac-run-systemd\") pod \"ovnkube-node-zpf4d\" (UID: \"5819fd64-9ac0-4436-b8fe-dcd3a27827ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpf4d"
Oct 31 17:26:39.463275 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.460603    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5819fd64-9ac0-4436-b8fe-dcd3a27827ac-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-zpf4d\" (UID: \"5819fd64-9ac0-4436-b8fe-dcd3a27827ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpf4d"
Oct 31 17:26:39.463275 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.460671    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/422b9ca9-3e0e-4fe0-860d-7c7d24b179fa-etc-sysconfig\") pod \"tuned-rvsdp\" (UID: \"422b9ca9-3e0e-4fe0-860d-7c7d24b179fa\") " pod="openshift-cluster-node-tuning-operator/tuned-rvsdp"
Oct 31 17:26:39.463275 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.460744    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5819fd64-9ac0-4436-b8fe-dcd3a27827ac-run-openvswitch\") pod \"ovnkube-node-zpf4d\" (UID: \"5819fd64-9ac0-4436-b8fe-dcd3a27827ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpf4d"
Oct 31 17:26:39.463275 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.460804    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/0f353547-2feb-42f4-aae4-b0f1a30efa34-socket-dir\") pod \"aws-ebs-csi-driver-node-2v874\" (UID: \"0f353547-2feb-42f4-aae4-b0f1a30efa34\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2v874"
Oct 31 17:26:39.463275 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.460967    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e-multus-daemon-config\") pod \"multus-rl4hl\" (UID: \"47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e\") " pod="openshift-multus/multus-rl4hl"
Oct 31 17:26:39.463275 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.461013    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b8bbaf33-7cd9-4a30-86f7-6e20ac45b0a0-tmp-dir\") pod \"node-resolver-7pfr4\" (UID: \"b8bbaf33-7cd9-4a30-86f7-6e20ac45b0a0\") " pod="openshift-dns/node-resolver-7pfr4"
Oct 31 17:26:39.463275 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.461023    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5819fd64-9ac0-4436-b8fe-dcd3a27827ac-host-cni-bin\") pod \"ovnkube-node-zpf4d\" (UID: \"5819fd64-9ac0-4436-b8fe-dcd3a27827ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpf4d"
Oct 31 17:26:39.463275 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.461133    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e-cni-binary-copy\") pod \"multus-rl4hl\" (UID: \"47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e\") " pod="openshift-multus/multus-rl4hl"
Oct 31 17:26:39.463275 ip-10-0-129-127 kubenswrapper[2578]: E1031 17:26:39.461154    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 31 17:26:39.463275 ip-10-0-129-127 kubenswrapper[2578]: E1031 17:26:39.461216    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 31 17:26:39.463275 ip-10-0-129-127 kubenswrapper[2578]: E1031 17:26:39.461237    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6d7781d6-8f73-4f1d-bb9a-0212e2eb6e4b-original-pull-secret podName:6d7781d6-8f73-4f1d-bb9a-0212e2eb6e4b nodeName:}" failed. No retries permitted until 2026-10-31 17:26:39.961198242 +0000 UTC m=+2.126716635 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6d7781d6-8f73-4f1d-bb9a-0212e2eb6e4b-original-pull-secret") pod "global-pull-secret-syncer-kmkdh" (UID: "6d7781d6-8f73-4f1d-bb9a-0212e2eb6e4b") : object "kube-system"/"original-pull-secret" not registered
Oct 31 17:26:39.463275 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.461244    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/422b9ca9-3e0e-4fe0-860d-7c7d24b179fa-etc-sysctl-d\") pod \"tuned-rvsdp\" (UID: \"422b9ca9-3e0e-4fe0-860d-7c7d24b179fa\") " pod="openshift-cluster-node-tuning-operator/tuned-rvsdp"
Oct 31 17:26:39.463275 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.461154    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/72516605-a894-4a62-84c7-834122a01fb8-tuning-conf-dir\") pod \"multus-additional-cni-plugins-fhgqw\" (UID: \"72516605-a894-4a62-84c7-834122a01fb8\") " pod="openshift-multus/multus-additional-cni-plugins-fhgqw"
Oct 31 17:26:39.463275 ip-10-0-129-127 kubenswrapper[2578]: E1031 17:26:39.461266    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/138bfcc8-f199-4ffd-b5d2-8d83ade6199a-metrics-certs podName:138bfcc8-f199-4ffd-b5d2-8d83ade6199a nodeName:}" failed. No retries permitted until 2026-10-31 17:26:39.961249574 +0000 UTC m=+2.126767959 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/138bfcc8-f199-4ffd-b5d2-8d83ade6199a-metrics-certs") pod "network-metrics-daemon-pnjdm" (UID: "138bfcc8-f199-4ffd-b5d2-8d83ade6199a") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 31 17:26:39.463275 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.461331    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/0f353547-2feb-42f4-aae4-b0f1a30efa34-etc-selinux\") pod \"aws-ebs-csi-driver-node-2v874\" (UID: \"0f353547-2feb-42f4-aae4-b0f1a30efa34\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2v874"
Oct 31 17:26:39.463789 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.461377    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/422b9ca9-3e0e-4fe0-860d-7c7d24b179fa-etc-sysctl-conf\") pod \"tuned-rvsdp\" (UID: \"422b9ca9-3e0e-4fe0-860d-7c7d24b179fa\") " pod="openshift-cluster-node-tuning-operator/tuned-rvsdp"
Oct 31 17:26:39.463789 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.461912    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/72516605-a894-4a62-84c7-834122a01fb8-cni-binary-copy\") pod \"multus-additional-cni-plugins-fhgqw\" (UID: \"72516605-a894-4a62-84c7-834122a01fb8\") " pod="openshift-multus/multus-additional-cni-plugins-fhgqw"
Oct 31 17:26:39.463789 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.461912    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/2f646206-1186-4fcf-a9ca-fe92371d4d2b-konnectivity-ca\") pod \"konnectivity-agent-mnbmt\" (UID: \"2f646206-1186-4fcf-a9ca-fe92371d4d2b\") " pod="kube-system/konnectivity-agent-mnbmt"
Oct 31 17:26:39.463789 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.461973    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/87e61dcd-75f6-4d3f-8ed9-7d35b06a9bb3-serviceca\") pod \"node-ca-qxnvq\" (UID: \"87e61dcd-75f6-4d3f-8ed9-7d35b06a9bb3\") " pod="openshift-image-registry/node-ca-qxnvq"
Oct 31 17:26:39.463789 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.462414    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/422b9ca9-3e0e-4fe0-860d-7c7d24b179fa-tmp\") pod \"tuned-rvsdp\" (UID: \"422b9ca9-3e0e-4fe0-860d-7c7d24b179fa\") " pod="openshift-cluster-node-tuning-operator/tuned-rvsdp"
Oct 31 17:26:39.463789 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.462433    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/422b9ca9-3e0e-4fe0-860d-7c7d24b179fa-etc-tuned\") pod \"tuned-rvsdp\" (UID: \"422b9ca9-3e0e-4fe0-860d-7c7d24b179fa\") " pod="openshift-cluster-node-tuning-operator/tuned-rvsdp"
Oct 31 17:26:39.463789 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.462721    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/2f646206-1186-4fcf-a9ca-fe92371d4d2b-agent-certs\") pod \"konnectivity-agent-mnbmt\" (UID: \"2f646206-1186-4fcf-a9ca-fe92371d4d2b\") " pod="kube-system/konnectivity-agent-mnbmt"
Oct 31 17:26:39.463789 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.463757    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/5819fd64-9ac0-4436-b8fe-dcd3a27827ac-ovn-node-metrics-cert\") pod \"ovnkube-node-zpf4d\" (UID: \"5819fd64-9ac0-4436-b8fe-dcd3a27827ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpf4d"
Oct 31 17:26:39.479951 ip-10-0-129-127 kubenswrapper[2578]: E1031 17:26:39.479926    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 31 17:26:39.479951 ip-10-0-129-127 kubenswrapper[2578]: E1031 17:26:39.479955    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 31 17:26:39.480128 ip-10-0-129-127 kubenswrapper[2578]: E1031 17:26:39.479968    2578 projected.go:196] Error preparing data for projected volume kube-api-access-c6k68 for pod openshift-network-diagnostics/network-check-target-tskzw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 31 17:26:39.480128 ip-10-0-129-127 kubenswrapper[2578]: E1031 17:26:39.480023    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fb9c203a-af67-4df4-88dc-74cf9eab3623-kube-api-access-c6k68 podName:fb9c203a-af67-4df4-88dc-74cf9eab3623 nodeName:}" failed. No retries permitted until 2026-10-31 17:26:39.980005525 +0000 UTC m=+2.145523909 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-c6k68" (UniqueName: "kubernetes.io/projected/fb9c203a-af67-4df4-88dc-74cf9eab3623-kube-api-access-c6k68") pod "network-check-target-tskzw" (UID: "fb9c203a-af67-4df4-88dc-74cf9eab3623") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 31 17:26:39.480802 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.480777    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g2n2p\" (UniqueName: \"kubernetes.io/projected/5819fd64-9ac0-4436-b8fe-dcd3a27827ac-kube-api-access-g2n2p\") pod \"ovnkube-node-zpf4d\" (UID: \"5819fd64-9ac0-4436-b8fe-dcd3a27827ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpf4d"
Oct 31 17:26:39.480802 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.481210    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xxrgs\" (UniqueName: \"kubernetes.io/projected/72516605-a894-4a62-84c7-834122a01fb8-kube-api-access-xxrgs\") pod \"multus-additional-cni-plugins-fhgqw\" (UID: \"72516605-a894-4a62-84c7-834122a01fb8\") " pod="openshift-multus/multus-additional-cni-plugins-fhgqw"
Oct 31 17:26:39.482294 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.482258    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vv9nf\" (UniqueName: \"kubernetes.io/projected/87e61dcd-75f6-4d3f-8ed9-7d35b06a9bb3-kube-api-access-vv9nf\") pod \"node-ca-qxnvq\" (UID: \"87e61dcd-75f6-4d3f-8ed9-7d35b06a9bb3\") " pod="openshift-image-registry/node-ca-qxnvq"
Oct 31 17:26:39.482407 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.482331    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-28ft5\" (UniqueName: \"kubernetes.io/projected/47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e-kube-api-access-28ft5\") pod \"multus-rl4hl\" (UID: \"47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e\") " pod="openshift-multus/multus-rl4hl"
Oct 31 17:26:39.482664 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.482645    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-59c8z\" (UniqueName: \"kubernetes.io/projected/422b9ca9-3e0e-4fe0-860d-7c7d24b179fa-kube-api-access-59c8z\") pod \"tuned-rvsdp\" (UID: \"422b9ca9-3e0e-4fe0-860d-7c7d24b179fa\") " pod="openshift-cluster-node-tuning-operator/tuned-rvsdp"
Oct 31 17:26:39.482779 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.482643    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lc2r4\" (UniqueName: \"kubernetes.io/projected/03a15842-25a2-4fbe-b418-8867070c4482-kube-api-access-lc2r4\") pod \"iptables-alerter-jzcgs\" (UID: \"03a15842-25a2-4fbe-b418-8867070c4482\") " pod="openshift-network-operator/iptables-alerter-jzcgs"
Oct 31 17:26:39.483039 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.483020    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d6ck5\" (UniqueName: \"kubernetes.io/projected/138bfcc8-f199-4ffd-b5d2-8d83ade6199a-kube-api-access-d6ck5\") pod \"network-metrics-daemon-pnjdm\" (UID: \"138bfcc8-f199-4ffd-b5d2-8d83ade6199a\") " pod="openshift-multus/network-metrics-daemon-pnjdm"
Oct 31 17:26:39.483420 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.483403    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-grrfv\" (UniqueName: \"kubernetes.io/projected/b8bbaf33-7cd9-4a30-86f7-6e20ac45b0a0-kube-api-access-grrfv\") pod \"node-resolver-7pfr4\" (UID: \"b8bbaf33-7cd9-4a30-86f7-6e20ac45b0a0\") " pod="openshift-dns/node-resolver-7pfr4"
Oct 31 17:26:39.483851 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.483834    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qfvft\" (UniqueName: \"kubernetes.io/projected/0f353547-2feb-42f4-aae4-b0f1a30efa34-kube-api-access-qfvft\") pod \"aws-ebs-csi-driver-node-2v874\" (UID: \"0f353547-2feb-42f4-aae4-b0f1a30efa34\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2v874"
Oct 31 17:26:39.506145 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:39.506119    2578 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7881d2520ea389377a82c2df6a5dd230.slice/crio-9c8e2f2a20f397a3b0cf89117bb25ddffba6af2b982702b73388a88b5fa317ec WatchSource:0}: Error finding container 9c8e2f2a20f397a3b0cf89117bb25ddffba6af2b982702b73388a88b5fa317ec: Status 404 returned error can't find the container with id 9c8e2f2a20f397a3b0cf89117bb25ddffba6af2b982702b73388a88b5fa317ec
Oct 31 17:26:39.512581 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.512565    2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 31 17:26:39.622602 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.622577    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-qxnvq"
Oct 31 17:26:39.625009 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:39.624989    2578 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5136faca2de3de7a06b0d414d9c6fca7.slice/crio-0c07b7277e444faad0e0d4df98b62cbe2593d83155d3456ed11cc10e0c1cb0a1 WatchSource:0}: Error finding container 0c07b7277e444faad0e0d4df98b62cbe2593d83155d3456ed11cc10e0c1cb0a1: Status 404 returned error can't find the container with id 0c07b7277e444faad0e0d4df98b62cbe2593d83155d3456ed11cc10e0c1cb0a1
Oct 31 17:26:39.627862 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.627837    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-7pfr4"
Oct 31 17:26:39.631601 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:39.631581    2578 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87e61dcd_75f6_4d3f_8ed9_7d35b06a9bb3.slice/crio-1f249b0a61041d3373d202f197eb2fbec975e76055493776ceb40bec5ba9e2e9 WatchSource:0}: Error finding container 1f249b0a61041d3373d202f197eb2fbec975e76055493776ceb40bec5ba9e2e9: Status 404 returned error can't find the container with id 1f249b0a61041d3373d202f197eb2fbec975e76055493776ceb40bec5ba9e2e9
Oct 31 17:26:39.638007 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:39.637986    2578 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8bbaf33_7cd9_4a30_86f7_6e20ac45b0a0.slice/crio-91c4d7126773ac299bbe247165646f3983bd101b41b642c15c218947d5a5cb20 WatchSource:0}: Error finding container 91c4d7126773ac299bbe247165646f3983bd101b41b642c15c218947d5a5cb20: Status 404 returned error can't find the container with id 91c4d7126773ac299bbe247165646f3983bd101b41b642c15c218947d5a5cb20
Oct 31 17:26:39.659331 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.659310    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-jzcgs"
Oct 31 17:26:39.668499 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:39.668480    2578 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03a15842_25a2_4fbe_b418_8867070c4482.slice/crio-6902bc06a69b2dddc4f6a14a5bb71c7bb8c616b1df22f17971bd4ac0509c5d82 WatchSource:0}: Error finding container 6902bc06a69b2dddc4f6a14a5bb71c7bb8c616b1df22f17971bd4ac0509c5d82: Status 404 returned error can't find the container with id 6902bc06a69b2dddc4f6a14a5bb71c7bb8c616b1df22f17971bd4ac0509c5d82
Oct 31 17:26:39.682242 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.682224    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-zpf4d"
Oct 31 17:26:39.684528 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.684502    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-mnbmt"
Oct 31 17:26:39.692520 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:39.692493    2578 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5819fd64_9ac0_4436_b8fe_dcd3a27827ac.slice/crio-83d5a40518f5f439cd926b8e39cca9f98e9ca3e2eec94225ef61822409f0d4f0 WatchSource:0}: Error finding container 83d5a40518f5f439cd926b8e39cca9f98e9ca3e2eec94225ef61822409f0d4f0: Status 404 returned error can't find the container with id 83d5a40518f5f439cd926b8e39cca9f98e9ca3e2eec94225ef61822409f0d4f0
Oct 31 17:26:39.696685 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:39.696664    2578 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f646206_1186_4fcf_a9ca_fe92371d4d2b.slice/crio-d7726a7ce570a8dbc898086a80c9b12b7a4bf0a9d3b410971f156c62bb1bce58 WatchSource:0}: Error finding container d7726a7ce570a8dbc898086a80c9b12b7a4bf0a9d3b410971f156c62bb1bce58: Status 404 returned error can't find the container with id d7726a7ce570a8dbc898086a80c9b12b7a4bf0a9d3b410971f156c62bb1bce58
Oct 31 17:26:39.701633 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.701609    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2v874"
Oct 31 17:26:39.703767 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.703751    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-rvsdp"
Oct 31 17:26:39.711747 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:39.711706    2578 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f353547_2feb_42f4_aae4_b0f1a30efa34.slice/crio-90cafa85c22771a4587dddc9700a96eb3a9362f1c6cb677aee45f07f3df54cfd WatchSource:0}: Error finding container 90cafa85c22771a4587dddc9700a96eb3a9362f1c6cb677aee45f07f3df54cfd: Status 404 returned error can't find the container with id 90cafa85c22771a4587dddc9700a96eb3a9362f1c6cb677aee45f07f3df54cfd
Oct 31 17:26:39.713734 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:39.713712    2578 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod422b9ca9_3e0e_4fe0_860d_7c7d24b179fa.slice/crio-379d6ab597e1cef8f11781b2f461339cdba6e171fab8d149b5895c48de687e5e WatchSource:0}: Error finding container 379d6ab597e1cef8f11781b2f461339cdba6e171fab8d149b5895c48de687e5e: Status 404 returned error can't find the container with id 379d6ab597e1cef8f11781b2f461339cdba6e171fab8d149b5895c48de687e5e
Oct 31 17:26:39.731346 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.731325    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-rl4hl"
Oct 31 17:26:39.738528 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.738511    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-fhgqw"
Oct 31 17:26:39.740178 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:39.740160    2578 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod47e8b7f2_09d2_4ac3_90c4_d0cee4f3e58e.slice/crio-ad384a3f97c70e236255e95551e8f49530e0f5a0372f0f8d575be60293cae337 WatchSource:0}: Error finding container ad384a3f97c70e236255e95551e8f49530e0f5a0372f0f8d575be60293cae337: Status 404 returned error can't find the container with id ad384a3f97c70e236255e95551e8f49530e0f5a0372f0f8d575be60293cae337
Oct 31 17:26:39.746663 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.746638    2578 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 31 17:26:39.746966 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:26:39.746950    2578 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72516605_a894_4a62_84c7_834122a01fb8.slice/crio-2e192b44eef76d29b1099e8e7c7e3953d486d35036580c92a97be1dbd93dc046 WatchSource:0}: Error finding container 2e192b44eef76d29b1099e8e7c7e3953d486d35036580c92a97be1dbd93dc046: Status 404 returned error can't find the container with id 2e192b44eef76d29b1099e8e7c7e3953d486d35036580c92a97be1dbd93dc046
Oct 31 17:26:39.964686 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.964594    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/138bfcc8-f199-4ffd-b5d2-8d83ade6199a-metrics-certs\") pod \"network-metrics-daemon-pnjdm\" (UID: \"138bfcc8-f199-4ffd-b5d2-8d83ade6199a\") " pod="openshift-multus/network-metrics-daemon-pnjdm"
Oct 31 17:26:39.964686 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:39.964654    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6d7781d6-8f73-4f1d-bb9a-0212e2eb6e4b-original-pull-secret\") pod \"global-pull-secret-syncer-kmkdh\" (UID: \"6d7781d6-8f73-4f1d-bb9a-0212e2eb6e4b\") " pod="kube-system/global-pull-secret-syncer-kmkdh"
Oct 31 17:26:39.964919 ip-10-0-129-127 kubenswrapper[2578]: E1031 17:26:39.964817    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 31 17:26:39.964919 ip-10-0-129-127 kubenswrapper[2578]: E1031 17:26:39.964872    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6d7781d6-8f73-4f1d-bb9a-0212e2eb6e4b-original-pull-secret podName:6d7781d6-8f73-4f1d-bb9a-0212e2eb6e4b nodeName:}" failed. No retries permitted until 2026-10-31 17:26:40.964857788 +0000 UTC m=+3.130376169 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6d7781d6-8f73-4f1d-bb9a-0212e2eb6e4b-original-pull-secret") pod "global-pull-secret-syncer-kmkdh" (UID: "6d7781d6-8f73-4f1d-bb9a-0212e2eb6e4b") : object "kube-system"/"original-pull-secret" not registered
Oct 31 17:26:39.965226 ip-10-0-129-127 kubenswrapper[2578]: E1031 17:26:39.965206    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 31 17:26:39.965320 ip-10-0-129-127 kubenswrapper[2578]: E1031 17:26:39.965263    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/138bfcc8-f199-4ffd-b5d2-8d83ade6199a-metrics-certs podName:138bfcc8-f199-4ffd-b5d2-8d83ade6199a nodeName:}" failed. No retries permitted until 2026-10-31 17:26:40.965250497 +0000 UTC m=+3.130768878 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/138bfcc8-f199-4ffd-b5d2-8d83ade6199a-metrics-certs") pod "network-metrics-daemon-pnjdm" (UID: "138bfcc8-f199-4ffd-b5d2-8d83ade6199a") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 31 17:26:40.015952 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:40.015923    2578 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 31 17:26:40.065430 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:40.065399    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c6k68\" (UniqueName: \"kubernetes.io/projected/fb9c203a-af67-4df4-88dc-74cf9eab3623-kube-api-access-c6k68\") pod \"network-check-target-tskzw\" (UID: \"fb9c203a-af67-4df4-88dc-74cf9eab3623\") " pod="openshift-network-diagnostics/network-check-target-tskzw"
Oct 31 17:26:40.065617 ip-10-0-129-127 kubenswrapper[2578]: E1031 17:26:40.065599    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 31 17:26:40.065703 ip-10-0-129-127 kubenswrapper[2578]: E1031 17:26:40.065623    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 31 17:26:40.065703 ip-10-0-129-127 kubenswrapper[2578]: E1031 17:26:40.065638    2578 projected.go:196] Error preparing data for projected volume kube-api-access-c6k68 for pod openshift-network-diagnostics/network-check-target-tskzw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 31 17:26:40.065703 ip-10-0-129-127 kubenswrapper[2578]: E1031 17:26:40.065702    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fb9c203a-af67-4df4-88dc-74cf9eab3623-kube-api-access-c6k68 podName:fb9c203a-af67-4df4-88dc-74cf9eab3623 nodeName:}" failed. No retries permitted until 2026-10-31 17:26:41.065675389 +0000 UTC m=+3.231193773 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-c6k68" (UniqueName: "kubernetes.io/projected/fb9c203a-af67-4df4-88dc-74cf9eab3623-kube-api-access-c6k68") pod "network-check-target-tskzw" (UID: "fb9c203a-af67-4df4-88dc-74cf9eab3623") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 31 17:26:40.321245 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:40.321173    2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-30 17:21:40 +0000 UTC" deadline="2028-03-29 17:31:20.509885251 +0000 UTC"
Oct 31 17:26:40.322076 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:40.321254    2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12360h4m40.18864632s"
Oct 31 17:26:40.392918 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:40.392890    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kmkdh"
Oct 31 17:26:40.393068 ip-10-0-129-127 kubenswrapper[2578]: E1031 17:26:40.393043    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kmkdh" podUID="6d7781d6-8f73-4f1d-bb9a-0212e2eb6e4b"
Oct 31 17:26:40.412363 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:40.412234    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-rvsdp" event={"ID":"422b9ca9-3e0e-4fe0-860d-7c7d24b179fa","Type":"ContainerStarted","Data":"379d6ab597e1cef8f11781b2f461339cdba6e171fab8d149b5895c48de687e5e"}
Oct 31 17:26:40.438511 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:40.438477    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zpf4d" event={"ID":"5819fd64-9ac0-4436-b8fe-dcd3a27827ac","Type":"ContainerStarted","Data":"83d5a40518f5f439cd926b8e39cca9f98e9ca3e2eec94225ef61822409f0d4f0"}
Oct 31 17:26:40.440868 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:40.440839    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-jzcgs" event={"ID":"03a15842-25a2-4fbe-b418-8867070c4482","Type":"ContainerStarted","Data":"6902bc06a69b2dddc4f6a14a5bb71c7bb8c616b1df22f17971bd4ac0509c5d82"}
Oct 31 17:26:40.457150 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:40.457119    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-7pfr4" event={"ID":"b8bbaf33-7cd9-4a30-86f7-6e20ac45b0a0","Type":"ContainerStarted","Data":"91c4d7126773ac299bbe247165646f3983bd101b41b642c15c218947d5a5cb20"}
Oct 31 17:26:40.461444 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:40.461415    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-qxnvq" event={"ID":"87e61dcd-75f6-4d3f-8ed9-7d35b06a9bb3","Type":"ContainerStarted","Data":"1f249b0a61041d3373d202f197eb2fbec975e76055493776ceb40bec5ba9e2e9"}
Oct 31 17:26:40.470474 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:40.470379    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-127.ec2.internal" event={"ID":"5136faca2de3de7a06b0d414d9c6fca7","Type":"ContainerStarted","Data":"0c07b7277e444faad0e0d4df98b62cbe2593d83155d3456ed11cc10e0c1cb0a1"}
Oct 31 17:26:40.486928 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:40.486837    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-127.ec2.internal" event={"ID":"7881d2520ea389377a82c2df6a5dd230","Type":"ContainerStarted","Data":"9c8e2f2a20f397a3b0cf89117bb25ddffba6af2b982702b73388a88b5fa317ec"}
Oct 31 17:26:40.499438 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:40.499403    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fhgqw" event={"ID":"72516605-a894-4a62-84c7-834122a01fb8","Type":"ContainerStarted","Data":"2e192b44eef76d29b1099e8e7c7e3953d486d35036580c92a97be1dbd93dc046"}
Oct 31 17:26:40.518505 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:40.518476    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-rl4hl" event={"ID":"47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e","Type":"ContainerStarted","Data":"ad384a3f97c70e236255e95551e8f49530e0f5a0372f0f8d575be60293cae337"}
Oct 31 17:26:40.527332 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:40.527296    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2v874" event={"ID":"0f353547-2feb-42f4-aae4-b0f1a30efa34","Type":"ContainerStarted","Data":"90cafa85c22771a4587dddc9700a96eb3a9362f1c6cb677aee45f07f3df54cfd"}
Oct 31 17:26:40.563932 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:40.563893    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-mnbmt" event={"ID":"2f646206-1186-4fcf-a9ca-fe92371d4d2b","Type":"ContainerStarted","Data":"d7726a7ce570a8dbc898086a80c9b12b7a4bf0a9d3b410971f156c62bb1bce58"}
Oct 31 17:26:40.971161 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:40.971127    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6d7781d6-8f73-4f1d-bb9a-0212e2eb6e4b-original-pull-secret\") pod \"global-pull-secret-syncer-kmkdh\" (UID: \"6d7781d6-8f73-4f1d-bb9a-0212e2eb6e4b\") " pod="kube-system/global-pull-secret-syncer-kmkdh"
Oct 31 17:26:40.971381 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:40.971206    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/138bfcc8-f199-4ffd-b5d2-8d83ade6199a-metrics-certs\") pod \"network-metrics-daemon-pnjdm\" (UID: \"138bfcc8-f199-4ffd-b5d2-8d83ade6199a\") " pod="openshift-multus/network-metrics-daemon-pnjdm"
Oct 31 17:26:40.971381 ip-10-0-129-127 kubenswrapper[2578]: E1031 17:26:40.971338    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 31 17:26:40.971508 ip-10-0-129-127 kubenswrapper[2578]: E1031 17:26:40.971392    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/138bfcc8-f199-4ffd-b5d2-8d83ade6199a-metrics-certs podName:138bfcc8-f199-4ffd-b5d2-8d83ade6199a nodeName:}" failed. No retries permitted until 2026-10-31 17:26:42.971378094 +0000 UTC m=+5.136896461 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/138bfcc8-f199-4ffd-b5d2-8d83ade6199a-metrics-certs") pod "network-metrics-daemon-pnjdm" (UID: "138bfcc8-f199-4ffd-b5d2-8d83ade6199a") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 31 17:26:40.971831 ip-10-0-129-127 kubenswrapper[2578]: E1031 17:26:40.971808    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 31 17:26:40.971900 ip-10-0-129-127 kubenswrapper[2578]: E1031 17:26:40.971870    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6d7781d6-8f73-4f1d-bb9a-0212e2eb6e4b-original-pull-secret podName:6d7781d6-8f73-4f1d-bb9a-0212e2eb6e4b nodeName:}" failed. No retries permitted until 2026-10-31 17:26:42.971854757 +0000 UTC m=+5.137373131 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6d7781d6-8f73-4f1d-bb9a-0212e2eb6e4b-original-pull-secret") pod "global-pull-secret-syncer-kmkdh" (UID: "6d7781d6-8f73-4f1d-bb9a-0212e2eb6e4b") : object "kube-system"/"original-pull-secret" not registered
Oct 31 17:26:41.072584 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:41.071759    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c6k68\" (UniqueName: \"kubernetes.io/projected/fb9c203a-af67-4df4-88dc-74cf9eab3623-kube-api-access-c6k68\") pod \"network-check-target-tskzw\" (UID: \"fb9c203a-af67-4df4-88dc-74cf9eab3623\") " pod="openshift-network-diagnostics/network-check-target-tskzw"
Oct 31 17:26:41.072584 ip-10-0-129-127 kubenswrapper[2578]: E1031 17:26:41.071896    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 31 17:26:41.072584 ip-10-0-129-127 kubenswrapper[2578]: E1031 17:26:41.071910    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 31 17:26:41.072584 ip-10-0-129-127 kubenswrapper[2578]: E1031 17:26:41.071919    2578 projected.go:196] Error preparing data for projected volume kube-api-access-c6k68 for pod openshift-network-diagnostics/network-check-target-tskzw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 31 17:26:41.072584 ip-10-0-129-127 kubenswrapper[2578]: E1031 17:26:41.071964    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fb9c203a-af67-4df4-88dc-74cf9eab3623-kube-api-access-c6k68 podName:fb9c203a-af67-4df4-88dc-74cf9eab3623 nodeName:}" failed. No retries permitted until 2026-10-31 17:26:43.071950711 +0000 UTC m=+5.237469091 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-c6k68" (UniqueName: "kubernetes.io/projected/fb9c203a-af67-4df4-88dc-74cf9eab3623-kube-api-access-c6k68") pod "network-check-target-tskzw" (UID: "fb9c203a-af67-4df4-88dc-74cf9eab3623") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 31 17:26:41.322062 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:41.321905    2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-30 17:21:40 +0000 UTC" deadline="2028-07-18 11:41:42.45732565 +0000 UTC"
Oct 31 17:26:41.322062 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:41.321965    2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15018h15m1.13536515s"
Oct 31 17:26:41.375807 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:41.375760    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tskzw"
Oct 31 17:26:41.375975 ip-10-0-129-127 kubenswrapper[2578]: E1031 17:26:41.375895    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tskzw" podUID="fb9c203a-af67-4df4-88dc-74cf9eab3623"
Oct 31 17:26:41.376291 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:41.376252    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pnjdm"
Oct 31 17:26:41.376385 ip-10-0-129-127 kubenswrapper[2578]: E1031 17:26:41.376333    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pnjdm" podUID="138bfcc8-f199-4ffd-b5d2-8d83ade6199a"
Oct 31 17:26:42.375188 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:42.375144    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kmkdh"
Oct 31 17:26:42.375798 ip-10-0-129-127 kubenswrapper[2578]: E1031 17:26:42.375282    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kmkdh" podUID="6d7781d6-8f73-4f1d-bb9a-0212e2eb6e4b"
Oct 31 17:26:42.990258 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:42.990176    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/138bfcc8-f199-4ffd-b5d2-8d83ade6199a-metrics-certs\") pod \"network-metrics-daemon-pnjdm\" (UID: \"138bfcc8-f199-4ffd-b5d2-8d83ade6199a\") " pod="openshift-multus/network-metrics-daemon-pnjdm"
Oct 31 17:26:42.990258 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:42.990232    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6d7781d6-8f73-4f1d-bb9a-0212e2eb6e4b-original-pull-secret\") pod \"global-pull-secret-syncer-kmkdh\" (UID: \"6d7781d6-8f73-4f1d-bb9a-0212e2eb6e4b\") " pod="kube-system/global-pull-secret-syncer-kmkdh"
Oct 31 17:26:42.990487 ip-10-0-129-127 kubenswrapper[2578]: E1031 17:26:42.990401    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 31 17:26:42.990487 ip-10-0-129-127 kubenswrapper[2578]: E1031 17:26:42.990463    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6d7781d6-8f73-4f1d-bb9a-0212e2eb6e4b-original-pull-secret podName:6d7781d6-8f73-4f1d-bb9a-0212e2eb6e4b nodeName:}" failed. No retries permitted until 2026-10-31 17:26:46.99044636 +0000 UTC m=+9.155964729 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6d7781d6-8f73-4f1d-bb9a-0212e2eb6e4b-original-pull-secret") pod "global-pull-secret-syncer-kmkdh" (UID: "6d7781d6-8f73-4f1d-bb9a-0212e2eb6e4b") : object "kube-system"/"original-pull-secret" not registered
Oct 31 17:26:42.990639 ip-10-0-129-127 kubenswrapper[2578]: E1031 17:26:42.990555    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 31 17:26:42.990639 ip-10-0-129-127 kubenswrapper[2578]: E1031 17:26:42.990603    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/138bfcc8-f199-4ffd-b5d2-8d83ade6199a-metrics-certs podName:138bfcc8-f199-4ffd-b5d2-8d83ade6199a nodeName:}" failed. No retries permitted until 2026-10-31 17:26:46.990587928 +0000 UTC m=+9.156106303 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/138bfcc8-f199-4ffd-b5d2-8d83ade6199a-metrics-certs") pod "network-metrics-daemon-pnjdm" (UID: "138bfcc8-f199-4ffd-b5d2-8d83ade6199a") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 31 17:26:43.091609 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:43.091558    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c6k68\" (UniqueName: \"kubernetes.io/projected/fb9c203a-af67-4df4-88dc-74cf9eab3623-kube-api-access-c6k68\") pod \"network-check-target-tskzw\" (UID: \"fb9c203a-af67-4df4-88dc-74cf9eab3623\") " pod="openshift-network-diagnostics/network-check-target-tskzw"
Oct 31 17:26:43.091778 ip-10-0-129-127 kubenswrapper[2578]: E1031 17:26:43.091733    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 31 17:26:43.091778 ip-10-0-129-127 kubenswrapper[2578]: E1031 17:26:43.091758    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 31 17:26:43.091778 ip-10-0-129-127 kubenswrapper[2578]: E1031 17:26:43.091771    2578 projected.go:196] Error preparing data for projected volume kube-api-access-c6k68 for pod openshift-network-diagnostics/network-check-target-tskzw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 31 17:26:43.091940 ip-10-0-129-127 kubenswrapper[2578]: E1031 17:26:43.091830    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fb9c203a-af67-4df4-88dc-74cf9eab3623-kube-api-access-c6k68 podName:fb9c203a-af67-4df4-88dc-74cf9eab3623 nodeName:}" failed. No retries permitted until 2026-10-31 17:26:47.091810262 +0000 UTC m=+9.257328634 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-c6k68" (UniqueName: "kubernetes.io/projected/fb9c203a-af67-4df4-88dc-74cf9eab3623-kube-api-access-c6k68") pod "network-check-target-tskzw" (UID: "fb9c203a-af67-4df4-88dc-74cf9eab3623") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 31 17:26:43.376005 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:43.375186    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tskzw"
Oct 31 17:26:43.376005 ip-10-0-129-127 kubenswrapper[2578]: E1031 17:26:43.375328    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tskzw" podUID="fb9c203a-af67-4df4-88dc-74cf9eab3623"
Oct 31 17:26:43.376005 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:43.375736    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pnjdm"
Oct 31 17:26:43.376005 ip-10-0-129-127 kubenswrapper[2578]: E1031 17:26:43.375850    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pnjdm" podUID="138bfcc8-f199-4ffd-b5d2-8d83ade6199a"
Oct 31 17:26:44.383716 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:44.383420    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kmkdh"
Oct 31 17:26:44.384238 ip-10-0-129-127 kubenswrapper[2578]: E1031 17:26:44.383835    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kmkdh" podUID="6d7781d6-8f73-4f1d-bb9a-0212e2eb6e4b"
Oct 31 17:26:44.603994 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:44.603394    2578 generic.go:356] "Generic (PLEG): container finished" podID="72516605-a894-4a62-84c7-834122a01fb8" containerID="0d5aa77f90efe1ba96132e50caed925da65017c10c5a09ad886a5e5d98c93c2d" exitCode=0
Oct 31 17:26:44.603994 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:44.603558    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fhgqw" event={"ID":"72516605-a894-4a62-84c7-834122a01fb8","Type":"ContainerDied","Data":"0d5aa77f90efe1ba96132e50caed925da65017c10c5a09ad886a5e5d98c93c2d"}
Oct 31 17:26:44.611431 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:44.610674    2578 generic.go:356] "Generic (PLEG): container finished" podID="7881d2520ea389377a82c2df6a5dd230" containerID="28df02d866c4382f7332e300ecdd5e635b5f4ff8018778ce0f4c266e6f6a9a66" exitCode=0
Oct 31 17:26:44.611431 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:44.610718    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-127.ec2.internal" event={"ID":"7881d2520ea389377a82c2df6a5dd230","Type":"ContainerDied","Data":"28df02d866c4382f7332e300ecdd5e635b5f4ff8018778ce0f4c266e6f6a9a66"}
Oct 31 17:26:45.376105 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:45.376070    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tskzw"
Oct 31 17:26:45.376303 ip-10-0-129-127 kubenswrapper[2578]: E1031 17:26:45.376207    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tskzw" podUID="fb9c203a-af67-4df4-88dc-74cf9eab3623"
Oct 31 17:26:45.376685 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:45.376655    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pnjdm"
Oct 31 17:26:45.376801 ip-10-0-129-127 kubenswrapper[2578]: E1031 17:26:45.376775    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pnjdm" podUID="138bfcc8-f199-4ffd-b5d2-8d83ade6199a"
Oct 31 17:26:45.625147 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:45.625111    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-127.ec2.internal" event={"ID":"5136faca2de3de7a06b0d414d9c6fca7","Type":"ContainerStarted","Data":"8411ddf233654f7d3c14723783cca577f7b8e71023396d89dfa6620c9f5a4700"}
Oct 31 17:26:45.632068 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:45.632010    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-127.ec2.internal" event={"ID":"7881d2520ea389377a82c2df6a5dd230","Type":"ContainerStarted","Data":"854d2a4788afed5a0ed0422ed9fce13da26d540fb8e030acb58a99a87e871139"}
Oct 31 17:26:45.637281 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:45.635078    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-mnbmt" event={"ID":"2f646206-1186-4fcf-a9ca-fe92371d4d2b","Type":"ContainerStarted","Data":"344027354cfa8b725480a9891e34bc431cab5178a7228b423b17fc32f8c5b1be"}
Oct 31 17:26:45.710592 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:45.710536    2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-127.ec2.internal" podStartSLOduration=6.710521877 podStartE2EDuration="6.710521877s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-31 17:26:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 17:26:45.710420073 +0000 UTC m=+7.875938494" watchObservedRunningTime="2026-10-31 17:26:45.710521877 +0000 UTC m=+7.876040263"
Oct 31 17:26:45.711165 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:45.711133    2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-127.ec2.internal" podStartSLOduration=6.711124149 podStartE2EDuration="6.711124149s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-31 17:26:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 17:26:45.667989967 +0000 UTC m=+7.833508357" watchObservedRunningTime="2026-10-31 17:26:45.711124149 +0000 UTC m=+7.876642538"
Oct 31 17:26:45.754702 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:45.754644    2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/konnectivity-agent-mnbmt" podStartSLOduration=2.00244432 podStartE2EDuration="6.754622577s" totalImagesPullingTime="4.752178257s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 17:26:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 17:26:45.753384335 +0000 UTC m=+7.918902727" watchObservedRunningTime="2026-10-31 17:26:45.754622577 +0000 UTC m=+7.920140967"
Oct 31 17:26:46.103200 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:46.103051    2578 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-mnbmt"
Oct 31 17:26:46.103976 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:46.103950    2578 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-mnbmt"
Oct 31 17:26:46.375701 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:46.375637    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kmkdh"
Oct 31 17:26:46.375839 ip-10-0-129-127 kubenswrapper[2578]: E1031 17:26:46.375776    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kmkdh" podUID="6d7781d6-8f73-4f1d-bb9a-0212e2eb6e4b"
Oct 31 17:26:46.640943 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:46.640872    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-qxnvq" event={"ID":"87e61dcd-75f6-4d3f-8ed9-7d35b06a9bb3","Type":"ContainerStarted","Data":"eee911b0eb878cd0f6c9fa81b6ce9929ffe511fb7add9a4bca739288cd5443d5"}
Oct 31 17:26:46.645591 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:46.645562    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2v874" event={"ID":"0f353547-2feb-42f4-aae4-b0f1a30efa34","Type":"ContainerStarted","Data":"e1276272320e5f6190fb466fdeeaa53e823cc17d189890e4d5aea92a24cb610e"}
Oct 31 17:26:46.683577 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:46.683517    2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-image-registry/node-ca-qxnvq" podStartSLOduration=1.9080221480000001 podStartE2EDuration="7.683498501s" totalImagesPullingTime="5.775476353s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-31 17:26:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 17:26:46.683486484 +0000 UTC m=+8.849004873" watchObservedRunningTime="2026-10-31 17:26:46.683498501 +0000 UTC m=+8.849016892"
Oct 31 17:26:47.029171 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:47.029084    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/138bfcc8-f199-4ffd-b5d2-8d83ade6199a-metrics-certs\") pod \"network-metrics-daemon-pnjdm\" (UID: \"138bfcc8-f199-4ffd-b5d2-8d83ade6199a\") " pod="openshift-multus/network-metrics-daemon-pnjdm"
Oct 31 17:26:47.029171 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:47.029131    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6d7781d6-8f73-4f1d-bb9a-0212e2eb6e4b-original-pull-secret\") pod \"global-pull-secret-syncer-kmkdh\" (UID: \"6d7781d6-8f73-4f1d-bb9a-0212e2eb6e4b\") " pod="kube-system/global-pull-secret-syncer-kmkdh"
Oct 31 17:26:47.029397 ip-10-0-129-127 kubenswrapper[2578]: E1031 17:26:47.029278    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 31 17:26:47.029397 ip-10-0-129-127 kubenswrapper[2578]: E1031 17:26:47.029354    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6d7781d6-8f73-4f1d-bb9a-0212e2eb6e4b-original-pull-secret podName:6d7781d6-8f73-4f1d-bb9a-0212e2eb6e4b nodeName:}" failed. No retries permitted until 2026-10-31 17:26:55.029315082 +0000 UTC m=+17.194833458 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6d7781d6-8f73-4f1d-bb9a-0212e2eb6e4b-original-pull-secret") pod "global-pull-secret-syncer-kmkdh" (UID: "6d7781d6-8f73-4f1d-bb9a-0212e2eb6e4b") : object "kube-system"/"original-pull-secret" not registered
Oct 31 17:26:47.029796 ip-10-0-129-127 kubenswrapper[2578]: E1031 17:26:47.029774    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 31 17:26:47.029902 ip-10-0-129-127 kubenswrapper[2578]: E1031 17:26:47.029833    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/138bfcc8-f199-4ffd-b5d2-8d83ade6199a-metrics-certs podName:138bfcc8-f199-4ffd-b5d2-8d83ade6199a nodeName:}" failed. No retries permitted until 2026-10-31 17:26:55.029816445 +0000 UTC m=+17.195334816 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/138bfcc8-f199-4ffd-b5d2-8d83ade6199a-metrics-certs") pod "network-metrics-daemon-pnjdm" (UID: "138bfcc8-f199-4ffd-b5d2-8d83ade6199a") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 31 17:26:47.130281 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:47.130249    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c6k68\" (UniqueName: \"kubernetes.io/projected/fb9c203a-af67-4df4-88dc-74cf9eab3623-kube-api-access-c6k68\") pod \"network-check-target-tskzw\" (UID: \"fb9c203a-af67-4df4-88dc-74cf9eab3623\") " pod="openshift-network-diagnostics/network-check-target-tskzw"
Oct 31 17:26:47.130496 ip-10-0-129-127 kubenswrapper[2578]: E1031 17:26:47.130438    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 31 17:26:47.130496 ip-10-0-129-127 kubenswrapper[2578]: E1031 17:26:47.130467    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 31 17:26:47.130635 ip-10-0-129-127 kubenswrapper[2578]: E1031 17:26:47.130505    2578 projected.go:196] Error preparing data for projected volume kube-api-access-c6k68 for pod openshift-network-diagnostics/network-check-target-tskzw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 31 17:26:47.130635 ip-10-0-129-127 kubenswrapper[2578]: E1031 17:26:47.130570    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fb9c203a-af67-4df4-88dc-74cf9eab3623-kube-api-access-c6k68 podName:fb9c203a-af67-4df4-88dc-74cf9eab3623 nodeName:}" failed. No retries permitted until 2026-10-31 17:26:55.130550639 +0000 UTC m=+17.296069006 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-c6k68" (UniqueName: "kubernetes.io/projected/fb9c203a-af67-4df4-88dc-74cf9eab3623-kube-api-access-c6k68") pod "network-check-target-tskzw" (UID: "fb9c203a-af67-4df4-88dc-74cf9eab3623") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 31 17:26:47.376202 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:47.375480    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tskzw"
Oct 31 17:26:47.376202 ip-10-0-129-127 kubenswrapper[2578]: E1031 17:26:47.375621    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tskzw" podUID="fb9c203a-af67-4df4-88dc-74cf9eab3623"
Oct 31 17:26:47.376202 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:47.375820    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pnjdm"
Oct 31 17:26:47.376202 ip-10-0-129-127 kubenswrapper[2578]: E1031 17:26:47.375922    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pnjdm" podUID="138bfcc8-f199-4ffd-b5d2-8d83ade6199a"
Oct 31 17:26:47.647993 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:47.647918    2578 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 31 17:26:48.204710 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:48.204634    2578 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Oct 31 17:26:48.316791 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:48.316675    2578 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-10-31T17:26:48.204707944Z","UUID":"eb878fe0-0ab6-49b1-a00d-32f7f1cb5bb7","Handler":null,"Name":"","Endpoint":""}
Oct 31 17:26:48.320007 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:48.319976    2578 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Oct 31 17:26:48.320208 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:48.320062    2578 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Oct 31 17:26:48.376251 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:48.376215    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kmkdh"
Oct 31 17:26:48.376425 ip-10-0-129-127 kubenswrapper[2578]: E1031 17:26:48.376339    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kmkdh" podUID="6d7781d6-8f73-4f1d-bb9a-0212e2eb6e4b"
Oct 31 17:26:48.653953 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:48.653916    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-rl4hl" event={"ID":"47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e","Type":"ContainerStarted","Data":"63f5ee2b8fce00c96664f82ca9658724607be009125d64af2a76fa16a9f3ed90"}
Oct 31 17:26:48.657625 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:48.657593    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2v874" event={"ID":"0f353547-2feb-42f4-aae4-b0f1a30efa34","Type":"ContainerStarted","Data":"c4e257ef6bcfd06f55274f1213111e138394740281bd4cd835e5ca25e8a5966c"}
Oct 31 17:26:49.377774 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:49.377740    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tskzw"
Oct 31 17:26:49.377940 ip-10-0-129-127 kubenswrapper[2578]: E1031 17:26:49.377882    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tskzw" podUID="fb9c203a-af67-4df4-88dc-74cf9eab3623"
Oct 31 17:26:49.378314 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:49.378288    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pnjdm"
Oct 31 17:26:49.378442 ip-10-0-129-127 kubenswrapper[2578]: E1031 17:26:49.378419    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pnjdm" podUID="138bfcc8-f199-4ffd-b5d2-8d83ade6199a"
Oct 31 17:26:49.664118 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:49.664077    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2v874" event={"ID":"0f353547-2feb-42f4-aae4-b0f1a30efa34","Type":"ContainerStarted","Data":"83d31c954670fe5d0c2553b3aee136aedc1f2922129b6a33679b36154df8d235"}
Oct 31 17:26:49.666362 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:49.666327    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-7pfr4" event={"ID":"b8bbaf33-7cd9-4a30-86f7-6e20ac45b0a0","Type":"ContainerStarted","Data":"5d2b83adc7b5f11cb0d1358cc8eae65209b86c0397d62567316e24f312495a86"}
Oct 31 17:26:49.668569 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:49.668482    2578 generic.go:356] "Generic (PLEG): container finished" podID="72516605-a894-4a62-84c7-834122a01fb8" containerID="d8271d4a0a9d9c44e5c182ff2acd03ee9b9c0f6a5f54249a839dcbdba7db8502" exitCode=0
Oct 31 17:26:49.669505 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:49.669480    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fhgqw" event={"ID":"72516605-a894-4a62-84c7-834122a01fb8","Type":"ContainerDied","Data":"d8271d4a0a9d9c44e5c182ff2acd03ee9b9c0f6a5f54249a839dcbdba7db8502"}
Oct 31 17:26:49.710866 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:49.710722    2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2v874" podStartSLOduration=1.5451231060000001 podStartE2EDuration="10.710710532s" totalImagesPullingTime="9.165587426s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-31 17:26:39 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 17:26:49.710698395 +0000 UTC m=+11.876216784" watchObservedRunningTime="2026-10-31 17:26:49.710710532 +0000 UTC m=+11.876228921"
Oct 31 17:26:49.711082 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:49.711042    2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-rl4hl" podStartSLOduration=2.030456252 podStartE2EDuration="10.711036793s" totalImagesPullingTime="8.680580541s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-31 17:26:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 17:26:48.700408225 +0000 UTC m=+10.865926615" watchObservedRunningTime="2026-10-31 17:26:49.711036793 +0000 UTC m=+11.876555182"
Oct 31 17:26:49.802464 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:49.802397    2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-dns/node-resolver-7pfr4" podStartSLOduration=1.973480449 podStartE2EDuration="10.802382092s" totalImagesPullingTime="8.828901643s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-31 17:26:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 17:26:49.802109584 +0000 UTC m=+11.967627976" watchObservedRunningTime="2026-10-31 17:26:49.802382092 +0000 UTC m=+11.967900481"
Oct 31 17:26:50.375321 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:50.375088    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kmkdh"
Oct 31 17:26:50.375477 ip-10-0-129-127 kubenswrapper[2578]: E1031 17:26:50.375444    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kmkdh" podUID="6d7781d6-8f73-4f1d-bb9a-0212e2eb6e4b"
Oct 31 17:26:50.673617 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:50.673576    2578 generic.go:356] "Generic (PLEG): container finished" podID="72516605-a894-4a62-84c7-834122a01fb8" containerID="ccc7e8c6c149b9d72e46ba296d41e8cf0e76c37142c75a5252e405880139a908" exitCode=0
Oct 31 17:26:50.674092 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:50.673667    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fhgqw" event={"ID":"72516605-a894-4a62-84c7-834122a01fb8","Type":"ContainerDied","Data":"ccc7e8c6c149b9d72e46ba296d41e8cf0e76c37142c75a5252e405880139a908"}
Oct 31 17:26:50.676246 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:50.676216    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-jzcgs" event={"ID":"03a15842-25a2-4fbe-b418-8867070c4482","Type":"ContainerStarted","Data":"77fae45b6a3562e4697a445427eb345ec86e2dd23397f5fe1cf2ad9126336256"}
Oct 31 17:26:50.711692 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:50.711620    2578 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-mnbmt"
Oct 31 17:26:50.711792 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:50.711730    2578 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 31 17:26:50.712161 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:50.712140    2578 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-mnbmt"
Oct 31 17:26:50.778325 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:50.778277    2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-jzcgs" podStartSLOduration=2.5955979449999997 podStartE2EDuration="11.778263694s" totalImagesPullingTime="9.182665749s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-31 17:26:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 17:26:50.777917579 +0000 UTC m=+12.943435971" watchObservedRunningTime="2026-10-31 17:26:50.778263694 +0000 UTC m=+12.943782085"
Oct 31 17:26:51.375787 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:51.375704    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tskzw"
Oct 31 17:26:51.375978 ip-10-0-129-127 kubenswrapper[2578]: E1031 17:26:51.375869    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tskzw" podUID="fb9c203a-af67-4df4-88dc-74cf9eab3623"
Oct 31 17:26:51.375978 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:51.375907    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pnjdm"
Oct 31 17:26:51.376104 ip-10-0-129-127 kubenswrapper[2578]: E1031 17:26:51.375995    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pnjdm" podUID="138bfcc8-f199-4ffd-b5d2-8d83ade6199a"
Oct 31 17:26:51.680947 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:51.680916    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fhgqw" event={"ID":"72516605-a894-4a62-84c7-834122a01fb8","Type":"ContainerStarted","Data":"b9c7fd4595e7225d0dc5f6e8b3774db596f0e12f6d0e25b47955b86aff870774"}
Oct 31 17:26:51.682688 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:51.682655    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-rvsdp" event={"ID":"422b9ca9-3e0e-4fe0-860d-7c7d24b179fa","Type":"ContainerStarted","Data":"6c4bc6397f92cf2c156bd09049c66c6627e5c974f06b78095f88f7f8d4008954"}
Oct 31 17:26:52.378390 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:52.378366    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kmkdh"
Oct 31 17:26:52.378576 ip-10-0-129-127 kubenswrapper[2578]: E1031 17:26:52.378495    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kmkdh" podUID="6d7781d6-8f73-4f1d-bb9a-0212e2eb6e4b"
Oct 31 17:26:52.687164 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:52.687081    2578 generic.go:356] "Generic (PLEG): container finished" podID="72516605-a894-4a62-84c7-834122a01fb8" containerID="b9c7fd4595e7225d0dc5f6e8b3774db596f0e12f6d0e25b47955b86aff870774" exitCode=0
Oct 31 17:26:52.687683 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:52.687174    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fhgqw" event={"ID":"72516605-a894-4a62-84c7-834122a01fb8","Type":"ContainerDied","Data":"b9c7fd4595e7225d0dc5f6e8b3774db596f0e12f6d0e25b47955b86aff870774"}
Oct 31 17:26:52.760203 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:52.760159    2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-rvsdp" podStartSLOduration=2.390493111 podStartE2EDuration="13.760144849s" totalImagesPullingTime="11.369651738s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-31 17:26:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 17:26:51.765469319 +0000 UTC m=+13.930987706" watchObservedRunningTime="2026-10-31 17:26:52.760144849 +0000 UTC m=+14.925663235"
Oct 31 17:26:53.375687 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:53.375657    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pnjdm"
Oct 31 17:26:53.375687 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:53.375690    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tskzw"
Oct 31 17:26:53.375932 ip-10-0-129-127 kubenswrapper[2578]: E1031 17:26:53.375798    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pnjdm" podUID="138bfcc8-f199-4ffd-b5d2-8d83ade6199a"
Oct 31 17:26:53.375932 ip-10-0-129-127 kubenswrapper[2578]: E1031 17:26:53.375908    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tskzw" podUID="fb9c203a-af67-4df4-88dc-74cf9eab3623"
Oct 31 17:26:54.376225 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:54.375783    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kmkdh"
Oct 31 17:26:54.376225 ip-10-0-129-127 kubenswrapper[2578]: E1031 17:26:54.375900    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kmkdh" podUID="6d7781d6-8f73-4f1d-bb9a-0212e2eb6e4b"
Oct 31 17:26:54.694665 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:54.694370    2578 generic.go:356] "Generic (PLEG): container finished" podID="72516605-a894-4a62-84c7-834122a01fb8" containerID="26459fede7cfba8105fa171964839c80920c85a67e4f5d5437a00c4979361ac4" exitCode=0
Oct 31 17:26:54.694665 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:54.694436    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fhgqw" event={"ID":"72516605-a894-4a62-84c7-834122a01fb8","Type":"ContainerDied","Data":"26459fede7cfba8105fa171964839c80920c85a67e4f5d5437a00c4979361ac4"}
Oct 31 17:26:55.096055 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:55.096020    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/138bfcc8-f199-4ffd-b5d2-8d83ade6199a-metrics-certs\") pod \"network-metrics-daemon-pnjdm\" (UID: \"138bfcc8-f199-4ffd-b5d2-8d83ade6199a\") " pod="openshift-multus/network-metrics-daemon-pnjdm"
Oct 31 17:26:55.096199 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:55.096065    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6d7781d6-8f73-4f1d-bb9a-0212e2eb6e4b-original-pull-secret\") pod \"global-pull-secret-syncer-kmkdh\" (UID: \"6d7781d6-8f73-4f1d-bb9a-0212e2eb6e4b\") " pod="kube-system/global-pull-secret-syncer-kmkdh"
Oct 31 17:26:55.096199 ip-10-0-129-127 kubenswrapper[2578]: E1031 17:26:55.096178    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 31 17:26:55.096260 ip-10-0-129-127 kubenswrapper[2578]: E1031 17:26:55.096239    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/138bfcc8-f199-4ffd-b5d2-8d83ade6199a-metrics-certs podName:138bfcc8-f199-4ffd-b5d2-8d83ade6199a nodeName:}" failed. No retries permitted until 2026-10-31 17:27:11.096224619 +0000 UTC m=+33.261742983 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/138bfcc8-f199-4ffd-b5d2-8d83ade6199a-metrics-certs") pod "network-metrics-daemon-pnjdm" (UID: "138bfcc8-f199-4ffd-b5d2-8d83ade6199a") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 31 17:26:55.096301 ip-10-0-129-127 kubenswrapper[2578]: E1031 17:26:55.096183    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 31 17:26:55.096334 ip-10-0-129-127 kubenswrapper[2578]: E1031 17:26:55.096301    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6d7781d6-8f73-4f1d-bb9a-0212e2eb6e4b-original-pull-secret podName:6d7781d6-8f73-4f1d-bb9a-0212e2eb6e4b nodeName:}" failed. No retries permitted until 2026-10-31 17:27:11.096290929 +0000 UTC m=+33.261809292 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6d7781d6-8f73-4f1d-bb9a-0212e2eb6e4b-original-pull-secret") pod "global-pull-secret-syncer-kmkdh" (UID: "6d7781d6-8f73-4f1d-bb9a-0212e2eb6e4b") : object "kube-system"/"original-pull-secret" not registered
Oct 31 17:26:55.196367 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:55.196342    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c6k68\" (UniqueName: \"kubernetes.io/projected/fb9c203a-af67-4df4-88dc-74cf9eab3623-kube-api-access-c6k68\") pod \"network-check-target-tskzw\" (UID: \"fb9c203a-af67-4df4-88dc-74cf9eab3623\") " pod="openshift-network-diagnostics/network-check-target-tskzw"
Oct 31 17:26:55.196513 ip-10-0-129-127 kubenswrapper[2578]: E1031 17:26:55.196489    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 31 17:26:55.196513 ip-10-0-129-127 kubenswrapper[2578]: E1031 17:26:55.196519    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 31 17:26:55.196641 ip-10-0-129-127 kubenswrapper[2578]: E1031 17:26:55.196531    2578 projected.go:196] Error preparing data for projected volume kube-api-access-c6k68 for pod openshift-network-diagnostics/network-check-target-tskzw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 31 17:26:55.196641 ip-10-0-129-127 kubenswrapper[2578]: E1031 17:26:55.196586    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fb9c203a-af67-4df4-88dc-74cf9eab3623-kube-api-access-c6k68 podName:fb9c203a-af67-4df4-88dc-74cf9eab3623 nodeName:}" failed. No retries permitted until 2026-10-31 17:27:11.196567775 +0000 UTC m=+33.362086154 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-c6k68" (UniqueName: "kubernetes.io/projected/fb9c203a-af67-4df4-88dc-74cf9eab3623-kube-api-access-c6k68") pod "network-check-target-tskzw" (UID: "fb9c203a-af67-4df4-88dc-74cf9eab3623") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 31 17:26:55.375933 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:55.375910    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tskzw"
Oct 31 17:26:55.376020 ip-10-0-129-127 kubenswrapper[2578]: E1031 17:26:55.376000    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tskzw" podUID="fb9c203a-af67-4df4-88dc-74cf9eab3623"
Oct 31 17:26:55.376090 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:55.376005    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pnjdm"
Oct 31 17:26:55.376090 ip-10-0-129-127 kubenswrapper[2578]: E1031 17:26:55.376084    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pnjdm" podUID="138bfcc8-f199-4ffd-b5d2-8d83ade6199a"
Oct 31 17:26:55.699523 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:55.699316    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zpf4d_5819fd64-9ac0-4436-b8fe-dcd3a27827ac/ovn-acl-logging/0.log"
Oct 31 17:26:55.699916 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:55.699788    2578 generic.go:356] "Generic (PLEG): container finished" podID="5819fd64-9ac0-4436-b8fe-dcd3a27827ac" containerID="36abb598a1b9a73459f99958a8a088146210713d7eccd28e691c9b3db452839f" exitCode=1
Oct 31 17:26:55.699916 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:55.699851    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zpf4d" event={"ID":"5819fd64-9ac0-4436-b8fe-dcd3a27827ac","Type":"ContainerStarted","Data":"f1d68c1bb956c6667da6ed2293a48bcfe4c44806c655c3f9f560568885360a48"}
Oct 31 17:26:55.699983 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:55.699928    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zpf4d" event={"ID":"5819fd64-9ac0-4436-b8fe-dcd3a27827ac","Type":"ContainerStarted","Data":"95cac0c879ee7d4114ae40aa6d8bf84873c61d595c2b1a6f335dc22c49628467"}
Oct 31 17:26:55.699983 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:55.699978    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zpf4d" event={"ID":"5819fd64-9ac0-4436-b8fe-dcd3a27827ac","Type":"ContainerStarted","Data":"45f9ef716eb2c5b11b773ab536ac5cde12f5cbee29e9bf34cfaaede83363289b"}
Oct 31 17:26:55.700073 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:55.700025    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zpf4d" event={"ID":"5819fd64-9ac0-4436-b8fe-dcd3a27827ac","Type":"ContainerStarted","Data":"7d77995c2e708cc96be83d3612a13776b1d859e6d851b00bf93fff009d3d3202"}
Oct 31 17:26:55.700105 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:55.700070    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zpf4d" event={"ID":"5819fd64-9ac0-4436-b8fe-dcd3a27827ac","Type":"ContainerDied","Data":"36abb598a1b9a73459f99958a8a088146210713d7eccd28e691c9b3db452839f"}
Oct 31 17:26:55.700145 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:55.700117    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zpf4d" event={"ID":"5819fd64-9ac0-4436-b8fe-dcd3a27827ac","Type":"ContainerStarted","Data":"01c5e6a4c228252c75a70cd3b20aa19ca5ca8ace7d3c8d7551f5e166a75dae1a"}
Oct 31 17:26:55.702949 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:55.702927    2578 generic.go:356] "Generic (PLEG): container finished" podID="72516605-a894-4a62-84c7-834122a01fb8" containerID="efce52614641091eab395134ba4291921b92744395442faa68c5c105fea5d93b" exitCode=0
Oct 31 17:26:55.703040 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:55.702959    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fhgqw" event={"ID":"72516605-a894-4a62-84c7-834122a01fb8","Type":"ContainerDied","Data":"efce52614641091eab395134ba4291921b92744395442faa68c5c105fea5d93b"}
Oct 31 17:26:56.378149 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:56.378122    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kmkdh"
Oct 31 17:26:56.378288 ip-10-0-129-127 kubenswrapper[2578]: E1031 17:26:56.378212    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kmkdh" podUID="6d7781d6-8f73-4f1d-bb9a-0212e2eb6e4b"
Oct 31 17:26:56.708550 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:56.708459    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fhgqw" event={"ID":"72516605-a894-4a62-84c7-834122a01fb8","Type":"ContainerStarted","Data":"530b8b57ae95e1dc828bf3df76d7c558125eacec69c236c5571a2f7a277a9219"}
Oct 31 17:26:56.782386 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:56.782333    2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-fhgqw" podStartSLOduration=7.098381539 podStartE2EDuration="17.782321699s" totalImagesPullingTime="10.68394016s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-31 17:26:39 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 17:26:56.767929709 +0000 UTC m=+18.933448095" watchObservedRunningTime="2026-10-31 17:26:56.782321699 +0000 UTC m=+18.947840081"
Oct 31 17:26:57.376003 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:57.375973    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pnjdm"
Oct 31 17:26:57.376146 ip-10-0-129-127 kubenswrapper[2578]: E1031 17:26:57.376068    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pnjdm" podUID="138bfcc8-f199-4ffd-b5d2-8d83ade6199a"
Oct 31 17:26:57.376146 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:57.376123    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tskzw"
Oct 31 17:26:57.376228 ip-10-0-129-127 kubenswrapper[2578]: E1031 17:26:57.376198    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tskzw" podUID="fb9c203a-af67-4df4-88dc-74cf9eab3623"
Oct 31 17:26:57.714034 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:57.714011    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zpf4d_5819fd64-9ac0-4436-b8fe-dcd3a27827ac/ovn-acl-logging/0.log"
Oct 31 17:26:57.714554 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:57.714476    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zpf4d" event={"ID":"5819fd64-9ac0-4436-b8fe-dcd3a27827ac","Type":"ContainerStarted","Data":"2e338cc0517d08fecb2c57c7f6ea7610f70d53c8d2d721f023a0c32c25f62898"}
Oct 31 17:26:58.378467 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:58.378149    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kmkdh"
Oct 31 17:26:58.378619 ip-10-0-129-127 kubenswrapper[2578]: E1031 17:26:58.378593    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kmkdh" podUID="6d7781d6-8f73-4f1d-bb9a-0212e2eb6e4b"
Oct 31 17:26:59.375357 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:59.375329    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pnjdm"
Oct 31 17:26:59.375357 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:26:59.375360    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tskzw"
Oct 31 17:26:59.375791 ip-10-0-129-127 kubenswrapper[2578]: E1031 17:26:59.375486    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pnjdm" podUID="138bfcc8-f199-4ffd-b5d2-8d83ade6199a"
Oct 31 17:26:59.375791 ip-10-0-129-127 kubenswrapper[2578]: E1031 17:26:59.375608    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tskzw" podUID="fb9c203a-af67-4df4-88dc-74cf9eab3623"
Oct 31 17:27:00.375746 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:00.375717    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kmkdh"
Oct 31 17:27:00.376340 ip-10-0-129-127 kubenswrapper[2578]: E1031 17:27:00.375838    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kmkdh" podUID="6d7781d6-8f73-4f1d-bb9a-0212e2eb6e4b"
Oct 31 17:27:00.723461 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:00.723376    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zpf4d_5819fd64-9ac0-4436-b8fe-dcd3a27827ac/ovn-acl-logging/0.log"
Oct 31 17:27:00.723772 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:00.723750    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zpf4d" event={"ID":"5819fd64-9ac0-4436-b8fe-dcd3a27827ac","Type":"ContainerStarted","Data":"cf4e29a5031a8f0e7f47fa50700bd876b0bf12905246d823e7ffe5cb75898159"}
Oct 31 17:27:00.724013 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:00.723996    2578 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-zpf4d"
Oct 31 17:27:00.724089 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:00.724079    2578 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-zpf4d"
Oct 31 17:27:00.724188 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:00.724174    2578 scope.go:122] "RemoveContainer" containerID="36abb598a1b9a73459f99958a8a088146210713d7eccd28e691c9b3db452839f"
Oct 31 17:27:00.741134 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:00.741113    2578 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-zpf4d"
Oct 31 17:27:00.832774 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:00.832735    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:27:01.003242 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:01.002993    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:27:01.375412 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:01.375386    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tskzw"
Oct 31 17:27:01.375597 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:01.375421    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pnjdm"
Oct 31 17:27:01.375597 ip-10-0-129-127 kubenswrapper[2578]: E1031 17:27:01.375530    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tskzw" podUID="fb9c203a-af67-4df4-88dc-74cf9eab3623"
Oct 31 17:27:01.375708 ip-10-0-129-127 kubenswrapper[2578]: E1031 17:27:01.375651    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pnjdm" podUID="138bfcc8-f199-4ffd-b5d2-8d83ade6199a"
Oct 31 17:27:01.730525 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:01.730468    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zpf4d_5819fd64-9ac0-4436-b8fe-dcd3a27827ac/ovn-acl-logging/0.log"
Oct 31 17:27:01.730914 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:01.730888    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zpf4d" event={"ID":"5819fd64-9ac0-4436-b8fe-dcd3a27827ac","Type":"ContainerStarted","Data":"f07a370b15e0d5f72866790b0f3412d8d7c3fd52f13b491c679caeb1dfe85c4f"}
Oct 31 17:27:01.731183 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:01.731160    2578 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-zpf4d"
Oct 31 17:27:01.745700 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:01.745682    2578 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-zpf4d"
Oct 31 17:27:01.781618 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:01.781591    2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-kmkdh"]
Oct 31 17:27:01.782041 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:01.781711    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kmkdh"
Oct 31 17:27:01.782150 ip-10-0-129-127 kubenswrapper[2578]: E1031 17:27:01.782125    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kmkdh" podUID="6d7781d6-8f73-4f1d-bb9a-0212e2eb6e4b"
Oct 31 17:27:01.784587 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:01.784552    2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-pnjdm"]
Oct 31 17:27:01.784684 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:01.784639    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pnjdm"
Oct 31 17:27:01.784745 ip-10-0-129-127 kubenswrapper[2578]: E1031 17:27:01.784726    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pnjdm" podUID="138bfcc8-f199-4ffd-b5d2-8d83ade6199a"
Oct 31 17:27:01.785023 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:01.785007    2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-tskzw"]
Oct 31 17:27:01.785080 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:01.785070    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tskzw"
Oct 31 17:27:01.785151 ip-10-0-129-127 kubenswrapper[2578]: E1031 17:27:01.785137    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tskzw" podUID="fb9c203a-af67-4df4-88dc-74cf9eab3623"
Oct 31 17:27:01.795044 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:01.795006    2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-zpf4d" podStartSLOduration=7.342160135 podStartE2EDuration="22.794995712s" totalImagesPullingTime="15.452835577s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-31 17:26:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 17:27:01.792566003 +0000 UTC m=+23.958084388" watchObservedRunningTime="2026-10-31 17:27:01.794995712 +0000 UTC m=+23.960514088"
Oct 31 17:27:03.225634 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:03.225598    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:27:03.331226 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:03.331198    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:27:03.375434 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:03.375412    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pnjdm"
Oct 31 17:27:03.375570 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:03.375452    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kmkdh"
Oct 31 17:27:03.375570 ip-10-0-129-127 kubenswrapper[2578]: E1031 17:27:03.375527    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pnjdm" podUID="138bfcc8-f199-4ffd-b5d2-8d83ade6199a"
Oct 31 17:27:03.375646 ip-10-0-129-127 kubenswrapper[2578]: E1031 17:27:03.375606    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kmkdh" podUID="6d7781d6-8f73-4f1d-bb9a-0212e2eb6e4b"
Oct 31 17:27:03.375646 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:03.375640    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tskzw"
Oct 31 17:27:03.375708 ip-10-0-129-127 kubenswrapper[2578]: E1031 17:27:03.375692    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tskzw" podUID="fb9c203a-af67-4df4-88dc-74cf9eab3623"
Oct 31 17:27:05.375800 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:05.375767    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pnjdm"
Oct 31 17:27:05.375800 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:05.375794    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tskzw"
Oct 31 17:27:05.376286 ip-10-0-129-127 kubenswrapper[2578]: E1031 17:27:05.375866    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pnjdm" podUID="138bfcc8-f199-4ffd-b5d2-8d83ade6199a"
Oct 31 17:27:05.376286 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:05.375913    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kmkdh"
Oct 31 17:27:05.376286 ip-10-0-129-127 kubenswrapper[2578]: E1031 17:27:05.376017    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kmkdh" podUID="6d7781d6-8f73-4f1d-bb9a-0212e2eb6e4b"
Oct 31 17:27:05.376286 ip-10-0-129-127 kubenswrapper[2578]: E1031 17:27:05.376096    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tskzw" podUID="fb9c203a-af67-4df4-88dc-74cf9eab3623"
Oct 31 17:27:06.763248 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:06.763219    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:27:06.883568 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:06.883536    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:27:06.991483 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:06.991402    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:27:07.107163 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:07.107135    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:27:07.124574 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:07.124555    2578 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-127.ec2.internal" event="NodeReady"
Oct 31 17:27:07.124666 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:07.124648    2578 kubelet_node_status.go:477] "Fast updating node status as it just became ready"
Oct 31 17:27:07.177783 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:07.177763    2578 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-tg5qk"]
Oct 31 17:27:07.180376 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:07.180357    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-tg5qk"
Oct 31 17:27:07.180670 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:07.180636    2578 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-59fpc"]
Oct 31 17:27:07.183348 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:07.183330    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-59fpc"
Oct 31 17:27:07.196346 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:07.196329    2578 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-b5ln2"]
Oct 31 17:27:07.199259 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:07.199246    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-b5ln2"
Oct 31 17:27:07.201990 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:07.201971    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Oct 31 17:27:07.202078 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:07.202036    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Oct 31 17:27:07.202078 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:07.202069    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Oct 31 17:27:07.202180 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:07.202102    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Oct 31 17:27:07.202226 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:07.202211    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-dm9tc\""
Oct 31 17:27:07.202965 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:07.202945    2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-tg5qk"]
Oct 31 17:27:07.205573 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:07.205467    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Oct 31 17:27:07.210667 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:07.210650    2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-59fpc"]
Oct 31 17:27:07.222686 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:07.222666    2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-b5ln2"]
Oct 31 17:27:07.241013 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:07.240975    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:27:07.247250 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:07.247207    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Oct 31 17:27:07.247250 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:07.247208    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Oct 31 17:27:07.247363 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:07.247272    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-8gjvp\""
Oct 31 17:27:07.247363 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:07.247273    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-4g54t\""
Oct 31 17:27:07.287477 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:07.287455    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z8zhc\" (UniqueName: \"kubernetes.io/projected/9ee82620-6497-48e8-bb2c-1bb77a93f7ef-kube-api-access-z8zhc\") pod \"dns-default-59fpc\" (UID: \"9ee82620-6497-48e8-bb2c-1bb77a93f7ef\") " pod="openshift-dns/dns-default-59fpc"
Oct 31 17:27:07.287558 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:07.287484    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9ee82620-6497-48e8-bb2c-1bb77a93f7ef-tmp-dir\") pod \"dns-default-59fpc\" (UID: \"9ee82620-6497-48e8-bb2c-1bb77a93f7ef\") " pod="openshift-dns/dns-default-59fpc"
Oct 31 17:27:07.287558 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:07.287502    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/9ee82620-6497-48e8-bb2c-1bb77a93f7ef-config-volume\") pod \"dns-default-59fpc\" (UID: \"9ee82620-6497-48e8-bb2c-1bb77a93f7ef\") " pod="openshift-dns/dns-default-59fpc"
Oct 31 17:27:07.287558 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:07.287518    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/22096c98-1b93-4ea4-9a6f-6383df9eafe4-cert\") pod \"ingress-canary-b5ln2\" (UID: \"22096c98-1b93-4ea4-9a6f-6383df9eafe4\") " pod="openshift-ingress-canary/ingress-canary-b5ln2"
Oct 31 17:27:07.287665 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:07.287613    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/61462533-417a-4fbc-bd85-c56a0112db31-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tg5qk\" (UID: \"61462533-417a-4fbc-bd85-c56a0112db31\") " pod="openshift-insights/insights-runtime-extractor-tg5qk"
Oct 31 17:27:07.287665 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:07.287637    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/61462533-417a-4fbc-bd85-c56a0112db31-crio-socket\") pod \"insights-runtime-extractor-tg5qk\" (UID: \"61462533-417a-4fbc-bd85-c56a0112db31\") " pod="openshift-insights/insights-runtime-extractor-tg5qk"
Oct 31 17:27:07.287665 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:07.287654    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/61462533-417a-4fbc-bd85-c56a0112db31-data-volume\") pod \"insights-runtime-extractor-tg5qk\" (UID: \"61462533-417a-4fbc-bd85-c56a0112db31\") " pod="openshift-insights/insights-runtime-extractor-tg5qk"
Oct 31 17:27:07.287765 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:07.287674    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-49ggd\" (UniqueName: \"kubernetes.io/projected/22096c98-1b93-4ea4-9a6f-6383df9eafe4-kube-api-access-49ggd\") pod \"ingress-canary-b5ln2\" (UID: \"22096c98-1b93-4ea4-9a6f-6383df9eafe4\") " pod="openshift-ingress-canary/ingress-canary-b5ln2"
Oct 31 17:27:07.287799 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:07.287762    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9ee82620-6497-48e8-bb2c-1bb77a93f7ef-metrics-tls\") pod \"dns-default-59fpc\" (UID: \"9ee82620-6497-48e8-bb2c-1bb77a93f7ef\") " pod="openshift-dns/dns-default-59fpc"
Oct 31 17:27:07.287799 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:07.287787    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/61462533-417a-4fbc-bd85-c56a0112db31-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tg5qk\" (UID: \"61462533-417a-4fbc-bd85-c56a0112db31\") " pod="openshift-insights/insights-runtime-extractor-tg5qk"
Oct 31 17:27:07.287870 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:07.287803    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p2gtl\" (UniqueName: \"kubernetes.io/projected/61462533-417a-4fbc-bd85-c56a0112db31-kube-api-access-p2gtl\") pod \"insights-runtime-extractor-tg5qk\" (UID: \"61462533-417a-4fbc-bd85-c56a0112db31\") " pod="openshift-insights/insights-runtime-extractor-tg5qk"
Oct 31 17:27:07.294137 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:07.294113    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Oct 31 17:27:07.294199 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:07.294186    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Oct 31 17:27:07.366884 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:07.366856    2578 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-5fffc97bdf-ncfqp"]
Oct 31 17:27:07.369654 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:07.369640    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-5fffc97bdf-ncfqp"
Oct 31 17:27:07.375142 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:07.375112    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pnjdm"
Oct 31 17:27:07.375212 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:07.375184    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tskzw"
Oct 31 17:27:07.375348 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:07.375314    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kmkdh"
Oct 31 17:27:07.386599 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:07.386482    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Oct 31 17:27:07.386697 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:07.386625    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Oct 31 17:27:07.386697 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:07.386628    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Oct 31 17:27:07.386697 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:07.386645    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Oct 31 17:27:07.386862 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:07.386750    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-hpj5l\""
Oct 31 17:27:07.387387 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:07.387370    2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-5fffc97bdf-ncfqp"]
Oct 31 17:27:07.388161 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:07.388136    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/61462533-417a-4fbc-bd85-c56a0112db31-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tg5qk\" (UID: \"61462533-417a-4fbc-bd85-c56a0112db31\") " pod="openshift-insights/insights-runtime-extractor-tg5qk"
Oct 31 17:27:07.388244 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:07.388178    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/61462533-417a-4fbc-bd85-c56a0112db31-crio-socket\") pod \"insights-runtime-extractor-tg5qk\" (UID: \"61462533-417a-4fbc-bd85-c56a0112db31\") " pod="openshift-insights/insights-runtime-extractor-tg5qk"
Oct 31 17:27:07.388244 ip-10-0-129-127 kubenswrapper[2578]: E1031 17:27:07.388205    2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Oct 31 17:27:07.388344 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:07.388206    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/61462533-417a-4fbc-bd85-c56a0112db31-data-volume\") pod \"insights-runtime-extractor-tg5qk\" (UID: \"61462533-417a-4fbc-bd85-c56a0112db31\") " pod="openshift-insights/insights-runtime-extractor-tg5qk"
Oct 31 17:27:07.388344 ip-10-0-129-127 kubenswrapper[2578]: E1031 17:27:07.388262    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/61462533-417a-4fbc-bd85-c56a0112db31-kube-rbac-proxy-cm podName:61462533-417a-4fbc-bd85-c56a0112db31 nodeName:}" failed. No retries permitted until 2026-10-31 17:27:07.888243949 +0000 UTC m=+30.053762337 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/61462533-417a-4fbc-bd85-c56a0112db31-kube-rbac-proxy-cm") pod "insights-runtime-extractor-tg5qk" (UID: "61462533-417a-4fbc-bd85-c56a0112db31") : configmap "kube-rbac-proxy" not found
Oct 31 17:27:07.388344 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:07.388290    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-49ggd\" (UniqueName: \"kubernetes.io/projected/22096c98-1b93-4ea4-9a6f-6383df9eafe4-kube-api-access-49ggd\") pod \"ingress-canary-b5ln2\" (UID: \"22096c98-1b93-4ea4-9a6f-6383df9eafe4\") " pod="openshift-ingress-canary/ingress-canary-b5ln2"
Oct 31 17:27:07.388344 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:07.388343    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9ee82620-6497-48e8-bb2c-1bb77a93f7ef-metrics-tls\") pod \"dns-default-59fpc\" (UID: \"9ee82620-6497-48e8-bb2c-1bb77a93f7ef\") " pod="openshift-dns/dns-default-59fpc"
Oct 31 17:27:07.388557 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:07.388379    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/61462533-417a-4fbc-bd85-c56a0112db31-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tg5qk\" (UID: \"61462533-417a-4fbc-bd85-c56a0112db31\") " pod="openshift-insights/insights-runtime-extractor-tg5qk"
Oct 31 17:27:07.388557 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:07.388401    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/61462533-417a-4fbc-bd85-c56a0112db31-crio-socket\") pod \"insights-runtime-extractor-tg5qk\" (UID: \"61462533-417a-4fbc-bd85-c56a0112db31\") " pod="openshift-insights/insights-runtime-extractor-tg5qk"
Oct 31 17:27:07.388557 ip-10-0-129-127 kubenswrapper[2578]: E1031 17:27:07.388486    2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 31 17:27:07.388557 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:07.388510    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/61462533-417a-4fbc-bd85-c56a0112db31-data-volume\") pod \"insights-runtime-extractor-tg5qk\" (UID: \"61462533-417a-4fbc-bd85-c56a0112db31\") " pod="openshift-insights/insights-runtime-extractor-tg5qk"
Oct 31 17:27:07.388557 ip-10-0-129-127 kubenswrapper[2578]: E1031 17:27:07.388531    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/61462533-417a-4fbc-bd85-c56a0112db31-insights-runtime-extractor-tls podName:61462533-417a-4fbc-bd85-c56a0112db31 nodeName:}" failed. No retries permitted until 2026-10-31 17:27:07.888515631 +0000 UTC m=+30.054033997 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/61462533-417a-4fbc-bd85-c56a0112db31-insights-runtime-extractor-tls") pod "insights-runtime-extractor-tg5qk" (UID: "61462533-417a-4fbc-bd85-c56a0112db31") : secret "insights-runtime-extractor-tls" not found
Oct 31 17:27:07.388557 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:07.388555    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p2gtl\" (UniqueName: \"kubernetes.io/projected/61462533-417a-4fbc-bd85-c56a0112db31-kube-api-access-p2gtl\") pod \"insights-runtime-extractor-tg5qk\" (UID: \"61462533-417a-4fbc-bd85-c56a0112db31\") " pod="openshift-insights/insights-runtime-extractor-tg5qk"
Oct 31 17:27:07.388823 ip-10-0-129-127 kubenswrapper[2578]: E1031 17:27:07.388564    2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 31 17:27:07.388823 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:07.388590    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z8zhc\" (UniqueName: \"kubernetes.io/projected/9ee82620-6497-48e8-bb2c-1bb77a93f7ef-kube-api-access-z8zhc\") pod \"dns-default-59fpc\" (UID: \"9ee82620-6497-48e8-bb2c-1bb77a93f7ef\") " pod="openshift-dns/dns-default-59fpc"
Oct 31 17:27:07.388823 ip-10-0-129-127 kubenswrapper[2578]: E1031 17:27:07.388615    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9ee82620-6497-48e8-bb2c-1bb77a93f7ef-metrics-tls podName:9ee82620-6497-48e8-bb2c-1bb77a93f7ef nodeName:}" failed. No retries permitted until 2026-10-31 17:27:07.88859603 +0000 UTC m=+30.054114409 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9ee82620-6497-48e8-bb2c-1bb77a93f7ef-metrics-tls") pod "dns-default-59fpc" (UID: "9ee82620-6497-48e8-bb2c-1bb77a93f7ef") : secret "dns-default-metrics-tls" not found
Oct 31 17:27:07.388823 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:07.388766    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9ee82620-6497-48e8-bb2c-1bb77a93f7ef-tmp-dir\") pod \"dns-default-59fpc\" (UID: \"9ee82620-6497-48e8-bb2c-1bb77a93f7ef\") " pod="openshift-dns/dns-default-59fpc"
Oct 31 17:27:07.388823 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:07.388810    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/9ee82620-6497-48e8-bb2c-1bb77a93f7ef-config-volume\") pod \"dns-default-59fpc\" (UID: \"9ee82620-6497-48e8-bb2c-1bb77a93f7ef\") " pod="openshift-dns/dns-default-59fpc"
Oct 31 17:27:07.389192 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:07.388839    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/22096c98-1b93-4ea4-9a6f-6383df9eafe4-cert\") pod \"ingress-canary-b5ln2\" (UID: \"22096c98-1b93-4ea4-9a6f-6383df9eafe4\") " pod="openshift-ingress-canary/ingress-canary-b5ln2"
Oct 31 17:27:07.389192 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:07.389081    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9ee82620-6497-48e8-bb2c-1bb77a93f7ef-tmp-dir\") pod \"dns-default-59fpc\" (UID: \"9ee82620-6497-48e8-bb2c-1bb77a93f7ef\") " pod="openshift-dns/dns-default-59fpc"
Oct 31 17:27:07.389530 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:07.389506    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/9ee82620-6497-48e8-bb2c-1bb77a93f7ef-config-volume\") pod \"dns-default-59fpc\" (UID: \"9ee82620-6497-48e8-bb2c-1bb77a93f7ef\") " pod="openshift-dns/dns-default-59fpc"
Oct 31 17:27:07.391165 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:07.391148    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Oct 31 17:27:07.391378 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:07.391220    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Oct 31 17:27:07.391513 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:07.391493    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Oct 31 17:27:07.391574 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:07.391516    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-nk5tf\""
Oct 31 17:27:07.391574 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:07.391500    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-z2hdt\""
Oct 31 17:27:07.391574 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:07.391538    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-tj66q\""
Oct 31 17:27:07.391665 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:07.391597    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Oct 31 17:27:07.393486 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:07.393468    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/22096c98-1b93-4ea4-9a6f-6383df9eafe4-cert\") pod \"ingress-canary-b5ln2\" (UID: \"22096c98-1b93-4ea4-9a6f-6383df9eafe4\") " pod="openshift-ingress-canary/ingress-canary-b5ln2"
Oct 31 17:27:07.409165 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:07.409144    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-49ggd\" (UniqueName: \"kubernetes.io/projected/22096c98-1b93-4ea4-9a6f-6383df9eafe4-kube-api-access-49ggd\") pod \"ingress-canary-b5ln2\" (UID: \"22096c98-1b93-4ea4-9a6f-6383df9eafe4\") " pod="openshift-ingress-canary/ingress-canary-b5ln2"
Oct 31 17:27:07.412507 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:07.412490    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z8zhc\" (UniqueName: \"kubernetes.io/projected/9ee82620-6497-48e8-bb2c-1bb77a93f7ef-kube-api-access-z8zhc\") pod \"dns-default-59fpc\" (UID: \"9ee82620-6497-48e8-bb2c-1bb77a93f7ef\") " pod="openshift-dns/dns-default-59fpc"
Oct 31 17:27:07.412801 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:07.412779    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p2gtl\" (UniqueName: \"kubernetes.io/projected/61462533-417a-4fbc-bd85-c56a0112db31-kube-api-access-p2gtl\") pod \"insights-runtime-extractor-tg5qk\" (UID: \"61462533-417a-4fbc-bd85-c56a0112db31\") " pod="openshift-insights/insights-runtime-extractor-tg5qk"
Oct 31 17:27:07.489571 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:07.489548    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5dkwn\" (UniqueName: \"kubernetes.io/projected/408bcc05-9690-4632-a479-de41718847ad-kube-api-access-5dkwn\") pod \"service-ca-5fffc97bdf-ncfqp\" (UID: \"408bcc05-9690-4632-a479-de41718847ad\") " pod="openshift-service-ca/service-ca-5fffc97bdf-ncfqp"
Oct 31 17:27:07.489676 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:07.489574    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/408bcc05-9690-4632-a479-de41718847ad-signing-key\") pod \"service-ca-5fffc97bdf-ncfqp\" (UID: \"408bcc05-9690-4632-a479-de41718847ad\") " pod="openshift-service-ca/service-ca-5fffc97bdf-ncfqp"
Oct 31 17:27:07.489732 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:07.489718    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/408bcc05-9690-4632-a479-de41718847ad-signing-cabundle\") pod \"service-ca-5fffc97bdf-ncfqp\" (UID: \"408bcc05-9690-4632-a479-de41718847ad\") " pod="openshift-service-ca/service-ca-5fffc97bdf-ncfqp"
Oct 31 17:27:07.509881 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:07.509835    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-b5ln2"
Oct 31 17:27:07.590185 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:07.590159    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/408bcc05-9690-4632-a479-de41718847ad-signing-cabundle\") pod \"service-ca-5fffc97bdf-ncfqp\" (UID: \"408bcc05-9690-4632-a479-de41718847ad\") " pod="openshift-service-ca/service-ca-5fffc97bdf-ncfqp"
Oct 31 17:27:07.590299 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:07.590200    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5dkwn\" (UniqueName: \"kubernetes.io/projected/408bcc05-9690-4632-a479-de41718847ad-kube-api-access-5dkwn\") pod \"service-ca-5fffc97bdf-ncfqp\" (UID: \"408bcc05-9690-4632-a479-de41718847ad\") " pod="openshift-service-ca/service-ca-5fffc97bdf-ncfqp"
Oct 31 17:27:07.590299 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:07.590219    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/408bcc05-9690-4632-a479-de41718847ad-signing-key\") pod \"service-ca-5fffc97bdf-ncfqp\" (UID: \"408bcc05-9690-4632-a479-de41718847ad\") " pod="openshift-service-ca/service-ca-5fffc97bdf-ncfqp"
Oct 31 17:27:07.590834 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:07.590806    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/408bcc05-9690-4632-a479-de41718847ad-signing-cabundle\") pod \"service-ca-5fffc97bdf-ncfqp\" (UID: \"408bcc05-9690-4632-a479-de41718847ad\") " pod="openshift-service-ca/service-ca-5fffc97bdf-ncfqp"
Oct 31 17:27:07.592590 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:07.592571    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/408bcc05-9690-4632-a479-de41718847ad-signing-key\") pod \"service-ca-5fffc97bdf-ncfqp\" (UID: \"408bcc05-9690-4632-a479-de41718847ad\") " pod="openshift-service-ca/service-ca-5fffc97bdf-ncfqp"
Oct 31 17:27:07.610656 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:07.610631    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5dkwn\" (UniqueName: \"kubernetes.io/projected/408bcc05-9690-4632-a479-de41718847ad-kube-api-access-5dkwn\") pod \"service-ca-5fffc97bdf-ncfqp\" (UID: \"408bcc05-9690-4632-a479-de41718847ad\") " pod="openshift-service-ca/service-ca-5fffc97bdf-ncfqp"
Oct 31 17:27:07.680645 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:07.680599    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-5fffc97bdf-ncfqp"
Oct 31 17:27:07.813057 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:07.813034    2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-b5ln2"]
Oct 31 17:27:07.815227 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:27:07.815205    2578 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod22096c98_1b93_4ea4_9a6f_6383df9eafe4.slice/crio-fcf1add41537fb1ea326adfb301b383a19868c74aedb40d86cf92ad0e07ae83c WatchSource:0}: Error finding container fcf1add41537fb1ea326adfb301b383a19868c74aedb40d86cf92ad0e07ae83c: Status 404 returned error can't find the container with id fcf1add41537fb1ea326adfb301b383a19868c74aedb40d86cf92ad0e07ae83c
Oct 31 17:27:07.859255 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:07.859213    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:27:07.892202 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:07.892177    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/61462533-417a-4fbc-bd85-c56a0112db31-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tg5qk\" (UID: \"61462533-417a-4fbc-bd85-c56a0112db31\") " pod="openshift-insights/insights-runtime-extractor-tg5qk"
Oct 31 17:27:07.892279 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:07.892231    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9ee82620-6497-48e8-bb2c-1bb77a93f7ef-metrics-tls\") pod \"dns-default-59fpc\" (UID: \"9ee82620-6497-48e8-bb2c-1bb77a93f7ef\") " pod="openshift-dns/dns-default-59fpc"
Oct 31 17:27:07.892318 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:07.892279    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/61462533-417a-4fbc-bd85-c56a0112db31-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tg5qk\" (UID: \"61462533-417a-4fbc-bd85-c56a0112db31\") " pod="openshift-insights/insights-runtime-extractor-tg5qk"
Oct 31 17:27:07.892318 ip-10-0-129-127 kubenswrapper[2578]: E1031 17:27:07.892300    2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Oct 31 17:27:07.892377 ip-10-0-129-127 kubenswrapper[2578]: E1031 17:27:07.892322    2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 31 17:27:07.892377 ip-10-0-129-127 kubenswrapper[2578]: E1031 17:27:07.892361    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/61462533-417a-4fbc-bd85-c56a0112db31-kube-rbac-proxy-cm podName:61462533-417a-4fbc-bd85-c56a0112db31 nodeName:}" failed. No retries permitted until 2026-10-31 17:27:08.892342402 +0000 UTC m=+31.057860783 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/61462533-417a-4fbc-bd85-c56a0112db31-kube-rbac-proxy-cm") pod "insights-runtime-extractor-tg5qk" (UID: "61462533-417a-4fbc-bd85-c56a0112db31") : configmap "kube-rbac-proxy" not found
Oct 31 17:27:07.892377 ip-10-0-129-127 kubenswrapper[2578]: E1031 17:27:07.892376    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9ee82620-6497-48e8-bb2c-1bb77a93f7ef-metrics-tls podName:9ee82620-6497-48e8-bb2c-1bb77a93f7ef nodeName:}" failed. No retries permitted until 2026-10-31 17:27:08.892369705 +0000 UTC m=+31.057888069 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9ee82620-6497-48e8-bb2c-1bb77a93f7ef-metrics-tls") pod "dns-default-59fpc" (UID: "9ee82620-6497-48e8-bb2c-1bb77a93f7ef") : secret "dns-default-metrics-tls" not found
Oct 31 17:27:07.892492 ip-10-0-129-127 kubenswrapper[2578]: E1031 17:27:07.892379    2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 31 17:27:07.892492 ip-10-0-129-127 kubenswrapper[2578]: E1031 17:27:07.892416    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/61462533-417a-4fbc-bd85-c56a0112db31-insights-runtime-extractor-tls podName:61462533-417a-4fbc-bd85-c56a0112db31 nodeName:}" failed. No retries permitted until 2026-10-31 17:27:08.89240453 +0000 UTC m=+31.057922909 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/61462533-417a-4fbc-bd85-c56a0112db31-insights-runtime-extractor-tls") pod "insights-runtime-extractor-tg5qk" (UID: "61462533-417a-4fbc-bd85-c56a0112db31") : secret "insights-runtime-extractor-tls" not found
Oct 31 17:27:07.974235 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:07.974219    2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-5fffc97bdf-ncfqp"]
Oct 31 17:27:07.976174 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:27:07.976153    2578 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod408bcc05_9690_4632_a479_de41718847ad.slice/crio-4afec5f5a20882c39cdbd49f1982ca090f6b8dfa8e67c1fe384d378475e71d92 WatchSource:0}: Error finding container 4afec5f5a20882c39cdbd49f1982ca090f6b8dfa8e67c1fe384d378475e71d92: Status 404 returned error can't find the container with id 4afec5f5a20882c39cdbd49f1982ca090f6b8dfa8e67c1fe384d378475e71d92
Oct 31 17:27:07.979364 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:07.979339    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:27:08.164706 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:08.164631    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:27:08.747167 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:08.747121    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-b5ln2" event={"ID":"22096c98-1b93-4ea4-9a6f-6383df9eafe4","Type":"ContainerStarted","Data":"fcf1add41537fb1ea326adfb301b383a19868c74aedb40d86cf92ad0e07ae83c"}
Oct 31 17:27:08.748428 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:08.748399    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-5fffc97bdf-ncfqp" event={"ID":"408bcc05-9690-4632-a479-de41718847ad","Type":"ContainerStarted","Data":"4afec5f5a20882c39cdbd49f1982ca090f6b8dfa8e67c1fe384d378475e71d92"}
Oct 31 17:27:08.900325 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:08.900287    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/61462533-417a-4fbc-bd85-c56a0112db31-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tg5qk\" (UID: \"61462533-417a-4fbc-bd85-c56a0112db31\") " pod="openshift-insights/insights-runtime-extractor-tg5qk"
Oct 31 17:27:08.900934 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:08.900362    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9ee82620-6497-48e8-bb2c-1bb77a93f7ef-metrics-tls\") pod \"dns-default-59fpc\" (UID: \"9ee82620-6497-48e8-bb2c-1bb77a93f7ef\") " pod="openshift-dns/dns-default-59fpc"
Oct 31 17:27:08.900934 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:08.900405    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/61462533-417a-4fbc-bd85-c56a0112db31-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tg5qk\" (UID: \"61462533-417a-4fbc-bd85-c56a0112db31\") " pod="openshift-insights/insights-runtime-extractor-tg5qk"
Oct 31 17:27:08.900934 ip-10-0-129-127 kubenswrapper[2578]: E1031 17:27:08.900409    2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Oct 31 17:27:08.900934 ip-10-0-129-127 kubenswrapper[2578]: E1031 17:27:08.900499    2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 31 17:27:08.900934 ip-10-0-129-127 kubenswrapper[2578]: E1031 17:27:08.900521    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/61462533-417a-4fbc-bd85-c56a0112db31-kube-rbac-proxy-cm podName:61462533-417a-4fbc-bd85-c56a0112db31 nodeName:}" failed. No retries permitted until 2026-10-31 17:27:10.900503522 +0000 UTC m=+33.066021886 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/61462533-417a-4fbc-bd85-c56a0112db31-kube-rbac-proxy-cm") pod "insights-runtime-extractor-tg5qk" (UID: "61462533-417a-4fbc-bd85-c56a0112db31") : configmap "kube-rbac-proxy" not found
Oct 31 17:27:08.900934 ip-10-0-129-127 kubenswrapper[2578]: E1031 17:27:08.900538    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9ee82620-6497-48e8-bb2c-1bb77a93f7ef-metrics-tls podName:9ee82620-6497-48e8-bb2c-1bb77a93f7ef nodeName:}" failed. No retries permitted until 2026-10-31 17:27:10.900526774 +0000 UTC m=+33.066045152 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9ee82620-6497-48e8-bb2c-1bb77a93f7ef-metrics-tls") pod "dns-default-59fpc" (UID: "9ee82620-6497-48e8-bb2c-1bb77a93f7ef") : secret "dns-default-metrics-tls" not found
Oct 31 17:27:08.900934 ip-10-0-129-127 kubenswrapper[2578]: E1031 17:27:08.900540    2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 31 17:27:08.900934 ip-10-0-129-127 kubenswrapper[2578]: E1031 17:27:08.900625    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/61462533-417a-4fbc-bd85-c56a0112db31-insights-runtime-extractor-tls podName:61462533-417a-4fbc-bd85-c56a0112db31 nodeName:}" failed. No retries permitted until 2026-10-31 17:27:10.900605502 +0000 UTC m=+33.066123867 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/61462533-417a-4fbc-bd85-c56a0112db31-insights-runtime-extractor-tls") pod "insights-runtime-extractor-tg5qk" (UID: "61462533-417a-4fbc-bd85-c56a0112db31") : secret "insights-runtime-extractor-tls" not found
Oct 31 17:27:10.917792 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:10.917770    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9ee82620-6497-48e8-bb2c-1bb77a93f7ef-metrics-tls\") pod \"dns-default-59fpc\" (UID: \"9ee82620-6497-48e8-bb2c-1bb77a93f7ef\") " pod="openshift-dns/dns-default-59fpc"
Oct 31 17:27:10.918209 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:10.917822    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/61462533-417a-4fbc-bd85-c56a0112db31-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tg5qk\" (UID: \"61462533-417a-4fbc-bd85-c56a0112db31\") " pod="openshift-insights/insights-runtime-extractor-tg5qk"
Oct 31 17:27:10.918209 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:10.917887    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/61462533-417a-4fbc-bd85-c56a0112db31-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tg5qk\" (UID: \"61462533-417a-4fbc-bd85-c56a0112db31\") " pod="openshift-insights/insights-runtime-extractor-tg5qk"
Oct 31 17:27:10.918209 ip-10-0-129-127 kubenswrapper[2578]: E1031 17:27:10.917923    2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 31 17:27:10.918209 ip-10-0-129-127 kubenswrapper[2578]: E1031 17:27:10.917963    2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 31 17:27:10.918209 ip-10-0-129-127 kubenswrapper[2578]: E1031 17:27:10.917979    2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Oct 31 17:27:10.918209 ip-10-0-129-127 kubenswrapper[2578]: E1031 17:27:10.917984    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9ee82620-6497-48e8-bb2c-1bb77a93f7ef-metrics-tls podName:9ee82620-6497-48e8-bb2c-1bb77a93f7ef nodeName:}" failed. No retries permitted until 2026-10-31 17:27:14.917965818 +0000 UTC m=+37.083484201 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9ee82620-6497-48e8-bb2c-1bb77a93f7ef-metrics-tls") pod "dns-default-59fpc" (UID: "9ee82620-6497-48e8-bb2c-1bb77a93f7ef") : secret "dns-default-metrics-tls" not found
Oct 31 17:27:10.918209 ip-10-0-129-127 kubenswrapper[2578]: E1031 17:27:10.918041    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/61462533-417a-4fbc-bd85-c56a0112db31-insights-runtime-extractor-tls podName:61462533-417a-4fbc-bd85-c56a0112db31 nodeName:}" failed. No retries permitted until 2026-10-31 17:27:14.918027244 +0000 UTC m=+37.083545611 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/61462533-417a-4fbc-bd85-c56a0112db31-insights-runtime-extractor-tls") pod "insights-runtime-extractor-tg5qk" (UID: "61462533-417a-4fbc-bd85-c56a0112db31") : secret "insights-runtime-extractor-tls" not found
Oct 31 17:27:10.918209 ip-10-0-129-127 kubenswrapper[2578]: E1031 17:27:10.918054    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/61462533-417a-4fbc-bd85-c56a0112db31-kube-rbac-proxy-cm podName:61462533-417a-4fbc-bd85-c56a0112db31 nodeName:}" failed. No retries permitted until 2026-10-31 17:27:14.918046468 +0000 UTC m=+37.083564834 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/61462533-417a-4fbc-bd85-c56a0112db31-kube-rbac-proxy-cm") pod "insights-runtime-extractor-tg5qk" (UID: "61462533-417a-4fbc-bd85-c56a0112db31") : configmap "kube-rbac-proxy" not found
Oct 31 17:27:10.919977 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:10.919952    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:27:11.034931 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:11.034728    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:27:11.118999 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:11.118965    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/138bfcc8-f199-4ffd-b5d2-8d83ade6199a-metrics-certs\") pod \"network-metrics-daemon-pnjdm\" (UID: \"138bfcc8-f199-4ffd-b5d2-8d83ade6199a\") " pod="openshift-multus/network-metrics-daemon-pnjdm"
Oct 31 17:27:11.119193 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:11.119009    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6d7781d6-8f73-4f1d-bb9a-0212e2eb6e4b-original-pull-secret\") pod \"global-pull-secret-syncer-kmkdh\" (UID: \"6d7781d6-8f73-4f1d-bb9a-0212e2eb6e4b\") " pod="kube-system/global-pull-secret-syncer-kmkdh"
Oct 31 17:27:11.119193 ip-10-0-129-127 kubenswrapper[2578]: E1031 17:27:11.119142    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Oct 31 17:27:11.119303 ip-10-0-129-127 kubenswrapper[2578]: E1031 17:27:11.119215    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/138bfcc8-f199-4ffd-b5d2-8d83ade6199a-metrics-certs podName:138bfcc8-f199-4ffd-b5d2-8d83ade6199a nodeName:}" failed. No retries permitted until 2026-10-31 17:27:43.119196027 +0000 UTC m=+65.284714395 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/138bfcc8-f199-4ffd-b5d2-8d83ade6199a-metrics-certs") pod "network-metrics-daemon-pnjdm" (UID: "138bfcc8-f199-4ffd-b5d2-8d83ade6199a") : secret "metrics-daemon-secret" not found
Oct 31 17:27:11.122136 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:11.122113    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6d7781d6-8f73-4f1d-bb9a-0212e2eb6e4b-original-pull-secret\") pod \"global-pull-secret-syncer-kmkdh\" (UID: \"6d7781d6-8f73-4f1d-bb9a-0212e2eb6e4b\") " pod="kube-system/global-pull-secret-syncer-kmkdh"
Oct 31 17:27:11.219668 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:11.219572    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c6k68\" (UniqueName: \"kubernetes.io/projected/fb9c203a-af67-4df4-88dc-74cf9eab3623-kube-api-access-c6k68\") pod \"network-check-target-tskzw\" (UID: \"fb9c203a-af67-4df4-88dc-74cf9eab3623\") " pod="openshift-network-diagnostics/network-check-target-tskzw"
Oct 31 17:27:11.222048 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:11.222023    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c6k68\" (UniqueName: \"kubernetes.io/projected/fb9c203a-af67-4df4-88dc-74cf9eab3623-kube-api-access-c6k68\") pod \"network-check-target-tskzw\" (UID: \"fb9c203a-af67-4df4-88dc-74cf9eab3623\") " pod="openshift-network-diagnostics/network-check-target-tskzw"
Oct 31 17:27:11.295087 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:11.295064    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tskzw"
Oct 31 17:27:11.303215 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:11.303196    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kmkdh"
Oct 31 17:27:11.616613 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:11.616586    2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-tskzw"]
Oct 31 17:27:11.624373 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:27:11.624341    2578 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb9c203a_af67_4df4_88dc_74cf9eab3623.slice/crio-05fd6e0cad7c3c567ee186139aeeee3ca4d8c9d1ff9304f905346df0c19494ba WatchSource:0}: Error finding container 05fd6e0cad7c3c567ee186139aeeee3ca4d8c9d1ff9304f905346df0c19494ba: Status 404 returned error can't find the container with id 05fd6e0cad7c3c567ee186139aeeee3ca4d8c9d1ff9304f905346df0c19494ba
Oct 31 17:27:11.628952 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:11.628932    2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-kmkdh"]
Oct 31 17:27:11.631602 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:27:11.631579    2578 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d7781d6_8f73_4f1d_bb9a_0212e2eb6e4b.slice/crio-218b9721df0830d0c7cab0413f740b319f6f6cbd1a9564068ed16caab7b39ef8 WatchSource:0}: Error finding container 218b9721df0830d0c7cab0413f740b319f6f6cbd1a9564068ed16caab7b39ef8: Status 404 returned error can't find the container with id 218b9721df0830d0c7cab0413f740b319f6f6cbd1a9564068ed16caab7b39ef8
Oct 31 17:27:11.757647 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:11.757611    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-tskzw" event={"ID":"fb9c203a-af67-4df4-88dc-74cf9eab3623","Type":"ContainerStarted","Data":"05fd6e0cad7c3c567ee186139aeeee3ca4d8c9d1ff9304f905346df0c19494ba"}
Oct 31 17:27:11.759510 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:11.759278    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-5fffc97bdf-ncfqp" event={"ID":"408bcc05-9690-4632-a479-de41718847ad","Type":"ContainerStarted","Data":"7bdac0481fa82cf589de80789b5fcef8fcf7b610273e072a2145ac0c3a36d8bf"}
Oct 31 17:27:11.761068 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:11.760998    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-b5ln2" event={"ID":"22096c98-1b93-4ea4-9a6f-6383df9eafe4","Type":"ContainerStarted","Data":"17742376acc0f043b0625134fdc9b72046bc0f173c570f03f218bb7ae9b1c6a6"}
Oct 31 17:27:11.762429 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:11.762408    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-kmkdh" event={"ID":"6d7781d6-8f73-4f1d-bb9a-0212e2eb6e4b","Type":"ContainerStarted","Data":"218b9721df0830d0c7cab0413f740b319f6f6cbd1a9564068ed16caab7b39ef8"}
Oct 31 17:27:11.823257 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:11.823110    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:27:11.842249 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:11.841237    2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-b5ln2" podStartSLOduration=1.754116173 podStartE2EDuration="4.841221359s" totalImagesPullingTime="3.087105186s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 17:27:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 17:27:11.840841937 +0000 UTC m=+34.006360324" watchObservedRunningTime="2026-10-31 17:27:11.841221359 +0000 UTC m=+34.006739748"
Oct 31 17:27:11.842249 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:11.841765    2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-service-ca/service-ca-5fffc97bdf-ncfqp" podStartSLOduration=1.9278438119999999 podStartE2EDuration="4.841757476s" totalImagesPullingTime="2.913913664s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 17:27:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 17:27:11.800110196 +0000 UTC m=+33.965628583" watchObservedRunningTime="2026-10-31 17:27:11.841757476 +0000 UTC m=+34.007275863"
Oct 31 17:27:12.031014 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:12.030937    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:27:12.156521 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:12.156483    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:27:12.354638 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:12.354600    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:27:12.517565 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:12.517528    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:27:13.544718 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:13.544680    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:27:13.726955 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:13.726925    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-7pfr4_b8bbaf33-7cd9-4a30-86f7-6e20ac45b0a0/dns-node-resolver/0.log"
Oct 31 17:27:13.742218 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:13.742182    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:27:13.951095 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:13.951004    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:27:14.147536 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:14.147499    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:27:14.253751 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:14.253668    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:27:14.364731 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:14.364697    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:27:14.927012 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:14.926987    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-qxnvq_87e61dcd-75f6-4d3f-8ed9-7d35b06a9bb3/node-ca/0.log"
Oct 31 17:27:14.954293 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:14.954261    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/61462533-417a-4fbc-bd85-c56a0112db31-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tg5qk\" (UID: \"61462533-417a-4fbc-bd85-c56a0112db31\") " pod="openshift-insights/insights-runtime-extractor-tg5qk"
Oct 31 17:27:14.954473 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:14.954313    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9ee82620-6497-48e8-bb2c-1bb77a93f7ef-metrics-tls\") pod \"dns-default-59fpc\" (UID: \"9ee82620-6497-48e8-bb2c-1bb77a93f7ef\") " pod="openshift-dns/dns-default-59fpc"
Oct 31 17:27:14.954473 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:14.954335    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/61462533-417a-4fbc-bd85-c56a0112db31-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tg5qk\" (UID: \"61462533-417a-4fbc-bd85-c56a0112db31\") " pod="openshift-insights/insights-runtime-extractor-tg5qk"
Oct 31 17:27:14.954473 ip-10-0-129-127 kubenswrapper[2578]: E1031 17:27:14.954457    2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 31 17:27:14.954628 ip-10-0-129-127 kubenswrapper[2578]: E1031 17:27:14.954518    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/61462533-417a-4fbc-bd85-c56a0112db31-insights-runtime-extractor-tls podName:61462533-417a-4fbc-bd85-c56a0112db31 nodeName:}" failed. No retries permitted until 2026-10-31 17:27:22.954500303 +0000 UTC m=+45.120018692 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/61462533-417a-4fbc-bd85-c56a0112db31-insights-runtime-extractor-tls") pod "insights-runtime-extractor-tg5qk" (UID: "61462533-417a-4fbc-bd85-c56a0112db31") : secret "insights-runtime-extractor-tls" not found
Oct 31 17:27:14.954902 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:14.954881    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/61462533-417a-4fbc-bd85-c56a0112db31-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tg5qk\" (UID: \"61462533-417a-4fbc-bd85-c56a0112db31\") " pod="openshift-insights/insights-runtime-extractor-tg5qk"
Oct 31 17:27:14.957620 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:14.957593    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9ee82620-6497-48e8-bb2c-1bb77a93f7ef-metrics-tls\") pod \"dns-default-59fpc\" (UID: \"9ee82620-6497-48e8-bb2c-1bb77a93f7ef\") " pod="openshift-dns/dns-default-59fpc"
Oct 31 17:27:15.001062 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:15.001030    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-59fpc"
Oct 31 17:27:15.326774 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:15.326610    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-b5ln2_22096c98-1b93-4ea4-9a6f-6383df9eafe4/serve-healthcheck-canary/0.log"
Oct 31 17:27:15.494869 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:15.494798    2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-59fpc"]
Oct 31 17:27:15.499396 ip-10-0-129-127 kubenswrapper[2578]: W1031 17:27:15.499365    2578 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ee82620_6497_48e8_bb2c_1bb77a93f7ef.slice/crio-46ce576c40eca23ecc0802b8b8286680492c154087372d52ce8d348fd0b158d6 WatchSource:0}: Error finding container 46ce576c40eca23ecc0802b8b8286680492c154087372d52ce8d348fd0b158d6: Status 404 returned error can't find the container with id 46ce576c40eca23ecc0802b8b8286680492c154087372d52ce8d348fd0b158d6
Oct 31 17:27:15.543637 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:15.543600    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:27:15.767386 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:15.767307    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:27:15.775321 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:15.775288    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-59fpc" event={"ID":"9ee82620-6497-48e8-bb2c-1bb77a93f7ef","Type":"ContainerStarted","Data":"46ce576c40eca23ecc0802b8b8286680492c154087372d52ce8d348fd0b158d6"}
Oct 31 17:27:15.777027 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:15.776995    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-tskzw" event={"ID":"fb9c203a-af67-4df4-88dc-74cf9eab3623","Type":"ContainerStarted","Data":"ec8ac87a4ab623884a952ff98caa64e0b1b5b52f49b31b3f805cfc215048fe84"}
Oct 31 17:27:15.777145 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:15.777131    2578 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-tskzw"
Oct 31 17:27:15.819263 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:15.819203    2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-tskzw" podStartSLOduration=33.238292244 podStartE2EDuration="36.819183665s" totalImagesPullingTime="3.580891421s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 17:26:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 17:27:15.818138889 +0000 UTC m=+37.983657277" watchObservedRunningTime="2026-10-31 17:27:15.819183665 +0000 UTC m=+37.984702058"
Oct 31 17:27:15.895139 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:15.895062    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:27:16.018064 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:16.017988    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:27:16.142700 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:16.142611    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:27:16.302927 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:16.302895    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:27:16.403612 ip-10-0-129-127 kubenswrapper[2578]: I1031 17:27:16.403581    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:27:17.112066 ip-10-0-129-127 systemd[1]: Stopping Kubernetes Kubelet...
Oct 31 17:27:17.122101 ip-10-0-129-127 systemd[1]: kubelet.service: Deactivated successfully.
Oct 31 17:27:17.122300 ip-10-0-129-127 systemd[1]: Stopped Kubernetes Kubelet.
Oct 31 17:27:17.122492 ip-10-0-129-127 systemd[1]: kubelet.service: Consumed 2.222s CPU time.
Oct 31 17:27:17.132098 ip-10-0-129-127 systemd[1]: Starting Kubernetes Kubelet...
Oct 31 17:27:17.268394 ip-10-0-129-127 kubenswrapper[4276]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 31 17:27:17.268394 ip-10-0-129-127 kubenswrapper[4276]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Oct 31 17:27:17.268394 ip-10-0-129-127 kubenswrapper[4276]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 31 17:27:17.268394 ip-10-0-129-127 kubenswrapper[4276]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 31 17:27:17.270557 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.270422    4276 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 31 17:27:17.270557 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.270556    4276 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 31 17:27:17.270557 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.270560    4276 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 31 17:27:17.270653 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.270564    4276 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 31 17:27:17.270653 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.270567    4276 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 31 17:27:17.270653 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.270570    4276 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 31 17:27:17.270653 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.270573    4276 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 31 17:27:17.270653 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.270575    4276 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 31 17:27:17.270653 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.270578    4276 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 31 17:27:17.270653 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.270581    4276 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 31 17:27:17.270653 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.270584    4276 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 31 17:27:17.270653 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.270586    4276 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 31 17:27:17.270653 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.270589    4276 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 31 17:27:17.270653 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.270592    4276 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 31 17:27:17.270653 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.270594    4276 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 31 17:27:17.270653 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.270598    4276 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 31 17:27:17.270653 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.270600    4276 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 31 17:27:17.270653 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.270603    4276 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 31 17:27:17.270653 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.270606    4276 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 31 17:27:17.270653 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.270608    4276 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 31 17:27:17.270653 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.270611    4276 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 31 17:27:17.270653 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.270613    4276 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 31 17:27:17.271149 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.270616    4276 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 31 17:27:17.271149 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.270618    4276 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 31 17:27:17.271149 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.270622    4276 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 31 17:27:17.271149 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.270624    4276 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 31 17:27:17.271149 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.270627    4276 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 31 17:27:17.271149 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.270630    4276 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 31 17:27:17.271149 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.270632    4276 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 31 17:27:17.271149 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.270643    4276 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 31 17:27:17.271149 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.270646    4276 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 31 17:27:17.271149 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.270651    4276 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 31 17:27:17.271149 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.270656    4276 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 31 17:27:17.271149 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.270660    4276 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 31 17:27:17.271149 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.270664    4276 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 31 17:27:17.271149 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.270667    4276 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 31 17:27:17.271149 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.270669    4276 feature_gate.go:386] unrecognized feature gate: Example
Oct 31 17:27:17.271149 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.270672    4276 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 31 17:27:17.271149 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.270675    4276 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 31 17:27:17.271149 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.270678    4276 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 31 17:27:17.271149 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.270681    4276 feature_gate.go:386] unrecognized feature gate: Example2
Oct 31 17:27:17.271149 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.270683    4276 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 31 17:27:17.271676 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.270686    4276 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 31 17:27:17.271676 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.270688    4276 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 31 17:27:17.271676 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.270691    4276 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 31 17:27:17.271676 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.270693    4276 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 31 17:27:17.271676 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.270696    4276 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 31 17:27:17.271676 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.270699    4276 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 31 17:27:17.271676 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.270702    4276 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 31 17:27:17.271676 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.270704    4276 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 31 17:27:17.271676 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.270707    4276 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 31 17:27:17.271676 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.270710    4276 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 31 17:27:17.271676 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.270712    4276 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 31 17:27:17.271676 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.270715    4276 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 31 17:27:17.271676 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.270718    4276 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 31 17:27:17.271676 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.270722    4276 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 31 17:27:17.271676 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.270726    4276 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 31 17:27:17.271676 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.270730    4276 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 31 17:27:17.271676 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.270732    4276 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 31 17:27:17.271676 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.270736    4276 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 31 17:27:17.271676 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.270739    4276 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 31 17:27:17.272158 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.270742    4276 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 31 17:27:17.272158 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.270744    4276 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 31 17:27:17.272158 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.270747    4276 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 31 17:27:17.272158 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.270749    4276 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 31 17:27:17.272158 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.270752    4276 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 31 17:27:17.272158 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.270755    4276 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 31 17:27:17.272158 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.270757    4276 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 31 17:27:17.272158 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.270760    4276 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 31 17:27:17.272158 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.270763    4276 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 31 17:27:17.272158 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.270766    4276 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 31 17:27:17.272158 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.270769    4276 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 31 17:27:17.272158 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.270771    4276 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 31 17:27:17.272158 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.270774    4276 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 31 17:27:17.272158 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.270776    4276 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 31 17:27:17.272158 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.270779    4276 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 31 17:27:17.272158 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.270781    4276 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 31 17:27:17.272158 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.270785    4276 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 31 17:27:17.272158 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.270789    4276 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 31 17:27:17.272158 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.270791    4276 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 31 17:27:17.272158 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.270794    4276 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 31 17:27:17.272679 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.270797    4276 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 31 17:27:17.272679 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.270799    4276 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 31 17:27:17.272679 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.270802    4276 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 31 17:27:17.272679 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.270804    4276 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 31 17:27:17.272679 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.270806    4276 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 31 17:27:17.272679 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.270809    4276 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 31 17:27:17.272679 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.270812    4276 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 31 17:27:17.272679 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.270814    4276 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 31 17:27:17.272679 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.270817    4276 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 31 17:27:17.272679 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.270819    4276 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 31 17:27:17.272679 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.270822    4276 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 31 17:27:17.272679 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.270824    4276 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 31 17:27:17.272679 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.270827    4276 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 31 17:27:17.272679 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.270829    4276 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 31 17:27:17.272679 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.270832    4276 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 31 17:27:17.272679 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.270835    4276 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 31 17:27:17.272679 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.270838    4276 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 31 17:27:17.272679 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.270840    4276 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 31 17:27:17.272679 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.270842    4276 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 31 17:27:17.272679 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.270845    4276 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 31 17:27:17.273270 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.270848    4276 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 31 17:27:17.273270 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.270851    4276 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 31 17:27:17.273270 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.270854    4276 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 31 17:27:17.273270 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.270857    4276 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 31 17:27:17.273270 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.270860    4276 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 31 17:27:17.273270 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.271628    4276 feature_gate.go:386] unrecognized feature gate: Example2
Oct 31 17:27:17.273270 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.271637    4276 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 31 17:27:17.273270 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.271640    4276 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 31 17:27:17.273270 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.271643    4276 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 31 17:27:17.273270 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.271646    4276 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 31 17:27:17.273270 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.271649    4276 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 31 17:27:17.273270 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.271653    4276 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 31 17:27:17.273270 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.271656    4276 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 31 17:27:17.273270 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.271659    4276 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 31 17:27:17.273270 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.271661    4276 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 31 17:27:17.273270 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.271664    4276 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 31 17:27:17.273270 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.271666    4276 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 31 17:27:17.273270 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.271669    4276 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 31 17:27:17.273270 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.271672    4276 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 31 17:27:17.273774 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.271674    4276 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 31 17:27:17.273774 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.271677    4276 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 31 17:27:17.273774 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.271680    4276 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 31 17:27:17.273774 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.271682    4276 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 31 17:27:17.273774 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.271685    4276 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 31 17:27:17.273774 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.271688    4276 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 31 17:27:17.273774 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.271691    4276 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 31 17:27:17.273774 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.271694    4276 feature_gate.go:386] unrecognized feature gate: Example
Oct 31 17:27:17.273774 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.271697    4276 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 31 17:27:17.273774 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.271700    4276 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 31 17:27:17.273774 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.271703    4276 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 31 17:27:17.273774 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.271706    4276 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 31 17:27:17.273774 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.271709    4276 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 31 17:27:17.273774 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.271711    4276 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 31 17:27:17.273774 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.271714    4276 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 31 17:27:17.273774 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.271716    4276 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 31 17:27:17.273774 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.271719    4276 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 31 17:27:17.273774 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.271722    4276 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 31 17:27:17.273774 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.271724    4276 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 31 17:27:17.273774 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.271727    4276 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 31 17:27:17.274268 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.271730    4276 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 31 17:27:17.274268 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.271732    4276 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 31 17:27:17.274268 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.271734    4276 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 31 17:27:17.274268 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.271737    4276 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 31 17:27:17.274268 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.271740    4276 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 31 17:27:17.274268 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.271743    4276 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 31 17:27:17.274268 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.271745    4276 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 31 17:27:17.274268 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.271748    4276 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 31 17:27:17.274268 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.271750    4276 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 31 17:27:17.274268 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.271753    4276 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 31 17:27:17.274268 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.271756    4276 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 31 17:27:17.274268 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.271758    4276 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 31 17:27:17.274268 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.271761    4276 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 31 17:27:17.274268 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.271763    4276 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 31 17:27:17.274268 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.271766    4276 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 31 17:27:17.274268 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.271768    4276 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 31 17:27:17.274268 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.271771    4276 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 31 17:27:17.274268 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.271773    4276 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 31 17:27:17.274268 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.271776    4276 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 31 17:27:17.274761 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.271779    4276 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 31 17:27:17.274761 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.271782    4276 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 31 17:27:17.274761 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.271785    4276 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 31 17:27:17.274761 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.271787    4276 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 31 17:27:17.274761 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.271790    4276 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 31 17:27:17.274761 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.271793    4276 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 31 17:27:17.274761 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.271795    4276 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 31 17:27:17.274761 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.271798    4276 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 31 17:27:17.274761 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.271800    4276 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 31 17:27:17.274761 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.271804    4276 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 31 17:27:17.274761 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.271809    4276 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 31 17:27:17.274761 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.271812    4276 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 31 17:27:17.274761 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.271814    4276 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 31 17:27:17.274761 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.271817    4276 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 31 17:27:17.274761 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.271819    4276 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 31 17:27:17.274761 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.271822    4276 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 31 17:27:17.274761 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.271824    4276 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 31 17:27:17.274761 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.271827    4276 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 31 17:27:17.274761 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.271830    4276 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 31 17:27:17.275245 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.271832    4276 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 31 17:27:17.275245 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.271835    4276 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 31 17:27:17.275245 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.271837    4276 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 31 17:27:17.275245 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.271839    4276 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 31 17:27:17.275245 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.271842    4276 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 31 17:27:17.275245 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.271844    4276 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 31 17:27:17.275245 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.271847    4276 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 31 17:27:17.275245 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.271849    4276 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 31 17:27:17.275245 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.271855    4276 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 31 17:27:17.275245 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.271857    4276 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 31 17:27:17.275245 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.271860    4276 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 31 17:27:17.275245 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.271862    4276 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 31 17:27:17.275245 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.271865    4276 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 31 17:27:17.275245 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.271868    4276 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 31 17:27:17.275245 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.271870    4276 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 31 17:27:17.275245 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.271873    4276 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 31 17:27:17.275245 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.271875    4276 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 31 17:27:17.275245 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.271878    4276 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 31 17:27:17.275245 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.271880    4276 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 31 17:27:17.275776 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.271884    4276 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 31 17:27:17.275776 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.271888    4276 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 31 17:27:17.275776 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.271890    4276 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 31 17:27:17.275776 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.271893    4276 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 31 17:27:17.275776 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.271896    4276 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 31 17:27:17.275776 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.271899    4276 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 31 17:27:17.275776 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.271901    4276 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 31 17:27:17.275776 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.271904    4276 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 31 17:27:17.275776 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.271906    4276 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 31 17:27:17.275776 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.271909    4276 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 31 17:27:17.275776 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.271911    4276 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 31 17:27:17.275776 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.271914    4276 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 31 17:27:17.275776 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.271918    4276 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 31 17:27:17.275776 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.271921    4276 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 31 17:27:17.275776 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.271924    4276 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 31 17:27:17.275776 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.272433    4276 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 31 17:27:17.275776 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.272438    4276 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 31 17:27:17.275776 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.272454    4276 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 31 17:27:17.275776 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.272457    4276 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 31 17:27:17.275776 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.272460    4276 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 31 17:27:17.276268 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.272464    4276 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 31 17:27:17.276268 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.272467    4276 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 31 17:27:17.276268 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.272470    4276 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 31 17:27:17.276268 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.272472    4276 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 31 17:27:17.276268 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.272475    4276 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 31 17:27:17.276268 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.272478    4276 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 31 17:27:17.276268 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.272481    4276 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 31 17:27:17.276268 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.272483    4276 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 31 17:27:17.276268 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.272486    4276 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 31 17:27:17.276268 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.272488    4276 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 31 17:27:17.276268 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.272491    4276 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 31 17:27:17.276268 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.272494    4276 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 31 17:27:17.276268 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.272496    4276 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 31 17:27:17.276268 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.272499    4276 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 31 17:27:17.276268 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.272501    4276 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 31 17:27:17.276268 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.272504    4276 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 31 17:27:17.276268 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.272506    4276 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 31 17:27:17.276268 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.272509    4276 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 31 17:27:17.276268 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.272511    4276 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 31 17:27:17.276268 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.272514    4276 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 31 17:27:17.276777 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.272516    4276 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 31 17:27:17.276777 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.272519    4276 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 31 17:27:17.276777 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.272522    4276 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 31 17:27:17.276777 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.272524    4276 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 31 17:27:17.276777 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.272527    4276 feature_gate.go:386] unrecognized feature gate: Example2
Oct 31 17:27:17.276777 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.272530    4276 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 31 17:27:17.276777 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.272532    4276 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 31 17:27:17.276777 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.272535    4276 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 31 17:27:17.276777 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.272538    4276 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 31 17:27:17.276777 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.272540    4276 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 31 17:27:17.276777 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.272543    4276 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 31 17:27:17.276777 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.272545    4276 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 31 17:27:17.276777 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.272548    4276 feature_gate.go:386] unrecognized feature gate: Example
Oct 31 17:27:17.276777 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.272551    4276 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 31 17:27:17.276777 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.272553    4276 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 31 17:27:17.276777 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.272556    4276 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 31 17:27:17.276777 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.272559    4276 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 31 17:27:17.276777 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.272561    4276 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 31 17:27:17.276777 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.272564    4276 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 31 17:27:17.277258 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.272568    4276 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 31 17:27:17.277258 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.272571    4276 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 31 17:27:17.277258 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.272574    4276 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 31 17:27:17.277258 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.272577    4276 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 31 17:27:17.277258 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.272580    4276 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 31 17:27:17.277258 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.272583    4276 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 31 17:27:17.277258 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.272586    4276 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 31 17:27:17.277258 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.272588    4276 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 31 17:27:17.277258 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.272591    4276 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 31 17:27:17.277258 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.272593    4276 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 31 17:27:17.277258 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.272596    4276 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 31 17:27:17.277258 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.272598    4276 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 31 17:27:17.277258 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.272601    4276 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 31 17:27:17.277258 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.272603    4276 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 31 17:27:17.277258 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.272606    4276 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 31 17:27:17.277258 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.272608    4276 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 31 17:27:17.277258 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.272611    4276 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 31 17:27:17.277258 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.272613    4276 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 31 17:27:17.277258 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.272616    4276 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 31 17:27:17.277777 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.272619    4276 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 31 17:27:17.277777 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.272621    4276 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 31 17:27:17.277777 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.272624    4276 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 31 17:27:17.277777 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.272627    4276 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 31 17:27:17.277777 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.272630    4276 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 31 17:27:17.277777 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.272633    4276 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 31 17:27:17.277777 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.272635    4276 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 31 17:27:17.277777 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.272638    4276 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 31 17:27:17.277777 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.272640    4276 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 31 17:27:17.277777 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.272643    4276 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 31 17:27:17.277777 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.272645    4276 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 31 17:27:17.277777 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.272648    4276 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 31 17:27:17.277777 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.272650    4276 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 31 17:27:17.277777 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.272653    4276 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 31 17:27:17.277777 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.272655    4276 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 31 17:27:17.277777 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.272658    4276 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 31 17:27:17.277777 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.272661    4276 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 31 17:27:17.277777 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.272663    4276 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 31 17:27:17.277777 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.272666    4276 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 31 17:27:17.278253 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.272668    4276 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 31 17:27:17.278253 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.272671    4276 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 31 17:27:17.278253 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.272673    4276 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 31 17:27:17.278253 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.272677    4276 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 31 17:27:17.278253 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.272683    4276 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 31 17:27:17.278253 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.272686    4276 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 31 17:27:17.278253 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.272690    4276 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 31 17:27:17.278253 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.272692    4276 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 31 17:27:17.278253 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.272695    4276 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 31 17:27:17.278253 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.272697    4276 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 31 17:27:17.278253 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.272700    4276 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 31 17:27:17.278253 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.272703    4276 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 31 17:27:17.278253 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.272705    4276 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 31 17:27:17.278253 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.272708    4276 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 31 17:27:17.278253 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.272711    4276 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 31 17:27:17.278253 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.272713    4276 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 31 17:27:17.278253 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.272716    4276 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 31 17:27:17.278253 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.272718    4276 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 31 17:27:17.278253 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.272721    4276 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 31 17:27:17.278734 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.272723    4276 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 31 17:27:17.278734 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.272726    4276 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 31 17:27:17.278734 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.272728    4276 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 31 17:27:17.278734 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.272731    4276 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 31 17:27:17.278734 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.272733    4276 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 31 17:27:17.278734 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.272824    4276 flags.go:64] FLAG: --address="0.0.0.0"
Oct 31 17:27:17.278734 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.272832    4276 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Oct 31 17:27:17.278734 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.272837    4276 flags.go:64] FLAG: --anonymous-auth="true"
Oct 31 17:27:17.278734 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.272842    4276 flags.go:64] FLAG: --application-metrics-count-limit="100"
Oct 31 17:27:17.278734 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.272847    4276 flags.go:64] FLAG: --authentication-token-webhook="false"
Oct 31 17:27:17.278734 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.272849    4276 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Oct 31 17:27:17.278734 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.272854    4276 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Oct 31 17:27:17.278734 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.272859    4276 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Oct 31 17:27:17.278734 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.272864    4276 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Oct 31 17:27:17.278734 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.272867    4276 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Oct 31 17:27:17.278734 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.272871    4276 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Oct 31 17:27:17.278734 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.272874    4276 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Oct 31 17:27:17.278734 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.272879    4276 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Oct 31 17:27:17.278734 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.272882    4276 flags.go:64] FLAG: --cgroup-root=""
Oct 31 17:27:17.278734 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.272885    4276 flags.go:64] FLAG: --cgroups-per-qos="true"
Oct 31 17:27:17.278734 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.272888    4276 flags.go:64] FLAG: --client-ca-file=""
Oct 31 17:27:17.278734 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.272894    4276 flags.go:64] FLAG: --cloud-provider="external"
Oct 31 17:27:17.278734 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.272897    4276 flags.go:64] FLAG: --cluster-dns="[]"
Oct 31 17:27:17.278734 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.272901    4276 flags.go:64] FLAG: --cluster-domain=""
Oct 31 17:27:17.279370 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.272904    4276 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Oct 31 17:27:17.279370 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.272907    4276 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Oct 31 17:27:17.279370 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.272911    4276 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Oct 31 17:27:17.279370 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.272914    4276 flags.go:64] FLAG: --container-log-max-files="5"
Oct 31 17:27:17.279370 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.272919    4276 flags.go:64] FLAG: --container-log-max-size="10Mi"
Oct 31 17:27:17.279370 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.272922    4276 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Oct 31 17:27:17.279370 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.272925    4276 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Oct 31 17:27:17.279370 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.272928    4276 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Oct 31 17:27:17.279370 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.272931    4276 flags.go:64] FLAG: --contention-profiling="false"
Oct 31 17:27:17.279370 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.272934    4276 flags.go:64] FLAG: --cpu-cfs-quota="true"
Oct 31 17:27:17.279370 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.272938    4276 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Oct 31 17:27:17.279370 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.272941    4276 flags.go:64] FLAG: --cpu-manager-policy="none"
Oct 31 17:27:17.279370 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.272944    4276 flags.go:64] FLAG: --cpu-manager-policy-options=""
Oct 31 17:27:17.279370 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.272948    4276 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Oct 31 17:27:17.279370 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.272951    4276 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Oct 31 17:27:17.279370 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.272954    4276 flags.go:64] FLAG: --enable-debugging-handlers="true"
Oct 31 17:27:17.279370 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.272957    4276 flags.go:64] FLAG: --enable-load-reader="false"
Oct 31 17:27:17.279370 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.272960    4276 flags.go:64] FLAG: --enable-server="true"
Oct 31 17:27:17.279370 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.272963    4276 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Oct 31 17:27:17.279370 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.272967    4276 flags.go:64] FLAG: --event-burst="100"
Oct 31 17:27:17.279370 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.272970    4276 flags.go:64] FLAG: --event-qps="50"
Oct 31 17:27:17.279370 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.272973    4276 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Oct 31 17:27:17.279370 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.272976    4276 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Oct 31 17:27:17.279370 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.272979    4276 flags.go:64] FLAG: --eviction-hard=""
Oct 31 17:27:17.280158 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.272983    4276 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Oct 31 17:27:17.280158 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.272988    4276 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Oct 31 17:27:17.280158 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.272992    4276 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Oct 31 17:27:17.280158 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.272995    4276 flags.go:64] FLAG: --eviction-soft=""
Oct 31 17:27:17.280158 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.272998    4276 flags.go:64] FLAG: --eviction-soft-grace-period=""
Oct 31 17:27:17.280158 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.273002    4276 flags.go:64] FLAG: --exit-on-lock-contention="false"
Oct 31 17:27:17.280158 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.273005    4276 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Oct 31 17:27:17.280158 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.273008    4276 flags.go:64] FLAG: --experimental-mounter-path=""
Oct 31 17:27:17.280158 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.273011    4276 flags.go:64] FLAG: --fail-cgroupv1="true"
Oct 31 17:27:17.280158 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.273014    4276 flags.go:64] FLAG: --fail-swap-on="true"
Oct 31 17:27:17.280158 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.273017    4276 flags.go:64] FLAG: --feature-gates=""
Oct 31 17:27:17.280158 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.273021    4276 flags.go:64] FLAG: --file-check-frequency="20s"
Oct 31 17:27:17.280158 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.273024    4276 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Oct 31 17:27:17.280158 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.273027    4276 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Oct 31 17:27:17.280158 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.273031    4276 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Oct 31 17:27:17.280158 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.273034    4276 flags.go:64] FLAG: --healthz-port="10248"
Oct 31 17:27:17.280158 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.273037    4276 flags.go:64] FLAG: --help="false"
Oct 31 17:27:17.280158 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.273040    4276 flags.go:64] FLAG: --hostname-override="ip-10-0-129-127.ec2.internal"
Oct 31 17:27:17.280158 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.273043    4276 flags.go:64] FLAG: --housekeeping-interval="10s"
Oct 31 17:27:17.280158 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.273046    4276 flags.go:64] FLAG: --http-check-frequency="20s"
Oct 31 17:27:17.280158 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.273049    4276 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Oct 31 17:27:17.280158 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.273053    4276 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Oct 31 17:27:17.280158 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.273056    4276 flags.go:64] FLAG: --image-gc-high-threshold="85"
Oct 31 17:27:17.280800 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.273060    4276 flags.go:64] FLAG: --image-gc-low-threshold="80"
Oct 31 17:27:17.280800 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.273063    4276 flags.go:64] FLAG: --image-service-endpoint=""
Oct 31 17:27:17.280800 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.273065    4276 flags.go:64] FLAG: --kernel-memcg-notification="false"
Oct 31 17:27:17.280800 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.273068    4276 flags.go:64] FLAG: --kube-api-burst="100"
Oct 31 17:27:17.280800 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.273071    4276 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Oct 31 17:27:17.280800 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.273075    4276 flags.go:64] FLAG: --kube-api-qps="50"
Oct 31 17:27:17.280800 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.273077    4276 flags.go:64] FLAG: --kube-reserved=""
Oct 31 17:27:17.280800 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.273080    4276 flags.go:64] FLAG: --kube-reserved-cgroup=""
Oct 31 17:27:17.280800 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.273083    4276 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Oct 31 17:27:17.280800 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.273087    4276 flags.go:64] FLAG: --kubelet-cgroups=""
Oct 31 17:27:17.280800 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.273091    4276 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Oct 31 17:27:17.280800 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.273094    4276 flags.go:64] FLAG: --lock-file=""
Oct 31 17:27:17.280800 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.273097    4276 flags.go:64] FLAG: --log-cadvisor-usage="false"
Oct 31 17:27:17.280800 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.273100    4276 flags.go:64] FLAG: --log-flush-frequency="5s"
Oct 31 17:27:17.280800 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.273105    4276 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Oct 31 17:27:17.280800 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.273110    4276 flags.go:64] FLAG: --log-json-split-stream="false"
Oct 31 17:27:17.280800 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.273113    4276 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Oct 31 17:27:17.280800 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.273116    4276 flags.go:64] FLAG: --log-text-split-stream="false"
Oct 31 17:27:17.280800 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.273119    4276 flags.go:64] FLAG: --logging-format="text"
Oct 31 17:27:17.280800 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.273122    4276 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Oct 31 17:27:17.280800 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.273125    4276 flags.go:64] FLAG: --make-iptables-util-chains="true"
Oct 31 17:27:17.280800 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.273128    4276 flags.go:64] FLAG: --manifest-url=""
Oct 31 17:27:17.280800 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.273131    4276 flags.go:64] FLAG: --manifest-url-header=""
Oct 31 17:27:17.280800 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.273139    4276 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Oct 31 17:27:17.280800 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.273143    4276 flags.go:64] FLAG: --max-open-files="1000000"
Oct 31 17:27:17.281416 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.273146    4276 flags.go:64] FLAG: --max-pods="110"
Oct 31 17:27:17.281416 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.273150    4276 flags.go:64] FLAG: --maximum-dead-containers="-1"
Oct 31 17:27:17.281416 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.273152    4276 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Oct 31 17:27:17.281416 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.273155    4276 flags.go:64] FLAG: --memory-manager-policy="None"
Oct 31 17:27:17.281416 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.273158    4276 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Oct 31 17:27:17.281416 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.273161    4276 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Oct 31 17:27:17.281416 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.273164    4276 flags.go:64] FLAG: --node-ip="0.0.0.0"
Oct 31 17:27:17.281416 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.273167    4276 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Oct 31 17:27:17.281416 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.273174    4276 flags.go:64] FLAG: --node-status-max-images="50"
Oct 31 17:27:17.281416 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.273177    4276 flags.go:64] FLAG: --node-status-update-frequency="10s"
Oct 31 17:27:17.281416 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.273180    4276 flags.go:64] FLAG: --oom-score-adj="-999"
Oct 31 17:27:17.281416 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.273184    4276 flags.go:64] FLAG: --pod-cidr=""
Oct 31 17:27:17.281416 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.273187    4276 flags.go:64] FLAG: --pod-manifest-path=""
Oct 31 17:27:17.281416 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.273190    4276 flags.go:64] FLAG: --pod-max-pids="-1"
Oct 31 17:27:17.281416 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.273193    4276 flags.go:64] FLAG: --pods-per-core="0"
Oct 31 17:27:17.281416 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.273196    4276 flags.go:64] FLAG: --port="10250"
Oct 31 17:27:17.281416 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.273199    4276 flags.go:64] FLAG: --protect-kernel-defaults="false"
Oct 31 17:27:17.281416 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.273204    4276 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-017ba90e9c135b582"
Oct 31 17:27:17.281416 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.273207    4276 flags.go:64] FLAG: --qos-reserved=""
Oct 31 17:27:17.281416 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.273210    4276 flags.go:64] FLAG: --read-only-port="10255"
Oct 31 17:27:17.281416 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.273213    4276 flags.go:64] FLAG: --register-node="true"
Oct 31 17:27:17.281416 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.273218    4276 flags.go:64] FLAG: --register-with-taints=""
Oct 31 17:27:17.281416 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.273222    4276 flags.go:64] FLAG: --registry-burst="10"
Oct 31 17:27:17.281416 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.273225    4276 flags.go:64] FLAG: --registry-qps="5"
Oct 31 17:27:17.281416 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.273228    4276 flags.go:64] FLAG: --reserved-cpus=""
Oct 31 17:27:17.281416 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.273231    4276 flags.go:64] FLAG: --reserved-memory=""
Oct 31 17:27:17.282223 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.273235    4276 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Oct 31 17:27:17.282223 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.273238    4276 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Oct 31 17:27:17.282223 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.273241    4276 flags.go:64] FLAG: --rotate-certificates="false"
Oct 31 17:27:17.282223 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.273244    4276 flags.go:64] FLAG: --rotate-server-certificates="false"
Oct 31 17:27:17.282223 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.273247    4276 flags.go:64] FLAG: --runonce="false"
Oct 31 17:27:17.282223 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.273252    4276 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Oct 31 17:27:17.282223 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.273256    4276 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Oct 31 17:27:17.282223 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.273259    4276 flags.go:64] FLAG: --seccomp-default="false"
Oct 31 17:27:17.282223 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.273262    4276 flags.go:64] FLAG: --serialize-image-pulls="true"
Oct 31 17:27:17.282223 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.273265    4276 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Oct 31 17:27:17.282223 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.273268    4276 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Oct 31 17:27:17.282223 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.273271    4276 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Oct 31 17:27:17.282223 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.273274    4276 flags.go:64] FLAG: --storage-driver-password="root"
Oct 31 17:27:17.282223 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.273277    4276 flags.go:64] FLAG: --storage-driver-secure="false"
Oct 31 17:27:17.282223 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.273280    4276 flags.go:64] FLAG: --storage-driver-table="stats"
Oct 31 17:27:17.282223 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.273284    4276 flags.go:64] FLAG: --storage-driver-user="root"
Oct 31 17:27:17.282223 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.273286    4276 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Oct 31 17:27:17.282223 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.273290    4276 flags.go:64] FLAG: --sync-frequency="1m0s"
Oct 31 17:27:17.282223 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.273293    4276 flags.go:64] FLAG: --system-cgroups=""
Oct 31 17:27:17.282223 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.273296    4276 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Oct 31 17:27:17.282223 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.273301    4276 flags.go:64] FLAG: --system-reserved-cgroup=""
Oct 31 17:27:17.282223 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.273304    4276 flags.go:64] FLAG: --tls-cert-file=""
Oct 31 17:27:17.282223 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.273307    4276 flags.go:64] FLAG: --tls-cipher-suites="[]"
Oct 31 17:27:17.282223 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.273312    4276 flags.go:64] FLAG: --tls-min-version=""
Oct 31 17:27:17.282223 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.273314    4276 flags.go:64] FLAG: --tls-private-key-file=""
Oct 31 17:27:17.282887 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.273317    4276 flags.go:64] FLAG: --topology-manager-policy="none"
Oct 31 17:27:17.282887 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.273320    4276 flags.go:64] FLAG: --topology-manager-policy-options=""
Oct 31 17:27:17.282887 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.273324    4276 flags.go:64] FLAG: --topology-manager-scope="container"
Oct 31 17:27:17.282887 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.273328    4276 flags.go:64] FLAG: --v="2"
Oct 31 17:27:17.282887 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.273332    4276 flags.go:64] FLAG: --version="false"
Oct 31 17:27:17.282887 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.273336    4276 flags.go:64] FLAG: --vmodule=""
Oct 31 17:27:17.282887 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.273340    4276 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Oct 31 17:27:17.282887 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.273344    4276 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Oct 31 17:27:17.282887 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.273527    4276 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 31 17:27:17.282887 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.273531    4276 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 31 17:27:17.282887 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.273534    4276 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 31 17:27:17.282887 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.273537    4276 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 31 17:27:17.282887 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.273539    4276 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 31 17:27:17.282887 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.273548    4276 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 31 17:27:17.282887 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.273551    4276 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 31 17:27:17.282887 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.273554    4276 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 31 17:27:17.282887 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.273556    4276 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 31 17:27:17.282887 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.273559    4276 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 31 17:27:17.282887 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.273562    4276 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 31 17:27:17.282887 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.273564    4276 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 31 17:27:17.282887 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.273567    4276 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 31 17:27:17.283397 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.273570    4276 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 31 17:27:17.283397 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.273572    4276 feature_gate.go:386] unrecognized feature gate: Example2
Oct 31 17:27:17.283397 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.273576    4276 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 31 17:27:17.283397 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.273579    4276 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 31 17:27:17.283397 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.273582    4276 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 31 17:27:17.283397 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.273585    4276 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 31 17:27:17.283397 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.273588    4276 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 31 17:27:17.283397 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.273591    4276 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 31 17:27:17.283397 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.273594    4276 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 31 17:27:17.283397 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.273598    4276 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 31 17:27:17.283397 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.273600    4276 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 31 17:27:17.283397 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.273603    4276 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 31 17:27:17.283397 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.273607    4276 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 31 17:27:17.283397 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.273612    4276 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 31 17:27:17.283397 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.273616    4276 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 31 17:27:17.283397 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.273620    4276 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 31 17:27:17.283397 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.273623    4276 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 31 17:27:17.283397 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.273626    4276 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 31 17:27:17.283397 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.273628    4276 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 31 17:27:17.283879 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.273632    4276 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 31 17:27:17.283879 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.273634    4276 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 31 17:27:17.283879 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.273637    4276 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 31 17:27:17.283879 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.273639    4276 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 31 17:27:17.283879 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.273642    4276 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 31 17:27:17.283879 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.273644    4276 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 31 17:27:17.283879 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.273649    4276 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 31 17:27:17.283879 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.273651    4276 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 31 17:27:17.283879 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.273654    4276 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 31 17:27:17.283879 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.273656    4276 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 31 17:27:17.283879 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.273659    4276 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 31 17:27:17.283879 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.273662    4276 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 31 17:27:17.283879 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.273664    4276 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 31 17:27:17.283879 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.273667    4276 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 31 17:27:17.283879 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.273669    4276 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 31 17:27:17.283879 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.273672    4276 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 31 17:27:17.283879 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.273674    4276 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 31 17:27:17.283879 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.273677    4276 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 31 17:27:17.283879 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.273679    4276 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 31 17:27:17.283879 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.273682    4276 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 31 17:27:17.284373 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.273685    4276 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 31 17:27:17.284373 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.273687    4276 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 31 17:27:17.284373 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.273691    4276 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 31 17:27:17.284373 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.273693    4276 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 31 17:27:17.284373 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.273696    4276 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 31 17:27:17.284373 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.273698    4276 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 31 17:27:17.284373 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.273703    4276 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 31 17:27:17.284373 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.273706    4276 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 31 17:27:17.284373 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.273708    4276 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 31 17:27:17.284373 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.273711    4276 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 31 17:27:17.284373 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.273713    4276 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 31 17:27:17.284373 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.273716    4276 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 31 17:27:17.284373 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.273718    4276 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 31 17:27:17.284373 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.273721    4276 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 31 17:27:17.284373 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.273723    4276 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 31 17:27:17.284373 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.273726    4276 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 31 17:27:17.284373 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.273729    4276 feature_gate.go:386] unrecognized feature gate: Example
Oct 31 17:27:17.284373 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.273731    4276 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 31 17:27:17.284373 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.273733    4276 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 31 17:27:17.284849 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.273736    4276 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 31 17:27:17.284849 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.273739    4276 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 31 17:27:17.284849 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.273741    4276 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 31 17:27:17.284849 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.273744    4276 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 31 17:27:17.284849 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.273747    4276 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 31 17:27:17.284849 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.273749    4276 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 31 17:27:17.284849 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.273752    4276 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 31 17:27:17.284849 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.273754    4276 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 31 17:27:17.284849 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.273757    4276 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 31 17:27:17.284849 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.273759    4276 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 31 17:27:17.284849 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.273762    4276 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 31 17:27:17.284849 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.273764    4276 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 31 17:27:17.284849 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.273767    4276 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 31 17:27:17.284849 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.273769    4276 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 31 17:27:17.284849 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.273772    4276 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 31 17:27:17.284849 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.273775    4276 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 31 17:27:17.284849 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.273778    4276 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 31 17:27:17.284849 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.273781    4276 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 31 17:27:17.284849 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.273784    4276 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 31 17:27:17.285326 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.273801    4276 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 31 17:27:17.285326 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.273805    4276 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 31 17:27:17.285326 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.273808    4276 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 31 17:27:17.285326 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.273811    4276 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 31 17:27:17.285326 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.273814    4276 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 31 17:27:17.285326 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.273817    4276 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 31 17:27:17.285326 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.273819    4276 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 31 17:27:17.285326 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.273821    4276 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 31 17:27:17.285326 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.273824    4276 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 31 17:27:17.285326 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.273827    4276 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 31 17:27:17.285326 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.273829    4276 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 31 17:27:17.285326 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.273831    4276 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 31 17:27:17.285326 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.273834    4276 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 31 17:27:17.285326 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.273837    4276 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 31 17:27:17.285326 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.273840    4276 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 31 17:27:17.285326 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.273842    4276 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 31 17:27:17.285818 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.273868    4276 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 31 17:27:17.285818 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.273904    4276 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 31 17:27:17.285818 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.273909    4276 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 31 17:27:17.285818 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.273911    4276 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 31 17:27:17.285818 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.273914    4276 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 31 17:27:17.285818 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.273918    4276 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 31 17:27:17.285818 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.273922    4276 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 31 17:27:17.285818 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.273925    4276 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 31 17:27:17.285818 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.273927    4276 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 31 17:27:17.285818 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.273930    4276 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 31 17:27:17.285818 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.273932    4276 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 31 17:27:17.285818 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.273935    4276 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 31 17:27:17.285818 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.273939    4276 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 31 17:27:17.285818 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.273941    4276 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 31 17:27:17.285818 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.273944    4276 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 31 17:27:17.285818 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.273947    4276 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 31 17:27:17.286223 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.273949    4276 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 31 17:27:17.286223 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.273952    4276 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 31 17:27:17.286223 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.273954    4276 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 31 17:27:17.286223 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.273957    4276 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 31 17:27:17.286223 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.273960    4276 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 31 17:27:17.286223 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.273962    4276 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 31 17:27:17.286223 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.273965    4276 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 31 17:27:17.286223 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.273967    4276 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 31 17:27:17.286223 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.273970    4276 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 31 17:27:17.286223 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.273973    4276 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 31 17:27:17.286223 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.273975    4276 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 31 17:27:17.286223 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.273977    4276 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 31 17:27:17.286223 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.273980    4276 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 31 17:27:17.286223 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.273983    4276 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 31 17:27:17.286223 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.273985    4276 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 31 17:27:17.286223 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.273988    4276 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 31 17:27:17.286223 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.273991    4276 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 31 17:27:17.286223 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.273994    4276 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 31 17:27:17.286223 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.273996    4276 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 31 17:27:17.286709 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.273999    4276 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 31 17:27:17.286709 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.274001    4276 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 31 17:27:17.286709 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.274004    4276 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 31 17:27:17.286709 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.274007    4276 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 31 17:27:17.286709 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.274010    4276 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 31 17:27:17.286709 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.274012    4276 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 31 17:27:17.286709 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.274015    4276 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 31 17:27:17.286709 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.274017    4276 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 31 17:27:17.286709 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.274019    4276 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 31 17:27:17.286709 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.274023    4276 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 31 17:27:17.286709 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.274026    4276 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 31 17:27:17.286709 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.274028    4276 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 31 17:27:17.286709 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.274031    4276 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 31 17:27:17.286709 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.274034    4276 feature_gate.go:386] unrecognized feature gate: Example
Oct 31 17:27:17.286709 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.274037    4276 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 31 17:27:17.286709 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.274039    4276 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 31 17:27:17.286709 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.274042    4276 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 31 17:27:17.286709 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.274044    4276 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 31 17:27:17.286709 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.274047    4276 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 31 17:27:17.286709 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.274049    4276 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 31 17:27:17.287207 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.274052    4276 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 31 17:27:17.287207 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.274054    4276 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 31 17:27:17.287207 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.274057    4276 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 31 17:27:17.287207 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.274060    4276 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 31 17:27:17.287207 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.274062    4276 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 31 17:27:17.287207 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.274065    4276 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 31 17:27:17.287207 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.274068    4276 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 31 17:27:17.287207 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.274070    4276 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 31 17:27:17.287207 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.274073    4276 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 31 17:27:17.287207 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.274076    4276 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 31 17:27:17.287207 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.274079    4276 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 31 17:27:17.287207 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.274081    4276 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 31 17:27:17.287207 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.274084    4276 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 31 17:27:17.287207 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.274086    4276 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 31 17:27:17.287207 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.274090    4276 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 31 17:27:17.287207 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.274093    4276 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 31 17:27:17.287207 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.274095    4276 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 31 17:27:17.287207 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.274098    4276 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 31 17:27:17.287207 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.274100    4276 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 31 17:27:17.287207 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.274102    4276 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 31 17:27:17.287754 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.274105    4276 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 31 17:27:17.287754 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.274109    4276 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 31 17:27:17.287754 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.274112    4276 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 31 17:27:17.287754 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.274114    4276 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 31 17:27:17.287754 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.274118    4276 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 31 17:27:17.287754 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.274120    4276 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 31 17:27:17.287754 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.274123    4276 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 31 17:27:17.287754 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.274126    4276 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 31 17:27:17.287754 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.274129    4276 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 31 17:27:17.287754 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.274131    4276 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 31 17:27:17.287754 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.274134    4276 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 31 17:27:17.287754 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.274137    4276 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 31 17:27:17.287754 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.274139    4276 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 31 17:27:17.287754 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.274142    4276 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 31 17:27:17.287754 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.274145    4276 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 31 17:27:17.287754 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.274147    4276 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 31 17:27:17.287754 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.274150    4276 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 31 17:27:17.287754 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.274153    4276 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 31 17:27:17.287754 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.274155    4276 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 31 17:27:17.287754 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.274158    4276 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 31 17:27:17.288301 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.274161    4276 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 31 17:27:17.288301 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.274163    4276 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 31 17:27:17.288301 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.274166    4276 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 31 17:27:17.288301 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.274169    4276 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 31 17:27:17.288301 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.274172    4276 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 31 17:27:17.288301 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.274174    4276 feature_gate.go:386] unrecognized feature gate: Example2
Oct 31 17:27:17.288301 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.274179    4276 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 31 17:27:17.288301 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.274182    4276 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 31 17:27:17.288301 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.274185    4276 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 31 17:27:17.288301 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.274188    4276 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 31 17:27:17.288301 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.274190    4276 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 31 17:27:17.288301 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.274193    4276 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 31 17:27:17.288301 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.278921    4276 server.go:525] "Kubelet version" kubeletVersion="v1.35.9"
Oct 31 17:27:17.288301 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.278935    4276 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Oct 31 17:27:17.288301 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.278951    4276 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Oct 31 17:27:17.288301 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.278956    4276 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Oct 31 17:27:17.288301 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279016    4276 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 31 17:27:17.288301 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279023    4276 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 31 17:27:17.288301 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279026    4276 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 31 17:27:17.288784 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279029    4276 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 31 17:27:17.288784 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279032    4276 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 31 17:27:17.288784 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279035    4276 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 31 17:27:17.288784 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279038    4276 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 31 17:27:17.288784 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279041    4276 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 31 17:27:17.288784 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279043    4276 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 31 17:27:17.288784 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279046    4276 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 31 17:27:17.288784 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279048    4276 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 31 17:27:17.288784 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279051    4276 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 31 17:27:17.288784 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279054    4276 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 31 17:27:17.288784 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279057    4276 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 31 17:27:17.288784 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279060    4276 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 31 17:27:17.288784 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279063    4276 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 31 17:27:17.288784 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279066    4276 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 31 17:27:17.288784 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279068    4276 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 31 17:27:17.288784 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279072    4276 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 31 17:27:17.288784 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279074    4276 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 31 17:27:17.288784 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279077    4276 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 31 17:27:17.288784 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279080    4276 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 31 17:27:17.288784 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279083    4276 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 31 17:27:17.289266 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279085    4276 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 31 17:27:17.289266 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279088    4276 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 31 17:27:17.289266 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279090    4276 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 31 17:27:17.289266 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279093    4276 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 31 17:27:17.289266 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279096    4276 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 31 17:27:17.289266 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279098    4276 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 31 17:27:17.289266 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279102    4276 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 31 17:27:17.289266 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279108    4276 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 31 17:27:17.289266 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279112    4276 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 31 17:27:17.289266 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279115    4276 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 31 17:27:17.289266 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279119    4276 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 31 17:27:17.289266 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279121    4276 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 31 17:27:17.289266 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279124    4276 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 31 17:27:17.289266 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279127    4276 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 31 17:27:17.289266 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279130    4276 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 31 17:27:17.289266 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279132    4276 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 31 17:27:17.289266 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279135    4276 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 31 17:27:17.289266 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279137    4276 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 31 17:27:17.289266 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279140    4276 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 31 17:27:17.289740 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279142    4276 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 31 17:27:17.289740 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279145    4276 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 31 17:27:17.289740 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279148    4276 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 31 17:27:17.289740 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279150    4276 feature_gate.go:386] unrecognized feature gate: Example2
Oct 31 17:27:17.289740 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279153    4276 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 31 17:27:17.289740 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279156    4276 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 31 17:27:17.289740 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279158    4276 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 31 17:27:17.289740 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279161    4276 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 31 17:27:17.289740 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279163    4276 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 31 17:27:17.289740 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279166    4276 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 31 17:27:17.289740 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279169    4276 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 31 17:27:17.289740 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279172    4276 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 31 17:27:17.289740 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279175    4276 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 31 17:27:17.289740 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279177    4276 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 31 17:27:17.289740 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279180    4276 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 31 17:27:17.289740 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279183    4276 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 31 17:27:17.289740 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279185    4276 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 31 17:27:17.289740 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279188    4276 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 31 17:27:17.289740 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279191    4276 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 31 17:27:17.289740 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279194    4276 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 31 17:27:17.290230 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279196    4276 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 31 17:27:17.290230 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279199    4276 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 31 17:27:17.290230 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279202    4276 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 31 17:27:17.290230 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279205    4276 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 31 17:27:17.290230 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279208    4276 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 31 17:27:17.290230 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279210    4276 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 31 17:27:17.290230 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279213    4276 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 31 17:27:17.290230 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279215    4276 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 31 17:27:17.290230 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279218    4276 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 31 17:27:17.290230 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279220    4276 feature_gate.go:386] unrecognized feature gate: Example
Oct 31 17:27:17.290230 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279223    4276 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 31 17:27:17.290230 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279226    4276 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 31 17:27:17.290230 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279228    4276 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 31 17:27:17.290230 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279231    4276 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 31 17:27:17.290230 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279233    4276 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 31 17:27:17.290230 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279236    4276 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 31 17:27:17.290230 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279239    4276 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 31 17:27:17.290230 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279241    4276 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 31 17:27:17.290230 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279244    4276 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 31 17:27:17.290230 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279246    4276 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 31 17:27:17.290754 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279249    4276 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 31 17:27:17.290754 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279251    4276 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 31 17:27:17.290754 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279254    4276 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 31 17:27:17.290754 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279257    4276 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 31 17:27:17.290754 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279260    4276 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 31 17:27:17.290754 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279262    4276 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 31 17:27:17.290754 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279265    4276 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 31 17:27:17.290754 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279268    4276 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 31 17:27:17.290754 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279270    4276 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 31 17:27:17.290754 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279273    4276 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 31 17:27:17.290754 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279276    4276 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 31 17:27:17.290754 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279279    4276 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 31 17:27:17.290754 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279282    4276 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 31 17:27:17.290754 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279284    4276 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 31 17:27:17.290754 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279287    4276 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 31 17:27:17.290754 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279289    4276 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 31 17:27:17.290754 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279292    4276 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 31 17:27:17.290754 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279294    4276 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 31 17:27:17.290754 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279297    4276 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 31 17:27:17.291227 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279299    4276 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 31 17:27:17.291227 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279302    4276 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 31 17:27:17.291227 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279305    4276 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 31 17:27:17.291227 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279307    4276 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 31 17:27:17.291227 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279310    4276 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 31 17:27:17.291227 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.279346    4276 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 31 17:27:17.291227 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279496    4276 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 31 17:27:17.291227 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279502    4276 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 31 17:27:17.291227 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279505    4276 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 31 17:27:17.291227 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279508    4276 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 31 17:27:17.291227 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279511    4276 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 31 17:27:17.291227 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279514    4276 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 31 17:27:17.291227 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279516    4276 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 31 17:27:17.291227 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279519    4276 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 31 17:27:17.291227 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279522    4276 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 31 17:27:17.291227 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279524    4276 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 31 17:27:17.291669 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279528    4276 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 31 17:27:17.291669 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279531    4276 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 31 17:27:17.291669 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279534    4276 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 31 17:27:17.291669 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279537    4276 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 31 17:27:17.291669 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279539    4276 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 31 17:27:17.291669 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279542    4276 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 31 17:27:17.291669 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279545    4276 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 31 17:27:17.291669 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279548    4276 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 31 17:27:17.291669 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279551    4276 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 31 17:27:17.291669 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279553    4276 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 31 17:27:17.291669 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279556    4276 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 31 17:27:17.291669 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279559    4276 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 31 17:27:17.291669 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279561    4276 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 31 17:27:17.291669 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279564    4276 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 31 17:27:17.291669 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279566    4276 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 31 17:27:17.291669 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279569    4276 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 31 17:27:17.291669 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279572    4276 feature_gate.go:386] unrecognized feature gate: Example2
Oct 31 17:27:17.291669 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279575    4276 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 31 17:27:17.291669 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279577    4276 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 31 17:27:17.291669 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279580    4276 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 31 17:27:17.292150 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279582    4276 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 31 17:27:17.292150 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279585    4276 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 31 17:27:17.292150 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279587    4276 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 31 17:27:17.292150 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279590    4276 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 31 17:27:17.292150 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279593    4276 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 31 17:27:17.292150 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279595    4276 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 31 17:27:17.292150 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279597    4276 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 31 17:27:17.292150 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279601    4276 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 31 17:27:17.292150 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279603    4276 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 31 17:27:17.292150 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279606    4276 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 31 17:27:17.292150 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279609    4276 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 31 17:27:17.292150 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279611    4276 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 31 17:27:17.292150 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279614    4276 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 31 17:27:17.292150 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279617    4276 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 31 17:27:17.292150 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279619    4276 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 31 17:27:17.292150 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279622    4276 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 31 17:27:17.292150 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279625    4276 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 31 17:27:17.292150 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279628    4276 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 31 17:27:17.292150 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279631    4276 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 31 17:27:17.292670 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279633    4276 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 31 17:27:17.292670 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279636    4276 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 31 17:27:17.292670 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279638    4276 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 31 17:27:17.292670 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279641    4276 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 31 17:27:17.292670 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279643    4276 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 31 17:27:17.292670 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279646    4276 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 31 17:27:17.292670 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279648    4276 feature_gate.go:386] unrecognized feature gate: Example
Oct 31 17:27:17.292670 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279651    4276 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 31 17:27:17.292670 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279654    4276 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 31 17:27:17.292670 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279656    4276 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 31 17:27:17.292670 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279659    4276 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 31 17:27:17.292670 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279661    4276 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 31 17:27:17.292670 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279664    4276 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 31 17:27:17.292670 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279668    4276 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 31 17:27:17.292670 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279671    4276 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 31 17:27:17.292670 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279675    4276 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 31 17:27:17.292670 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279677    4276 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 31 17:27:17.292670 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279680    4276 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 31 17:27:17.292670 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279683    4276 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 31 17:27:17.293133 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279685    4276 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 31 17:27:17.293133 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279688    4276 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 31 17:27:17.293133 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279690    4276 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 31 17:27:17.293133 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279693    4276 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 31 17:27:17.293133 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279695    4276 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 31 17:27:17.293133 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279698    4276 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 31 17:27:17.293133 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279700    4276 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 31 17:27:17.293133 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279703    4276 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 31 17:27:17.293133 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279706    4276 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 31 17:27:17.293133 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279709    4276 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 31 17:27:17.293133 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279713    4276 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 31 17:27:17.293133 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279716    4276 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 31 17:27:17.293133 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279720    4276 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 31 17:27:17.293133 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279723    4276 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 31 17:27:17.293133 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279726    4276 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 31 17:27:17.293133 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279728    4276 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 31 17:27:17.293133 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279731    4276 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 31 17:27:17.293133 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279733    4276 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 31 17:27:17.293133 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279736    4276 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 31 17:27:17.293721 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279739    4276 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 31 17:27:17.293721 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279741    4276 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 31 17:27:17.293721 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279744    4276 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 31 17:27:17.293721 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279746    4276 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 31 17:27:17.293721 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279749    4276 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 31 17:27:17.293721 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279751    4276 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 31 17:27:17.293721 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279754    4276 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 31 17:27:17.293721 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279756    4276 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 31 17:27:17.293721 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279759    4276 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 31 17:27:17.293721 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279762    4276 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 31 17:27:17.293721 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279764    4276 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 31 17:27:17.293721 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279766    4276 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 31 17:27:17.293721 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279769    4276 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 31 17:27:17.293721 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279772    4276 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 31 17:27:17.293721 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279774    4276 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 31 17:27:17.293721 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279777    4276 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 31 17:27:17.293721 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279779    4276 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 31 17:27:17.293721 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279782    4276 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 31 17:27:17.293721 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279784    4276 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 31 17:27:17.294215 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.279808    4276 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 31 17:27:17.294215 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279846    4276 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 31 17:27:17.294215 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279849    4276 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 31 17:27:17.294215 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279853    4276 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 31 17:27:17.294215 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279856    4276 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 31 17:27:17.294215 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279858    4276 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 31 17:27:17.294215 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279861    4276 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 31 17:27:17.294215 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279864    4276 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 31 17:27:17.294215 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279869    4276 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 31 17:27:17.294215 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279872    4276 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 31 17:27:17.294215 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279875    4276 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 31 17:27:17.294215 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279878    4276 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 31 17:27:17.294215 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279881    4276 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 31 17:27:17.294215 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279886    4276 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 31 17:27:17.294215 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279889    4276 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 31 17:27:17.294619 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279892    4276 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 31 17:27:17.294619 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279895    4276 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 31 17:27:17.294619 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279898    4276 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 31 17:27:17.294619 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279901    4276 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 31 17:27:17.294619 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279904    4276 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 31 17:27:17.294619 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279907    4276 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 31 17:27:17.294619 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279909    4276 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 31 17:27:17.294619 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279912    4276 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 31 17:27:17.294619 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279914    4276 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 31 17:27:17.294619 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279917    4276 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 31 17:27:17.294619 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279920    4276 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 31 17:27:17.294619 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279922    4276 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 31 17:27:17.294619 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279925    4276 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 31 17:27:17.294619 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279928    4276 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 31 17:27:17.294619 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279931    4276 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 31 17:27:17.294619 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279933    4276 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 31 17:27:17.294619 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279936    4276 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 31 17:27:17.294619 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279938    4276 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 31 17:27:17.294619 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279941    4276 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 31 17:27:17.294619 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279943    4276 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 31 17:27:17.295147 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279946    4276 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 31 17:27:17.295147 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279949    4276 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 31 17:27:17.295147 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279952    4276 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 31 17:27:17.295147 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279954    4276 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 31 17:27:17.295147 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279957    4276 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 31 17:27:17.295147 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279961    4276 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 31 17:27:17.295147 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279964    4276 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 31 17:27:17.295147 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279966    4276 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 31 17:27:17.295147 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279969    4276 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 31 17:27:17.295147 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279972    4276 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 31 17:27:17.295147 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279975    4276 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 31 17:27:17.295147 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279977    4276 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 31 17:27:17.295147 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279980    4276 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 31 17:27:17.295147 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279983    4276 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 31 17:27:17.295147 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279985    4276 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 31 17:27:17.295147 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279988    4276 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 31 17:27:17.295147 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279991    4276 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 31 17:27:17.295147 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279993    4276 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 31 17:27:17.295147 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279996    4276 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 31 17:27:17.295147 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.279998    4276 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 31 17:27:17.295941 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.280001    4276 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 31 17:27:17.295941 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.280003    4276 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 31 17:27:17.295941 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.280006    4276 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 31 17:27:17.295941 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.280008    4276 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 31 17:27:17.295941 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.280011    4276 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 31 17:27:17.295941 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.280013    4276 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 31 17:27:17.295941 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.280016    4276 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 31 17:27:17.295941 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.280018    4276 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 31 17:27:17.295941 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.280021    4276 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 31 17:27:17.295941 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.280023    4276 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 31 17:27:17.295941 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.280026    4276 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 31 17:27:17.295941 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.280028    4276 feature_gate.go:386] unrecognized feature gate: Example2
Oct 31 17:27:17.295941 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.280031    4276 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 31 17:27:17.295941 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.280036    4276 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 31 17:27:17.295941 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.280039    4276 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 31 17:27:17.295941 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.280041    4276 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 31 17:27:17.295941 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.280044    4276 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 31 17:27:17.295941 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.280048    4276 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 31 17:27:17.295941 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.280052    4276 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 31 17:27:17.295941 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.280054    4276 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 31 17:27:17.296820 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.280057    4276 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 31 17:27:17.296820 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.280059    4276 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 31 17:27:17.296820 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.280062    4276 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 31 17:27:17.296820 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.280064    4276 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 31 17:27:17.296820 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.280067    4276 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 31 17:27:17.296820 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.280070    4276 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 31 17:27:17.296820 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.280072    4276 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 31 17:27:17.296820 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.280075    4276 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 31 17:27:17.296820 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.280078    4276 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 31 17:27:17.296820 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.280080    4276 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 31 17:27:17.296820 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.280083    4276 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 31 17:27:17.296820 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.280085    4276 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 31 17:27:17.296820 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.280088    4276 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 31 17:27:17.296820 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.280091    4276 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 31 17:27:17.296820 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.280093    4276 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 31 17:27:17.296820 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.280096    4276 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 31 17:27:17.296820 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.280098    4276 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 31 17:27:17.296820 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.280101    4276 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 31 17:27:17.296820 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.280103    4276 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 31 17:27:17.297615 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.280106    4276 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 31 17:27:17.297615 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.280108    4276 feature_gate.go:386] unrecognized feature gate: Example
Oct 31 17:27:17.297615 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.280111    4276 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 31 17:27:17.297615 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.280113    4276 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 31 17:27:17.297615 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.280116    4276 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 31 17:27:17.297615 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.280118    4276 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 31 17:27:17.297615 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.280123    4276 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 31 17:27:17.297615 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.280125    4276 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 31 17:27:17.297615 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.280130    4276 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 31 17:27:17.297615 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.280134    4276 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 31 17:27:17.297615 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.280139    4276 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 31 17:27:17.297615 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.280142    4276 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 31 17:27:17.297615 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:17.280144    4276 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 31 17:27:17.297615 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.280192    4276 server.go:957] "Client rotation is on, will bootstrap in background"
Oct 31 17:27:17.297615 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.281145    4276 bootstrap.go:85] "Current kubeconfig file contents are still valid, no bootstrap necessary"
Oct 31 17:27:17.297615 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.281191    4276 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-client-current.pem"
Oct 31 17:27:17.297615 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.281367    4276 server.go:1014] "Starting client certificate rotation"
Oct 31 17:27:17.297615 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.281484    4276 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Oct 31 17:27:17.298075 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.281551    4276 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-10-30 17:21:39 +0000 UTC" deadline="2028-08-01 02:58:53.176190396 +0000 UTC"
Oct 31 17:27:17.298075 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.281598    4276 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15345h31m35.894595905s"
Oct 31 17:27:17.298075 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.281831    4276 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 31 17:27:17.298075 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.294674    4276 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 31 17:27:17.298075 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.297370    4276 log.go:25] "Validated CRI v1 runtime API"
Oct 31 17:27:17.298934 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.298920    4276 log.go:25] "Validated CRI v1 image API"
Oct 31 17:27:17.299703 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.299685    4276 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Oct 31 17:27:17.301176 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.301147    4276 fs.go:135] Filesystem UUIDs: map[3b3f032c-14aa-4995-a2c0-3ee30a204a2e:/dev/nvme0n1p4 502e58b0-16c8-4f02-bcf2-2c4ee8f17d33:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2]
Oct 31 17:27:17.301351 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.301173    4276 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/05fd6e0cad7c3c567ee186139aeeee3ca4d8c9d1ff9304f905346df0c19494ba/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/05fd6e0cad7c3c567ee186139aeeee3ca4d8c9d1ff9304f905346df0c19494ba/userdata/shm major:0 minor:227 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/0c07b7277e444faad0e0d4df98b62cbe2593d83155d3456ed11cc10e0c1cb0a1/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/0c07b7277e444faad0e0d4df98b62cbe2593d83155d3456ed11cc10e0c1cb0a1/userdata/shm major:0 minor:64 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/1f249b0a61041d3373d202f197eb2fbec975e76055493776ceb40bec5ba9e2e9/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/1f249b0a61041d3373d202f197eb2fbec975e76055493776ceb40bec5ba9e2e9/userdata/shm major:0 minor:68 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/218b9721df0830d0c7cab0413f740b319f6f6cbd1a9564068ed16caab7b39ef8/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/218b9721df0830d0c7cab0413f740b319f6f6cbd1a9564068ed16caab7b39ef8/userdata/shm major:0 minor:229 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/2e192b44eef76d29b1099e8e7c7e3953d486d35036580c92a97be1dbd93dc046/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/2e192b44eef76d29b1099e8e7c7e3953d486d35036580c92a97be1dbd93dc046/userdata/shm major:0 minor:98 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/46ce576c40eca23ecc0802b8b8286680492c154087372d52ce8d348fd0b158d6/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/46ce576c40eca23ecc0802b8b8286680492c154087372d52ce8d348fd0b158d6/userdata/shm major:0 minor:236 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/4afec5f5a20882c39cdbd49f1982ca090f6b8dfa8e67c1fe384d378475e71d92/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/4afec5f5a20882c39cdbd49f1982ca090f6b8dfa8e67c1fe384d378475e71d92/userdata/shm major:0 minor:207 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/6902bc06a69b2dddc4f6a14a5bb71c7bb8c616b1df22f17971bd4ac0509c5d82/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/6902bc06a69b2dddc4f6a14a5bb71c7bb8c616b1df22f17971bd4ac0509c5d82/userdata/shm major:0 minor:76 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/83d5a40518f5f439cd926b8e39cca9f98e9ca3e2eec94225ef61822409f0d4f0/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/83d5a40518f5f439cd926b8e39cca9f98e9ca3e2eec94225ef61822409f0d4f0/userdata/shm major:0 minor:80 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/90cafa85c22771a4587dddc9700a96eb3a9362f1c6cb677aee45f07f3df54cfd/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/90cafa85c22771a4587dddc9700a96eb3a9362f1c6cb677aee45f07f3df54cfd/userdata/shm major:0 minor:88 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/91c4d7126773ac299bbe247165646f3983bd101b41b642c15c218947d5a5cb20/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/91c4d7126773ac299bbe247165646f3983bd101b41b642c15c218947d5a5cb20/userdata/shm major:0 minor:72 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/9c8e2f2a20f397a3b0cf89117bb25ddffba6af2b982702b73388a88b5fa317ec/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/9c8e2f2a20f397a3b0cf89117bb25ddffba6af2b982702b73388a88b5fa317ec/userdata/shm major:0 minor:60 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/ad384a3f97c70e236255e95551e8f49530e0f5a0372f0f8d575be60293cae337/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/ad384a3f97c70e236255e95551e8f49530e0f5a0372f0f8d575be60293cae337/userdata/shm major:0 minor:94 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/d7726a7ce570a8dbc898086a80c9b12b7a4bf0a9d3b410971f156c62bb1bce58/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/d7726a7ce570a8dbc898086a80c9b12b7a4bf0a9d3b410971f156c62bb1bce58/userdata/shm major:0 minor:84 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/fcf1add41537fb1ea326adfb301b383a19868c74aedb40d86cf92ad0e07ae83c/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/fcf1add41537fb1ea326adfb301b383a19868c74aedb40d86cf92ad0e07ae83c/userdata/shm major:0 minor:203 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/03a15842-25a2-4fbe-b418-8867070c4482/volumes/kubernetes.io~projected/kube-api-access-lc2r4:{mountpoint:/var/lib/kubelet/pods/03a15842-25a2-4fbe-b418-8867070c4482/volumes/kubernetes.io~projected/kube-api-access-lc2r4 major:0 minor:53 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0f353547-2feb-42f4-aae4-b0f1a30efa34/volumes/kubernetes.io~projected/kube-api-access-qfvft:{mountpoint:/var/lib/kubelet/pods/0f353547-2feb-42f4-aae4-b0f1a30efa34/volumes/kubernetes.io~projected/kube-api-access-qfvft major:0 minor:59 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/138bfcc8-f199-4ffd-b5d2-8d83ade6199a/volumes/kubernetes.io~projected/kube-api-access-d6ck5:{mountpoint:/var/lib/kubelet/pods/138bfcc8-f199-4ffd-b5d2-8d83ade6199a/volumes/kubernetes.io~projected/kube-api-access-d6ck5 major:0 minor:57 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/22096c98-1b93-4ea4-9a6f-6383df9eafe4/volumes/kubernetes.io~projected/kube-api-access-49ggd:{mountpoint:/var/lib/kubelet/pods/22096c98-1b93-4ea4-9a6f-6383df9eafe4/volumes/kubernetes.io~projected/kube-api-access-49ggd major:0 minor:197 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/22096c98-1b93-4ea4-9a6f-6383df9eafe4/volumes/kubernetes.io~secret/cert:{mountpoint:/var/lib/kubelet/pods/22096c98-1b93-4ea4-9a6f-6383df9eafe4/volumes/kubernetes.io~secret/cert major:0 minor:45 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/2f646206-1186-4fcf-a9ca-fe92371d4d2b/volumes/kubernetes.io~secret/agent-certs:{mountpoint:/var/lib/kubelet/pods/2f646206-1186-4fcf-a9ca-fe92371d4d2b/volumes/kubernetes.io~secret/agent-certs major:0 minor:49 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/408bcc05-9690-4632-a479-de41718847ad/volumes/kubernetes.io~projected/kube-api-access-5dkwn:{mountpoint:/var/lib/kubelet/pods/408bcc05-9690-4632-a479-de41718847ad/volumes/kubernetes.io~projected/kube-api-access-5dkwn major:0 minor:206 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/408bcc05-9690-4632-a479-de41718847ad/volumes/kubernetes.io~secret/signing-key:{mountpoint:/var/lib/kubelet/pods/408bcc05-9690-4632-a479-de41718847ad/volumes/kubernetes.io~secret/signing-key major:0 minor:205 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/422b9ca9-3e0e-4fe0-860d-7c7d24b179fa/volumes/kubernetes.io~empty-dir/etc-tuned:{mountpoint:/var/lib/kubelet/pods/422b9ca9-3e0e-4fe0-860d-7c7d24b179fa/volumes/kubernetes.io~empty-dir/etc-tuned major:0 minor:47 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/422b9ca9-3e0e-4fe0-860d-7c7d24b179fa/volumes/kubernetes.io~empty-dir/tmp:{mountpoint:/var/lib/kubelet/pods/422b9ca9-3e0e-4fe0-860d-7c7d24b179fa/volumes/kubernetes.io~empty-dir/tmp major:0 minor:48 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/422b9ca9-3e0e-4fe0-860d-7c7d24b179fa/volumes/kubernetes.io~projected/kube-api-access-59c8z:{mountpoint:/var/lib/kubelet/pods/422b9ca9-3e0e-4fe0-860d-7c7d24b179fa/volumes/kubernetes.io~projected/kube-api-access-59c8z major:0 minor:56 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e/volumes/kubernetes.io~projected/kube-api-access-28ft5:{mountpoint:/var/lib/kubelet/pods/47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e/volumes/kubernetes.io~projected/kube-api-access-28ft5 major:0 minor:55 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/5819fd64-9ac0-4436-b8fe-dcd3a27827ac/volume-subpaths/run-systemd/ovnkube-controller/6:{mountpoint:/var/lib/kubelet/pods/5819fd64-9ac0-4436-b8fe-dcd3a27827ac/volume-subpaths/run-systemd/ovnkube-controller/6 major:0 minor:26 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/5819fd64-9ac0-4436-b8fe-dcd3a27827ac/volumes/kubernetes.io~projected/kube-api-access-g2n2p:{mountpoint:/var/lib/kubelet/pods/5819fd64-9ac0-4436-b8fe-dcd3a27827ac/volumes/kubernetes.io~projected/kube-api-access-g2n2p major:0 minor:51 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/5819fd64-9ac0-4436-b8fe-dcd3a27827ac/volumes/kubernetes.io~secret/ovn-node-metrics-cert:{mountpoint:/var/lib/kubelet/pods/5819fd64-9ac0-4436-b8fe-dcd3a27827ac/volumes/kubernetes.io~secret/ovn-node-metrics-cert major:0 minor:50 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/61462533-417a-4fbc-bd85-c56a0112db31/volumes/kubernetes.io~projected/kube-api-access-p2gtl:{mountpoint:/var/lib/kubelet/pods/61462533-417a-4fbc-bd85-c56a0112db31/volumes/kubernetes.io~projected/kube-api-access-p2gtl major:0 minor:202 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/6d7781d6-8f73-4f1d-bb9a-0212e2eb6e4b/volumes/kubernetes.io~secret/original-pull-secret:{mountpoint:/var/lib/kubelet/pods/6d7781d6-8f73-4f1d-bb9a-0212e2eb6e4b/volumes/kubernetes.io~secret/original-pull-secret major:0 minor:217 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/72516605-a894-4a62-84c7-834122a01fb8/volumes/kubernetes.io~projected/kube-api-access-xxrgs:{mountpoint:/var/lib/kubelet/pods/72516605-a894-4a62-84c7-834122a01fb8/volumes/kubernetes.io~projected/kube-api-access-xxrgs major:0 minor:52 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/87e61dcd-75f6-4d3f-8ed9-7d35b06a9bb3/volumes/kubernetes.io~projected/kube-api-access-vv9nf:{mountpoint:/var/lib/kubelet/pods/87e61dcd-75f6-4d3f-8ed9-7d35b06a9bb3/volumes/kubernetes.io~projected/kube-api-access-vv9nf major:0 minor:54 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/9ee82620-6497-48e8-bb2c-1bb77a93f7ef/volumes/kubernetes.io~projected/kube-api-access-z8zhc:{mountpoint:/var/lib/kubelet/pods/9ee82620-6497-48e8-bb2c-1bb77a93f7ef/volumes/kubernetes.io~projected/kube-api-access-z8zhc major:0 minor:201 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/9ee82620-6497-48e8-bb2c-1bb77a93f7ef/volumes/kubernetes.io~secret/metrics-tls:{mountpoint:/var/lib/kubelet/pods/9ee82620-6497-48e8-bb2c-1bb77a93f7ef/volumes/kubernetes.io~secret/metrics-tls major:0 minor:235 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/b8bbaf33-7cd9-4a30-86f7-6e20ac45b0a0/volumes/kubernetes.io~projected/kube-api-access-grrfv:{mountpoint:/var/lib/kubelet/pods/b8bbaf33-7cd9-4a30-86f7-6e20ac45b0a0/volumes/kubernetes.io~projected/kube-api-access-grrfv major:0 minor:58 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/fb9c203a-af67-4df4-88dc-74cf9eab3623/volumes/kubernetes.io~projected/kube-api-access-c6k68:{mountpoint:/var/lib/kubelet/pods/fb9c203a-af67-4df4-88dc-74cf9eab3623/volumes/kubernetes.io~projected/kube-api-access-c6k68 major:0 minor:218 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0} overlay_0-100:{mountpoint:/var/lib/containers/storage/overlay/6f20897cc907bcdd4b5369c959b13fb5e61f2903d6a9c4eeaffe79515f39f031/merged major:0 minor:100 fsType:overlay blockSize:0} overlay_0-102:{mountpoint:/var/lib/containers/storage/overlay/7ff002c432f230546fed07bf96cff5b097d6b57ddf8fffc9098c79d23738db29/merged major:0 minor:102 fsType:overlay blockSize:0} overlay_0-104:{mountpoint:/var/lib/containers/storage/overlay/faee44af04c6cbf3fee058d770bf53d25fea113f00985da820cb74c6cedc7f8d/merged major:0 minor:104 fsType:overlay blockSize:0} overlay_0-110:{mountpoint:/var/lib/containers/storage/overlay/f448e1b1da36a00adf15d371b2252918e0bbeedebd82b389884dcb1f3e47d529/merged major:0 minor:110 fsType:overlay blockSize:0} overlay_0-113:{mountpoint:/var/lib/containers/storage/overlay/b678c37ba4657f168e398d5498844011cb2844fa41fe8f4a39c2ae373e6cd495/merged major:0 minor:113 fsType:overlay blockSize:0} overlay_0-119:{mountpoint:/var/lib/containers/storage/overlay/031353f32ddc4b4dde657b010f3226aa88f7449e174cbe1280a3056e422b8c57/merged major:0 minor:119 fsType:overlay blockSize:0} overlay_0-124:{mountpoint:/var/lib/containers/storage/overlay/c464dff065d8d945dc0d0875ef1568c1a33c25d648d158ae40b28f4d8a0a67ad/merged major:0 minor:124 fsType:overlay blockSize:0} overlay_0-127:{mountpoint:/var/lib/containers/storage/overlay/d03b0f31dcf08df448b6e460fc91272d5061a1874bf5a1021926967ae50230b2/merged major:0 minor:127 fsType:overlay blockSize:0} overlay_0-132:{mountpoint:/var/lib/containers/storage/overlay/24b7f5558c4b15c718da5686e09f2f361bbc462b9c75ac0480da4551516d6659/merged major:0 minor:132 fsType:overlay blockSize:0} overlay_0-133:{mountpoint:/var/lib/containers/storage/overlay/29b4ce05316f849d3f64c47e4a468612d85c419f0b44c2794291067c087cd1f2/merged major:0 minor:133 fsType:overlay blockSize:0} overlay_0-144:{mountpoint:/var/lib/containers/storage/overlay/1e105644b2e443de6895f4af927dbda2fb2ef3089182ea39b8ea701908de5ad1/merged major:0 minor:144 fsType:overlay blockSize:0} overlay_0-149:{mountpoint:/var/lib/containers/storage/overlay/56bdd20cefe0982d724b703752123fbf824ac3b6683c9ebc2a77d5c9296bccc1/merged major:0 minor:149 fsType:overlay blockSize:0} overlay_0-152:{mountpoint:/var/lib/containers/storage/overlay/d057044feb2a738804d7e8003abbd0eddb42d9bdcdb3f564145294a2c34d57c2/merged major:0 minor:152 fsType:overlay blockSize:0} overlay_0-157:{mountpoint:/var/lib/containers/storage/overlay/3713019928b36554fa14b151e4ed9187558d7052c8fb7098169c54759d33bf6d/merged major:0 minor:157 fsType:overlay blockSize:0} overlay_0-162:{mountpoint:/var/lib/containers/storage/overlay/568b9d39aa96920d7d0f50d4ac93cfd07b407c9b65959ad76d1ed9049de2ad3c/merged major:0 minor:162 fsType:overlay blockSize:0} overlay_0-167:{mountpoint:/var/lib/containers/storage/overlay/a774727a3a0af8d04d41a2d0cc6d18335534ef013baa9dd3bb15cd4c052a7214/merged major:0 minor:167 fsType:overlay blockSize:0} overlay_0-172:{mountpoint:/var/lib/containers/storage/overlay/9f1897f3c75d7bec31bf7c6faf7b34e9135f732585862689188b5939662620ac/merged major:0 minor:172 fsType:overlay blockSize:0} overlay_0-177:{mountpoint:/var/lib/containers/storage/overlay/5fde24e1636a76950b39371d2ba70b535004c4f5523e47b902711c840c3fb3bf/merged major:0 minor:177 fsType:overlay blockSize:0} overlay_0-182:{mountpoint:/var/lib/containers/storage/overlay/8838ec550bc02f5360be16753adbc3d0e60d6a9b3e1e29586fc28aae7e5e2ead/merged major:0 minor:182 fsType:overlay blockSize:0} overlay_0-187:{mountpoint:/var/lib/containers/storage/overlay/1f011f63374d78f8cfaa4b0a99510349b6cfaef4ecec87fd2031775586623ba2/merged major:0 minor:187 fsType:overlay blockSize:0} overlay_0-192:{mountpoint:/var/lib/containers/storage/overlay/65e5cbf5420fb9ad72998dc15f6b95b4a626654acc4404b2e8a74f4a6cd7e638/merged major:0 minor:192 fsType:overlay blockSize:0} overlay_0-209:{mountpoint:/var/lib/containers/storage/overlay/7645122059ff370500992cd5584ba0138f1c1a406a5e129dc79961ae78aa631d/merged major:0 minor:209 fsType:overlay blockSize:0} overlay_0-211:{mountpoint:/var/lib/containers/storage/overlay/7a31a8f995a31513c7582c3a4bff913f9d5aebf789e20cada5e8258f58e9da37/merged major:0 minor:211 fsType:overlay blockSize:0} overlay_0-213:{mountpoint:/var/lib/containers/storage/overlay/6ea40b0f8c48523a37cda3c71651c223880e2b10e88318bdf68fec96d498d744/merged major:0 minor:213 fsType:overlay blockSize:0} overlay_0-215:{mountpoint:/var/lib/containers/storage/overlay/723198c5815a28271074448120046798fff597080404afd3cadb2e7ec2d9e79e/merged major:0 minor:215 fsType:overlay blockSize:0} overlay_0-231:{mountpoint:/var/lib/containers/storage/overlay/c38d8b4243aa5e78cf13b8dbd7e7df24a438449185909bca56c4014c8dba3101/merged major:0 minor:231 fsType:overlay blockSize:0} overlay_0-233:{mountpoint:/var/lib/containers/storage/overlay/e8847b67a93c48317314afb4b45def4eba175ee84aad412467e0fdd27bf983e6/merged major:0 minor:233 fsType:overlay blockSize:0} overlay_0-238:{mountpoint:/var/lib/containers/storage/overlay/1d3078b560404912ea31d3b497b68b03ce9b8896d3be30180f6c0319c524d636/merged major:0 minor:238 fsType:overlay blockSize:0} overlay_0-240:{mountpoint:/var/lib/containers/storage/overlay/6de92d1b879f67755ff3d55dd478554327cf39a7a6809c71a9c107d4ee0eef4c/merged major:0 minor:240 fsType:overlay blockSize:0} overlay_0-246:{mountpoint:/var/lib/containers/storage/overlay/c82ce79f5dc8f8b674a2f044694b29a80eb1b2a1bbbeef77716dc47237e858f1/merged major:0 minor:246 fsType:overlay blockSize:0} overlay_0-62:{mountpoint:/var/lib/containers/storage/overlay/e8817d54fbc3980bab16879469106ee1b0eeb624e1c2161e3148c5371d7aa893/merged major:0 minor:62 fsType:overlay blockSize:0} overlay_0-66:{mountpoint:/var/lib/containers/storage/overlay/33cd2feb227edeb2f809ae32225596151613bf7ec1d49e9b3acfbb9841943011/merged major:0 minor:66 fsType:overlay blockSize:0} overlay_0-70:{mountpoint:/var/lib/containers/storage/overlay/01c96a2d358df30802511032c3a3845cc74e04614dd567bb25f5cc6025c8c7ea/merged major:0 minor:70 fsType:overlay blockSize:0} overlay_0-74:{mountpoint:/var/lib/containers/storage/overlay/1cbb443df5190dd6ea51e37b34e61cc14a7d9e45d4c21a64075ede0e7b9b0449/merged major:0 minor:74 fsType:overlay blockSize:0} overlay_0-78:{mountpoint:/var/lib/containers/storage/overlay/49bc33e4edf67a89ce5cbb37b0bf7b43efcbcdaffb8cfd49067768785a66bbfd/merged major:0 minor:78 fsType:overlay blockSize:0} overlay_0-82:{mountpoint:/var/lib/containers/storage/overlay/fd3a938f6de6273edac05cd5f315e0b65da10f231d41a0162fa0656b3a814289/merged major:0 minor:82 fsType:overlay blockSize:0} overlay_0-86:{mountpoint:/var/lib/containers/storage/overlay/81ce3491e6c7793fa276d45c9a3551f8da8fae4e3a0ae9704c4496d8e1332b9b/merged major:0 minor:86 fsType:overlay blockSize:0} overlay_0-90:{mountpoint:/var/lib/containers/storage/overlay/331e7b4e817c4bb34f5ce1342d42199481bb277fb27104ed47e9dfffd3b37b18/merged major:0 minor:90 fsType:overlay blockSize:0} overlay_0-92:{mountpoint:/var/lib/containers/storage/overlay/731702cb9505be20f4e56471dd0d6ba722f1584b9aa1f90c39d5f698f714562c/merged major:0 minor:92 fsType:overlay blockSize:0} overlay_0-96:{mountpoint:/var/lib/containers/storage/overlay/cf44b5b8b0e9a6904740a982cff79a5dacdcb3935c4e115d9b972a7b367409af/merged major:0 minor:96 fsType:overlay blockSize:0}]
Oct 31 17:27:17.307775 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.307533    4276 manager.go:217] Machine: {Timestamp:2026-10-31 17:27:17.307061591 +0000 UTC m=+0.104806509 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3098145 MemoryCapacity:33164333056 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec22644a1a04d4d435bd958c94ea5c24 SystemUUID:ec22644a-1a04-d4d4-35bd-958c94ea5c24 BootID:c6e4931c-99a2-49b9-abc4-7c8389a6675d Filesystems:[{Device:overlay_0-127 DeviceMajor:0 DeviceMinor:127 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-172 DeviceMajor:0 DeviceMinor:172 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/5819fd64-9ac0-4436-b8fe-dcd3a27827ac/volumes/kubernetes.io~secret/ovn-node-metrics-cert DeviceMajor:0 DeviceMinor:50 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/22096c98-1b93-4ea4-9a6f-6383df9eafe4/volumes/kubernetes.io~projected/kube-api-access-49ggd DeviceMajor:0 DeviceMinor:197 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/408bcc05-9690-4632-a479-de41718847ad/volumes/kubernetes.io~projected/kube-api-access-5dkwn DeviceMajor:0 DeviceMinor:206 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/218b9721df0830d0c7cab0413f740b319f6f6cbd1a9564068ed16caab7b39ef8/userdata/shm DeviceMajor:0 DeviceMinor:229 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/72516605-a894-4a62-84c7-834122a01fb8/volumes/kubernetes.io~projected/kube-api-access-xxrgs DeviceMajor:0 DeviceMinor:52 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-162 DeviceMajor:0 DeviceMinor:162 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/61462533-417a-4fbc-bd85-c56a0112db31/volumes/kubernetes.io~projected/kube-api-access-p2gtl DeviceMajor:0 DeviceMinor:202 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/0f353547-2feb-42f4-aae4-b0f1a30efa34/volumes/kubernetes.io~projected/kube-api-access-qfvft DeviceMajor:0 DeviceMinor:59 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/9c8e2f2a20f397a3b0cf89117bb25ddffba6af2b982702b73388a88b5fa317ec/userdata/shm DeviceMajor:0 DeviceMinor:60 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-62 DeviceMajor:0 DeviceMinor:62 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/5819fd64-9ac0-4436-b8fe-dcd3a27827ac/volume-subpaths/run-systemd/ovnkube-controller/6 DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:/var/lib/kubelet/pods/408bcc05-9690-4632-a479-de41718847ad/volumes/kubernetes.io~secret/signing-key DeviceMajor:0 DeviceMinor:205 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/fb9c203a-af67-4df4-88dc-74cf9eab3623/volumes/kubernetes.io~projected/kube-api-access-c6k68 DeviceMajor:0 DeviceMinor:218 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/05fd6e0cad7c3c567ee186139aeeee3ca4d8c9d1ff9304f905346df0c19494ba/userdata/shm DeviceMajor:0 DeviceMinor:227 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-240 DeviceMajor:0 DeviceMinor:240 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16582164480 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/422b9ca9-3e0e-4fe0-860d-7c7d24b179fa/volumes/kubernetes.io~empty-dir/tmp DeviceMajor:0 DeviceMinor:48 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/b8bbaf33-7cd9-4a30-86f7-6e20ac45b0a0/volumes/kubernetes.io~projected/kube-api-access-grrfv DeviceMajor:0 DeviceMinor:58 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-70 DeviceMajor:0 DeviceMinor:70 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-74 DeviceMajor:0 DeviceMinor:74 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-149 DeviceMajor:0 DeviceMinor:149 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/9ee82620-6497-48e8-bb2c-1bb77a93f7ef/volumes/kubernetes.io~projected/kube-api-access-z8zhc DeviceMajor:0 DeviceMinor:201 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-157 DeviceMajor:0 DeviceMinor:157 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-167 DeviceMajor:0 DeviceMinor:167 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/422b9ca9-3e0e-4fe0-860d-7c7d24b179fa/volumes/kubernetes.io~empty-dir/etc-tuned DeviceMajor:0 DeviceMinor:47 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-152 DeviceMajor:0 DeviceMinor:152 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/2f646206-1186-4fcf-a9ca-fe92371d4d2b/volumes/kubernetes.io~secret/agent-certs DeviceMajor:0 DeviceMinor:49 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e/volumes/kubernetes.io~projected/kube-api-access-28ft5 DeviceMajor:0 DeviceMinor:55 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/422b9ca9-3e0e-4fe0-860d-7c7d24b179fa/volumes/kubernetes.io~projected/kube-api-access-59c8z DeviceMajor:0 DeviceMinor:56 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/0c07b7277e444faad0e0d4df98b62cbe2593d83155d3456ed11cc10e0c1cb0a1/userdata/shm DeviceMajor:0 DeviceMinor:64 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/1f249b0a61041d3373d202f197eb2fbec975e76055493776ceb40bec5ba9e2e9/userdata/shm DeviceMajor:0 DeviceMinor:68 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/ad384a3f97c70e236255e95551e8f49530e0f5a0372f0f8d575be60293cae337/userdata/shm DeviceMajor:0 DeviceMinor:94 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-133 DeviceMajor:0 DeviceMinor:133 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/var/lib/kubelet/pods/03a15842-25a2-4fbe-b418-8867070c4482/volumes/kubernetes.io~projected/kube-api-access-lc2r4 DeviceMajor:0 DeviceMinor:53 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/91c4d7126773ac299bbe247165646f3983bd101b41b642c15c218947d5a5cb20/userdata/shm DeviceMajor:0 DeviceMinor:72 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-177 DeviceMajor:0 DeviceMinor:177 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-182 DeviceMajor:0 DeviceMinor:182 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-187 DeviceMajor:0 DeviceMinor:187 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/fcf1add41537fb1ea326adfb301b383a19868c74aedb40d86cf92ad0e07ae83c/userdata/shm DeviceMajor:0 DeviceMinor:203 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-211 DeviceMajor:0 DeviceMinor:211 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/5819fd64-9ac0-4436-b8fe-dcd3a27827ac/volumes/kubernetes.io~projected/kube-api-access-g2n2p DeviceMajor:0 DeviceMinor:51 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-82 DeviceMajor:0 DeviceMinor:82 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/2e192b44eef76d29b1099e8e7c7e3953d486d35036580c92a97be1dbd93dc046/userdata/shm DeviceMajor:0 DeviceMinor:98 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-100 DeviceMajor:0 DeviceMinor:100 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-102 DeviceMajor:0 DeviceMinor:102 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-124 DeviceMajor:0 DeviceMinor:124 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-233 DeviceMajor:0 DeviceMinor:233 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-144 DeviceMajor:0 DeviceMinor:144 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/46ce576c40eca23ecc0802b8b8286680492c154087372d52ce8d348fd0b158d6/userdata/shm DeviceMajor:0 DeviceMinor:236 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16582168576 Type:vfs Inodes:1048576 HasInodes:true} {Device:/var/lib/kubelet/pods/138bfcc8-f199-4ffd-b5d2-8d83ade6199a/volumes/kubernetes.io~projected/kube-api-access-d6ck5 DeviceMajor:0 DeviceMinor:57 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/d7726a7ce570a8dbc898086a80c9b12b7a4bf0a9d3b410971f156c62bb1bce58/userdata/shm DeviceMajor:0 DeviceMinor:84 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-113 DeviceMajor:0 DeviceMinor:113 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-192 DeviceMajor:0 DeviceMinor:192 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-209 DeviceMajor:0 DeviceMinor:209 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-231 DeviceMajor:0 DeviceMinor:231 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/6d7781d6-8f73-4f1d-bb9a-0212e2eb6e4b/volumes/kubernetes.io~secret/original-pull-secret DeviceMajor:0 DeviceMinor:217 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/9ee82620-6497-48e8-bb2c-1bb77a93f7ef/volumes/kubernetes.io~secret/metrics-tls DeviceMajor:0 DeviceMinor:235 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-246 DeviceMajor:0 DeviceMinor:246 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:overlay_0-66 DeviceMajor:0 DeviceMinor:66 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/6902bc06a69b2dddc4f6a14a5bb71c7bb8c616b1df22f17971bd4ac0509c5d82/userdata/shm DeviceMajor:0 DeviceMinor:76 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-86 DeviceMajor:0 DeviceMinor:86 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/90cafa85c22771a4587dddc9700a96eb3a9362f1c6cb677aee45f07f3df54cfd/userdata/shm DeviceMajor:0 DeviceMinor:88 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-96 DeviceMajor:0 DeviceMinor:96 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-110 DeviceMajor:0 DeviceMinor:110 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-119 DeviceMajor:0 DeviceMinor:119 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-132 DeviceMajor:0 DeviceMinor:132 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/87e61dcd-75f6-4d3f-8ed9-7d35b06a9bb3/volumes/kubernetes.io~projected/kube-api-access-vv9nf DeviceMajor:0 DeviceMinor:54 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/83d5a40518f5f439cd926b8e39cca9f98e9ca3e2eec94225ef61822409f0d4f0/userdata/shm DeviceMajor:0 DeviceMinor:80 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/22096c98-1b93-4ea4-9a6f-6383df9eafe4/volumes/kubernetes.io~secret/cert DeviceMajor:0 DeviceMinor:45 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/4afec5f5a20882c39cdbd49f1982ca090f6b8dfa8e67c1fe384d378475e71d92/userdata/shm DeviceMajor:0 DeviceMinor:207 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-238 DeviceMajor:0 DeviceMinor:238 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-78 DeviceMajor:0 DeviceMinor:78 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-90 DeviceMajor:0 DeviceMinor:90 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-92 DeviceMajor:0 DeviceMinor:92 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-104 DeviceMajor:0 DeviceMinor:104 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-213 DeviceMajor:0 DeviceMinor:213 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-215 DeviceMajor:0 DeviceMinor:215 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:05fd6e0cad7c3c5 MacAddress:f6:df:9f:23:a0:bb Speed:10000 Mtu:8901} {Name:218b9721df0830d MacAddress:fe:5f:8c:00:91:10 Speed:10000 Mtu:8901} {Name:46ce576c40eca23 MacAddress:b6:83:db:fa:ec:a9 Speed:10000 Mtu:8901} {Name:4afec5f5a20882c MacAddress:aa:2c:8f:0a:9b:7d Speed:10000 Mtu:8901} {Name:br-ex MacAddress:02:ff:ed:5c:21:57 Speed:0 Mtu:9001} {Name:br-int MacAddress:8a:9c:01:34:61:57 Speed:0 Mtu:8901} {Name:ens5 MacAddress:02:ff:ed:5c:21:57 Speed:0 Mtu:9001} {Name:fcf1add41537fb1 MacAddress:52:eb:4c:1d:1e:56 Speed:10000 Mtu:8901} {Name:genev_sys_6081 MacAddress:c6:06:be:85:95:2d Speed:0 Mtu:65000} {Name:ovn-k8s-mp0 MacAddress:0a:58:0a:85:00:02 Speed:0 Mtu:8901} {Name:ovs-system MacAddress:be:bf:2f:5b:05:54 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164333056 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Oct 31 17:27:17.307775 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.307775    4276 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Oct 31 17:27:17.308021 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.307869    4276 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Oct 31 17:27:17.308169 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.308150    4276 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Oct 31 17:27:17.308283 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.308169    4276 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-129-127.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Oct 31 17:27:17.308324 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.308285    4276 topology_manager.go:143] "Creating topology manager with none policy"
Oct 31 17:27:17.308324 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.308293    4276 container_manager_linux.go:311] "Creating device plugin manager"
Oct 31 17:27:17.308324 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.308306    4276 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 31 17:27:17.308324 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.308324    4276 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 31 17:27:17.308433 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.308391    4276 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager"
Oct 31 17:27:17.308582 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.308571    4276 state_mem.go:41] "Initialized" logger="CPUManager state memory"
Oct 31 17:27:17.308713 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.308703    4276 server.go:1235] "Using root directory" path="/var/lib/kubelet"
Oct 31 17:27:17.308749 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.308737    4276 kubelet.go:493] "Attempting to sync node with API server"
Oct 31 17:27:17.308749 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.308748    4276 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests"
Oct 31 17:27:17.308835 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.308761    4276 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Oct 31 17:27:17.308835 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.308771    4276 kubelet.go:405] "Adding apiserver pod source"
Oct 31 17:27:17.308835 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.308778    4276 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Oct 31 17:27:17.309501 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.309484    4276 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint"
Oct 31 17:27:17.309501 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.309500    4276 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 31 17:27:17.309974 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.309957    4276 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1"
Oct 31 17:27:17.317867 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.317846    4276 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint"
Oct 31 17:27:17.317867 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.317867    4276 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 31 17:27:17.317985 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.317975    4276 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-server-current.pem"
Oct 31 17:27:17.318184 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.318174    4276 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Oct 31 17:27:17.318216 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.318203    4276 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Oct 31 17:27:17.318295 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.318286    4276 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Oct 31 17:27:17.318324 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.318304    4276 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Oct 31 17:27:17.318324 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.318314    4276 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Oct 31 17:27:17.318324 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.318323    4276 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Oct 31 17:27:17.318408 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.318331    4276 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Oct 31 17:27:17.318408 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.318340    4276 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Oct 31 17:27:17.318408 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.318346    4276 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Oct 31 17:27:17.318408 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.318352    4276 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Oct 31 17:27:17.318408 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.318359    4276 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Oct 31 17:27:17.318408 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.318364    4276 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Oct 31 17:27:17.318408 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.318373    4276 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Oct 31 17:27:17.318408 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.318382    4276 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Oct 31 17:27:17.318408 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.318409    4276 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Oct 31 17:27:17.318678 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.318418    4276 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Oct 31 17:27:17.321678 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.321664    4276 server.go:1263] "Started kubelet"
Oct 31 17:27:17.321761 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.321731    4276 server.go:182] "Starting to listen" address="0.0.0.0" port=10250
Oct 31 17:27:17.321813 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.321747    4276 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Oct 31 17:27:17.321860 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.321813    4276 server_v1.go:49] "podresources" method="list" useActivePods=true
Oct 31 17:27:17.322280 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.322263    4276 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Oct 31 17:27:17.322275 ip-10-0-129-127 systemd[1]: Started Kubernetes Kubelet.
Oct 31 17:27:17.323040 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.323022    4276 server.go:317] "Adding debug handlers to kubelet server"
Oct 31 17:27:17.324368 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.324353    4276 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer"
Oct 31 17:27:17.324621 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.324544    4276 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Oct 31 17:27:17.324699 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.324670    4276 volume_manager.go:309] "The desired_state_of_world populator starts"
Oct 31 17:27:17.324699 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.324684    4276 volume_manager.go:311] "Starting Kubelet Volume Manager"
Oct 31 17:27:17.324805 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.324678    4276 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-30 17:21:40 +0000 UTC" deadline="2028-05-30 18:34:07.090184801 +0000 UTC"
Oct 31 17:27:17.324805 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.324710    4276 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13849h6m49.765478208s"
Oct 31 17:27:17.324899 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.324805    4276 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Oct 31 17:27:17.325726 ip-10-0-129-127 kubenswrapper[4276]: E1031 17:27:17.325590    4276 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-129-127.ec2.internal\" not found"
Oct 31 17:27:17.326802 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.326756    4276 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="5819fd64-9ac0-4436-b8fe-dcd3a27827ac" volumeName="kubernetes.io/configmap/5819fd64-9ac0-4436-b8fe-dcd3a27827ac-ovnkube-config" seLinuxMountContext=""
Oct 31 17:27:17.326928 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.326803    4276 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="03a15842-25a2-4fbe-b418-8867070c4482" volumeName="kubernetes.io/projected/03a15842-25a2-4fbe-b418-8867070c4482-kube-api-access-lc2r4" seLinuxMountContext=""
Oct 31 17:27:17.326928 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.326815    4276 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2f646206-1186-4fcf-a9ca-fe92371d4d2b" volumeName="kubernetes.io/secret/2f646206-1186-4fcf-a9ca-fe92371d4d2b-agent-certs" seLinuxMountContext=""
Oct 31 17:27:17.326928 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.326826    4276 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="61462533-417a-4fbc-bd85-c56a0112db31" volumeName="kubernetes.io/configmap/61462533-417a-4fbc-bd85-c56a0112db31-kube-rbac-proxy-cm" seLinuxMountContext=""
Oct 31 17:27:17.326928 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.326837    4276 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="87e61dcd-75f6-4d3f-8ed9-7d35b06a9bb3" volumeName="kubernetes.io/configmap/87e61dcd-75f6-4d3f-8ed9-7d35b06a9bb3-serviceca" seLinuxMountContext=""
Oct 31 17:27:17.326928 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.326847    4276 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b8bbaf33-7cd9-4a30-86f7-6e20ac45b0a0" volumeName="kubernetes.io/empty-dir/b8bbaf33-7cd9-4a30-86f7-6e20ac45b0a0-tmp-dir" seLinuxMountContext=""
Oct 31 17:27:17.326928 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.326859    4276 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="408bcc05-9690-4632-a479-de41718847ad" volumeName="kubernetes.io/secret/408bcc05-9690-4632-a479-de41718847ad-signing-key" seLinuxMountContext=""
Oct 31 17:27:17.326928 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.326869    4276 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e" volumeName="kubernetes.io/projected/47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e-kube-api-access-28ft5" seLinuxMountContext=""
Oct 31 17:27:17.326928 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.326889    4276 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="5819fd64-9ac0-4436-b8fe-dcd3a27827ac" volumeName="kubernetes.io/configmap/5819fd64-9ac0-4436-b8fe-dcd3a27827ac-ovnkube-script-lib" seLinuxMountContext=""
Oct 31 17:27:17.326928 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.326900    4276 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9ee82620-6497-48e8-bb2c-1bb77a93f7ef" volumeName="kubernetes.io/projected/9ee82620-6497-48e8-bb2c-1bb77a93f7ef-kube-api-access-z8zhc" seLinuxMountContext=""
Oct 31 17:27:17.326928 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.326910    4276 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="138bfcc8-f199-4ffd-b5d2-8d83ade6199a" volumeName="kubernetes.io/projected/138bfcc8-f199-4ffd-b5d2-8d83ade6199a-kube-api-access-d6ck5" seLinuxMountContext=""
Oct 31 17:27:17.326928 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.326921    4276 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="22096c98-1b93-4ea4-9a6f-6383df9eafe4" volumeName="kubernetes.io/projected/22096c98-1b93-4ea4-9a6f-6383df9eafe4-kube-api-access-49ggd" seLinuxMountContext=""
Oct 31 17:27:17.327503 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.326933    4276 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2f646206-1186-4fcf-a9ca-fe92371d4d2b" volumeName="kubernetes.io/configmap/2f646206-1186-4fcf-a9ca-fe92371d4d2b-konnectivity-ca" seLinuxMountContext=""
Oct 31 17:27:17.327503 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.326944    4276 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="422b9ca9-3e0e-4fe0-860d-7c7d24b179fa" volumeName="kubernetes.io/projected/422b9ca9-3e0e-4fe0-860d-7c7d24b179fa-kube-api-access-59c8z" seLinuxMountContext=""
Oct 31 17:27:17.327503 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.326959    4276 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="5819fd64-9ac0-4436-b8fe-dcd3a27827ac" volumeName="kubernetes.io/projected/5819fd64-9ac0-4436-b8fe-dcd3a27827ac-kube-api-access-g2n2p" seLinuxMountContext=""
Oct 31 17:27:17.327503 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.326971    4276 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="72516605-a894-4a62-84c7-834122a01fb8" volumeName="kubernetes.io/configmap/72516605-a894-4a62-84c7-834122a01fb8-whereabouts-flatfile-configmap" seLinuxMountContext=""
Oct 31 17:27:17.327503 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.326992    4276 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="87e61dcd-75f6-4d3f-8ed9-7d35b06a9bb3" volumeName="kubernetes.io/projected/87e61dcd-75f6-4d3f-8ed9-7d35b06a9bb3-kube-api-access-vv9nf" seLinuxMountContext=""
Oct 31 17:27:17.327503 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.327004    4276 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fb9c203a-af67-4df4-88dc-74cf9eab3623" volumeName="kubernetes.io/projected/fb9c203a-af67-4df4-88dc-74cf9eab3623-kube-api-access-c6k68" seLinuxMountContext=""
Oct 31 17:27:17.327503 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.327016    4276 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="61462533-417a-4fbc-bd85-c56a0112db31" volumeName="kubernetes.io/projected/61462533-417a-4fbc-bd85-c56a0112db31-kube-api-access-p2gtl" seLinuxMountContext=""
Oct 31 17:27:17.327503 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.327027    4276 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="72516605-a894-4a62-84c7-834122a01fb8" volumeName="kubernetes.io/projected/72516605-a894-4a62-84c7-834122a01fb8-kube-api-access-xxrgs" seLinuxMountContext=""
Oct 31 17:27:17.327503 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.327038    4276 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9ee82620-6497-48e8-bb2c-1bb77a93f7ef" volumeName="kubernetes.io/secret/9ee82620-6497-48e8-bb2c-1bb77a93f7ef-metrics-tls" seLinuxMountContext=""
Oct 31 17:27:17.327503 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.327050    4276 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="22096c98-1b93-4ea4-9a6f-6383df9eafe4" volumeName="kubernetes.io/secret/22096c98-1b93-4ea4-9a6f-6383df9eafe4-cert" seLinuxMountContext=""
Oct 31 17:27:17.327503 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.327062    4276 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e" volumeName="kubernetes.io/configmap/47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e-multus-daemon-config" seLinuxMountContext=""
Oct 31 17:27:17.327503 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.327073    4276 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="5819fd64-9ac0-4436-b8fe-dcd3a27827ac" volumeName="kubernetes.io/configmap/5819fd64-9ac0-4436-b8fe-dcd3a27827ac-env-overrides" seLinuxMountContext=""
Oct 31 17:27:17.327503 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.327086    4276 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6d7781d6-8f73-4f1d-bb9a-0212e2eb6e4b" volumeName="kubernetes.io/secret/6d7781d6-8f73-4f1d-bb9a-0212e2eb6e4b-original-pull-secret" seLinuxMountContext=""
Oct 31 17:27:17.327503 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.327097    4276 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="408bcc05-9690-4632-a479-de41718847ad" volumeName="kubernetes.io/configmap/408bcc05-9690-4632-a479-de41718847ad-signing-cabundle" seLinuxMountContext=""
Oct 31 17:27:17.327503 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.327109    4276 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="408bcc05-9690-4632-a479-de41718847ad" volumeName="kubernetes.io/projected/408bcc05-9690-4632-a479-de41718847ad-kube-api-access-5dkwn" seLinuxMountContext=""
Oct 31 17:27:17.327503 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.327120    4276 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="5819fd64-9ac0-4436-b8fe-dcd3a27827ac" volumeName="kubernetes.io/secret/5819fd64-9ac0-4436-b8fe-dcd3a27827ac-ovn-node-metrics-cert" seLinuxMountContext=""
Oct 31 17:27:17.327503 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.327137    4276 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="61462533-417a-4fbc-bd85-c56a0112db31" volumeName="kubernetes.io/empty-dir/61462533-417a-4fbc-bd85-c56a0112db31-data-volume" seLinuxMountContext=""
Oct 31 17:27:17.327503 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.327148    4276 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="72516605-a894-4a62-84c7-834122a01fb8" volumeName="kubernetes.io/configmap/72516605-a894-4a62-84c7-834122a01fb8-cni-binary-copy" seLinuxMountContext=""
Oct 31 17:27:17.327503 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.327171    4276 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9ee82620-6497-48e8-bb2c-1bb77a93f7ef" volumeName="kubernetes.io/empty-dir/9ee82620-6497-48e8-bb2c-1bb77a93f7ef-tmp-dir" seLinuxMountContext=""
Oct 31 17:27:17.327503 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.327183    4276 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b8bbaf33-7cd9-4a30-86f7-6e20ac45b0a0" volumeName="kubernetes.io/projected/b8bbaf33-7cd9-4a30-86f7-6e20ac45b0a0-kube-api-access-grrfv" seLinuxMountContext=""
Oct 31 17:27:17.327503 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.327204    4276 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0f353547-2feb-42f4-aae4-b0f1a30efa34" volumeName="kubernetes.io/projected/0f353547-2feb-42f4-aae4-b0f1a30efa34-kube-api-access-qfvft" seLinuxMountContext=""
Oct 31 17:27:17.327503 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.327217    4276 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e" volumeName="kubernetes.io/configmap/47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e-cni-binary-copy" seLinuxMountContext=""
Oct 31 17:27:17.327503 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.327228    4276 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="72516605-a894-4a62-84c7-834122a01fb8" volumeName="kubernetes.io/configmap/72516605-a894-4a62-84c7-834122a01fb8-cni-sysctl-allowlist" seLinuxMountContext=""
Oct 31 17:27:17.327503 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.327240    4276 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9ee82620-6497-48e8-bb2c-1bb77a93f7ef" volumeName="kubernetes.io/configmap/9ee82620-6497-48e8-bb2c-1bb77a93f7ef-config-volume" seLinuxMountContext=""
Oct 31 17:27:17.327503 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.327252    4276 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="03a15842-25a2-4fbe-b418-8867070c4482" volumeName="kubernetes.io/configmap/03a15842-25a2-4fbe-b418-8867070c4482-iptables-alerter-script" seLinuxMountContext=""
Oct 31 17:27:17.327503 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.327263    4276 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="422b9ca9-3e0e-4fe0-860d-7c7d24b179fa" volumeName="kubernetes.io/empty-dir/422b9ca9-3e0e-4fe0-860d-7c7d24b179fa-etc-tuned" seLinuxMountContext=""
Oct 31 17:27:17.327503 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.327274    4276 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="422b9ca9-3e0e-4fe0-860d-7c7d24b179fa" volumeName="kubernetes.io/empty-dir/422b9ca9-3e0e-4fe0-860d-7c7d24b179fa-tmp" seLinuxMountContext=""
Oct 31 17:27:17.327503 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.327286    4276 reconstruct.go:97] "Volume reconstruction finished"
Oct 31 17:27:17.327503 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.327293    4276 reconciler.go:29] "Reconciler: start to sync state"
Oct 31 17:27:17.329035 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.329016    4276 factory.go:55] Registering systemd factory
Oct 31 17:27:17.329035 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.329036    4276 factory.go:223] Registration of the systemd container factory successfully
Oct 31 17:27:17.329322 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.329301    4276 factory.go:153] Registering CRI-O factory
Oct 31 17:27:17.329322 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.329321    4276 factory.go:223] Registration of the crio container factory successfully
Oct 31 17:27:17.329428 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.329409    4276 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Oct 31 17:27:17.329475 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.329431    4276 factory.go:103] Registering Raw factory
Oct 31 17:27:17.329475 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.329472    4276 manager.go:1199] Started watching for new ooms in manager
Oct 31 17:27:17.330088 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.330067    4276 manager.go:319] Starting recovery of all containers
Oct 31 17:27:17.344482 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.344306    4276 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zpf4d_5819fd64-9ac0-4436-b8fe-dcd3a27827ac/ovn-acl-logging/0.log"
Oct 31 17:27:17.344580 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.344359    4276 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zpf4d_5819fd64-9ac0-4436-b8fe-dcd3a27827ac/ovn-acl-logging/0.log"
Oct 31 17:27:17.358042 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.357986    4276 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Oct 31 17:27:17.359504 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.359431    4276 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Oct 31 17:27:17.359504 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.359484    4276 status_manager.go:249] "Starting to sync pod status with apiserver"
Oct 31 17:27:17.359656 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.359508    4276 kubelet.go:2521] "Starting kubelet main sync loop"
Oct 31 17:27:17.359656 ip-10-0-129-127 kubenswrapper[4276]: E1031 17:27:17.359548    4276 kubelet.go:2545] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]"
Oct 31 17:27:17.370465 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.370386    4276 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zpf4d_5819fd64-9ac0-4436-b8fe-dcd3a27827ac/ovn-acl-logging/0.log"
Oct 31 17:27:17.371036 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.371004    4276 generic.go:356] "Generic (PLEG): container finished" podID="5819fd64-9ac0-4436-b8fe-dcd3a27827ac" containerID="36abb598a1b9a73459f99958a8a088146210713d7eccd28e691c9b3db452839f" exitCode=1
Oct 31 17:27:17.371320 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.371224    4276 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 31 17:27:17.371392 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.371320    4276 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 31 17:27:17.371639 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.371528    4276 reconstruct.go:205] "DevicePaths of reconstructed volumes updated"
Oct 31 17:27:17.379828 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.379804    4276 generic.go:356] "Generic (PLEG): container finished" podID="72516605-a894-4a62-84c7-834122a01fb8" containerID="efce52614641091eab395134ba4291921b92744395442faa68c5c105fea5d93b" exitCode=0
Oct 31 17:27:17.379828 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.379826    4276 generic.go:356] "Generic (PLEG): container finished" podID="72516605-a894-4a62-84c7-834122a01fb8" containerID="26459fede7cfba8105fa171964839c80920c85a67e4f5d5437a00c4979361ac4" exitCode=0
Oct 31 17:27:17.379940 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.379833    4276 generic.go:356] "Generic (PLEG): container finished" podID="72516605-a894-4a62-84c7-834122a01fb8" containerID="b9c7fd4595e7225d0dc5f6e8b3774db596f0e12f6d0e25b47955b86aff870774" exitCode=0
Oct 31 17:27:17.379940 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.379838    4276 generic.go:356] "Generic (PLEG): container finished" podID="72516605-a894-4a62-84c7-834122a01fb8" containerID="ccc7e8c6c149b9d72e46ba296d41e8cf0e76c37142c75a5252e405880139a908" exitCode=0
Oct 31 17:27:17.379940 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.379845    4276 generic.go:356] "Generic (PLEG): container finished" podID="72516605-a894-4a62-84c7-834122a01fb8" containerID="d8271d4a0a9d9c44e5c182ff2acd03ee9b9c0f6a5f54249a839dcbdba7db8502" exitCode=0
Oct 31 17:27:17.379940 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.379850    4276 generic.go:356] "Generic (PLEG): container finished" podID="72516605-a894-4a62-84c7-834122a01fb8" containerID="0d5aa77f90efe1ba96132e50caed925da65017c10c5a09ad886a5e5d98c93c2d" exitCode=0
Oct 31 17:27:17.383057 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.383035    4276 generic.go:356] "Generic (PLEG): container finished" podID="7881d2520ea389377a82c2df6a5dd230" containerID="28df02d866c4382f7332e300ecdd5e635b5f4ff8018778ce0f4c266e6f6a9a66" exitCode=0
Oct 31 17:27:17.384896 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.384876    4276 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 31 17:27:17.387194 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.387169    4276 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 31 17:27:17.396712 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.396697    4276 manager.go:324] Recovery completed
Oct 31 17:27:17.410238 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.410223    4276 cpu_manager.go:226] "Starting" policy="none"
Oct 31 17:27:17.410238 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.410235    4276 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s"
Oct 31 17:27:17.410335 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.410250    4276 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory"
Oct 31 17:27:17.410394 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.410382    4276 state_mem.go:94] "Updated default CPUSet" logger="CPUManager state checkpoint.CPUManager state memory" cpuSet=""
Oct 31 17:27:17.410428 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.410395    4276 state_mem.go:102] "Updated CPUSet assignments" logger="CPUManager state checkpoint.CPUManager state memory" assignments={}
Oct 31 17:27:17.410428 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.410419    4276 state_checkpoint.go:142] "restored state from checkpoint" logger="CPUManager state checkpoint"
Oct 31 17:27:17.410428 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.410425    4276 state_checkpoint.go:143] "defaultCPUSet" logger="CPUManager state checkpoint" defaultCpuSet=""
Oct 31 17:27:17.410523 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.410430    4276 policy_none.go:50] "Start"
Oct 31 17:27:17.410523 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.410439    4276 memory_manager.go:187] "Starting memorymanager" policy="None"
Oct 31 17:27:17.410523 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.410462    4276 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Oct 31 17:27:17.410604 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.410579    4276 state_mem.go:77] "Updated machine memory state" logger="Memory Manager state checkpoint"
Oct 31 17:27:17.410604 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.410585    4276 state_checkpoint.go:85] "State checkpoint: restored state from checkpoint" logger="Memory Manager state checkpoint"
Oct 31 17:27:17.410604 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.410591    4276 policy_none.go:44] "Start"
Oct 31 17:27:17.414946 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.414502    4276 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"}
Oct 31 17:27:17.419530 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.419517    4276 manager.go:341] "Starting Device Plugin manager"
Oct 31 17:27:17.419607 ip-10-0-129-127 kubenswrapper[4276]: E1031 17:27:17.419544    4276 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Oct 31 17:27:17.419607 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.419555    4276 server.go:87] "Starting device plugin registration server"
Oct 31 17:27:17.419823 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.419811    4276 eviction_manager.go:194] "Eviction manager: starting control loop"
Oct 31 17:27:17.419882 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.419823    4276 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Oct 31 17:27:17.419939 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.419920    4276 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Oct 31 17:27:17.420069 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.420034    4276 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Oct 31 17:27:17.420135 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.420078    4276 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts"
Oct 31 17:27:17.420135 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.420085    4276 plugin_manager.go:121] "Starting Kubelet Plugin Manager"
Oct 31 17:27:17.420230 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.420151    4276 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-10-31T17:27:17.420067688Z","UUID":"1c4f025a-713b-4248-bf61-ae9dc49a3266","Handler":null,"Name":"","Endpoint":""}
Oct 31 17:27:17.421314 ip-10-0-129-127 kubenswrapper[4276]: E1031 17:27:17.421292    4276 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Oct 31 17:27:17.421510 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.421493    4276 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Oct 31 17:27:17.421584 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.421522    4276 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Oct 31 17:27:17.459820 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.459781    4276 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-129-127.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-127.ec2.internal"]
Oct 31 17:27:17.460350 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.460308    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-127.ec2.internal" event={"ID":"7881d2520ea389377a82c2df6a5dd230","Type":"ContainerStarted","Data":"854d2a4788afed5a0ed0422ed9fce13da26d540fb8e030acb58a99a87e871139"}
Oct 31 17:27:17.460462 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.460356    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-127.ec2.internal" event={"ID":"7881d2520ea389377a82c2df6a5dd230","Type":"ContainerDied","Data":"28df02d866c4382f7332e300ecdd5e635b5f4ff8018778ce0f4c266e6f6a9a66"}
Oct 31 17:27:17.460462 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.460378    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-127.ec2.internal" event={"ID":"7881d2520ea389377a82c2df6a5dd230","Type":"ContainerStarted","Data":"9c8e2f2a20f397a3b0cf89117bb25ddffba6af2b982702b73388a88b5fa317ec"}
Oct 31 17:27:17.460462 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.460408    4276 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-127.ec2.internal"
Oct 31 17:27:17.460462 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.460417    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-127.ec2.internal" event={"ID":"5136faca2de3de7a06b0d414d9c6fca7","Type":"ContainerStarted","Data":"8411ddf233654f7d3c14723783cca577f7b8e71023396d89dfa6620c9f5a4700"}
Oct 31 17:27:17.460462 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.460436    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-127.ec2.internal" event={"ID":"5136faca2de3de7a06b0d414d9c6fca7","Type":"ContainerStarted","Data":"0c07b7277e444faad0e0d4df98b62cbe2593d83155d3456ed11cc10e0c1cb0a1"}
Oct 31 17:27:17.460650 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.460555    4276 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-127.ec2.internal"
Oct 31 17:27:17.481350 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.481332    4276 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 31 17:27:17.481418 ip-10-0-129-127 kubenswrapper[4276]: E1031 17:27:17.481374    4276 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-129-127.ec2.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-127.ec2.internal"
Oct 31 17:27:17.481850 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.481840    4276 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 31 17:27:17.481895 ip-10-0-129-127 kubenswrapper[4276]: E1031 17:27:17.481870    4276 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-129-127.ec2.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-127.ec2.internal"
Oct 31 17:27:17.519994 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.519976    4276 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 31 17:27:17.521218 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.521200    4276 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-127.ec2.internal" event="NodeHasSufficientMemory"
Oct 31 17:27:17.521304 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.521235    4276 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-127.ec2.internal" event="NodeHasNoDiskPressure"
Oct 31 17:27:17.521304 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.521248    4276 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-127.ec2.internal" event="NodeHasSufficientPID"
Oct 31 17:27:17.521304 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.521287    4276 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-129-127.ec2.internal"
Oct 31 17:27:17.556617 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.556595    4276 kubelet_node_status.go:126] "Node was previously registered" node="ip-10-0-129-127.ec2.internal"
Oct 31 17:27:17.556684 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.556639    4276 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-129-127.ec2.internal"
Oct 31 17:27:17.573245 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.573228    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7881d2520ea389377a82c2df6a5dd230-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-127.ec2.internal\" (UID: \"7881d2520ea389377a82c2df6a5dd230\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-127.ec2.internal"
Oct 31 17:27:17.573323 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.573252    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7881d2520ea389377a82c2df6a5dd230-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-127.ec2.internal\" (UID: \"7881d2520ea389377a82c2df6a5dd230\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-127.ec2.internal"
Oct 31 17:27:17.573323 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.573268    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/5136faca2de3de7a06b0d414d9c6fca7-config\") pod \"kube-apiserver-proxy-ip-10-0-129-127.ec2.internal\" (UID: \"5136faca2de3de7a06b0d414d9c6fca7\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-127.ec2.internal"
Oct 31 17:27:17.673857 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.673776    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7881d2520ea389377a82c2df6a5dd230-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-127.ec2.internal\" (UID: \"7881d2520ea389377a82c2df6a5dd230\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-127.ec2.internal"
Oct 31 17:27:17.673857 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.673814    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7881d2520ea389377a82c2df6a5dd230-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-127.ec2.internal\" (UID: \"7881d2520ea389377a82c2df6a5dd230\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-127.ec2.internal"
Oct 31 17:27:17.673857 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.673836    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/5136faca2de3de7a06b0d414d9c6fca7-config\") pod \"kube-apiserver-proxy-ip-10-0-129-127.ec2.internal\" (UID: \"5136faca2de3de7a06b0d414d9c6fca7\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-127.ec2.internal"
Oct 31 17:27:17.674083 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.673887    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/5136faca2de3de7a06b0d414d9c6fca7-config\") pod \"kube-apiserver-proxy-ip-10-0-129-127.ec2.internal\" (UID: \"5136faca2de3de7a06b0d414d9c6fca7\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-127.ec2.internal"
Oct 31 17:27:17.674083 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.673895    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7881d2520ea389377a82c2df6a5dd230-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-127.ec2.internal\" (UID: \"7881d2520ea389377a82c2df6a5dd230\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-127.ec2.internal"
Oct 31 17:27:17.674083 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:17.673895    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7881d2520ea389377a82c2df6a5dd230-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-127.ec2.internal\" (UID: \"7881d2520ea389377a82c2df6a5dd230\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-127.ec2.internal"
Oct 31 17:27:18.308973 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.308943    4276 apiserver.go:52] "Watching apiserver"
Oct 31 17:27:18.354602 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.354578    4276 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Oct 31 17:27:18.354968 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.354949    4276 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-dns/node-resolver-7pfr4","openshift-ingress-canary/ingress-canary-b5ln2","openshift-multus/multus-additional-cni-plugins-fhgqw","openshift-service-ca/service-ca-5fffc97bdf-ncfqp","openshift-insights/insights-runtime-extractor-tg5qk","openshift-network-operator/iptables-alerter-jzcgs","openshift-ovn-kubernetes/ovnkube-node-zpf4d","kube-system/global-pull-secret-syncer-kmkdh","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2v874","openshift-multus/network-metrics-daemon-pnjdm","openshift-network-diagnostics/network-check-target-tskzw","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-127.ec2.internal","openshift-cluster-node-tuning-operator/tuned-rvsdp","openshift-dns/dns-default-59fpc","kube-system/kube-apiserver-proxy-ip-10-0-129-127.ec2.internal","openshift-image-registry/node-ca-qxnvq","openshift-multus/multus-rl4hl","kube-system/konnectivity-agent-mnbmt"]
Oct 31 17:27:18.360697 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.360674    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pnjdm"
Oct 31 17:27:18.362153 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.362134    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-tg5qk"
Oct 31 17:27:18.372183 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.372158    4276 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Oct 31 17:27:18.372302 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.372282    4276 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Oct 31 17:27:18.372361 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.372320    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-nk5tf\""
Oct 31 17:27:18.372361 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.372325    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-5bhk8\""
Oct 31 17:27:18.372476 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.372402    4276 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Oct 31 17:27:18.372476 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.372464    4276 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Oct 31 17:27:18.372476 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.372473    4276 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Oct 31 17:27:18.372606 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.372501    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Oct 31 17:27:18.372606 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.372507    4276 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Oct 31 17:27:18.372606 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.372511    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-k6pvq\""
Oct 31 17:27:18.372606 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.372528    4276 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Oct 31 17:27:18.372606 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.372497    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-g2t4b\""
Oct 31 17:27:18.372606 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.372514    4276 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Oct 31 17:27:18.372606 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.372557    4276 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Oct 31 17:27:18.372606 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.372604    4276 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Oct 31 17:27:18.373723 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.373711    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-x4c5q\""
Oct 31 17:27:18.373769 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.373738    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-xbzzg\""
Oct 31 17:27:18.373940 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.373928    4276 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Oct 31 17:27:18.373994 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.373983    4276 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Oct 31 17:27:18.374049 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.374039    4276 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Oct 31 17:27:18.374049 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.374044    4276 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Oct 31 17:27:18.374294 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.374282    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-8d7mp\""
Oct 31 17:27:18.383640 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.383626    4276 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Oct 31 17:27:18.383685 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.383638    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Oct 31 17:27:18.384218 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.384204    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Oct 31 17:27:18.384683 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.384664    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Oct 31 17:27:18.384772 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.384701    4276 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Oct 31 17:27:18.384806 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.384770    4276 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Oct 31 17:27:18.385563 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.385548    4276 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Oct 31 17:27:18.385638 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.385578    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-z2hdt\""
Oct 31 17:27:18.385638 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.385581    4276 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Oct 31 17:27:18.385731 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.385639    4276 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Oct 31 17:27:18.385731 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.385670    4276 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Oct 31 17:27:18.385731 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.385688    4276 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Oct 31 17:27:18.385731 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.385700    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Oct 31 17:27:18.385731 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.385710    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-zv4z8\""
Oct 31 17:27:18.385731 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.385718    4276 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Oct 31 17:27:18.385731 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.385728    4276 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Oct 31 17:27:18.386142 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.385745    4276 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Oct 31 17:27:18.386142 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.385747    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-4g54t\""
Oct 31 17:27:18.386142 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.385761    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-tj66q\""
Oct 31 17:27:18.386142 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.385813    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-9gvm6\""
Oct 31 17:27:18.386142 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.385871    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Oct 31 17:27:18.386142 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.385881    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-vvnlg\""
Oct 31 17:27:18.386142 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.385927    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-8gjvp\""
Oct 31 17:27:18.386142 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.385941    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Oct 31 17:27:18.386142 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.385951    4276 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Oct 31 17:27:18.386142 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.385969    4276 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Oct 31 17:27:18.386142 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.385985    4276 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Oct 31 17:27:18.386142 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.385998    4276 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Oct 31 17:27:18.386142 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.386005    4276 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Oct 31 17:27:18.386142 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.386042    4276 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Oct 31 17:27:18.386142 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.386050    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Oct 31 17:27:18.386142 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.386052    4276 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Oct 31 17:27:18.386142 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.386081    4276 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Oct 31 17:27:18.386142 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.386129    4276 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Oct 31 17:27:18.386142 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.386146    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-dm9tc\""
Oct 31 17:27:18.386921 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.386200    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-hpj5l\""
Oct 31 17:27:18.386921 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.386222    4276 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Oct 31 17:27:18.386921 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.386584    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Oct 31 17:27:18.396955 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.396940    4276 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-127.ec2.internal"
Oct 31 17:27:18.397082 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.397070    4276 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-127.ec2.internal"
Oct 31 17:27:18.420208 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.420190    4276 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 31 17:27:18.420280 ip-10-0-129-127 kubenswrapper[4276]: E1031 17:27:18.420225    4276 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-129-127.ec2.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-127.ec2.internal"
Oct 31 17:27:18.420610 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.420597    4276 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 31 17:27:18.420656 ip-10-0-129-127 kubenswrapper[4276]: E1031 17:27:18.420626    4276 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-129-127.ec2.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-127.ec2.internal"
Oct 31 17:27:18.426689 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.426674    4276 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Oct 31 17:27:18.479456 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.479422    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5819fd64-9ac0-4436-b8fe-dcd3a27827ac-run-openvswitch\") pod \"ovnkube-node-zpf4d\" (UID: \"5819fd64-9ac0-4436-b8fe-dcd3a27827ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpf4d"
Oct 31 17:27:18.479530 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.479463    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/72516605-a894-4a62-84c7-834122a01fb8-tuning-conf-dir\") pod \"multus-additional-cni-plugins-fhgqw\" (UID: \"72516605-a894-4a62-84c7-834122a01fb8\") " pod="openshift-multus/multus-additional-cni-plugins-fhgqw"
Oct 31 17:27:18.479530 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.479484    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9ee82620-6497-48e8-bb2c-1bb77a93f7ef-tmp-dir\") pod \"dns-default-59fpc\" (UID: \"9ee82620-6497-48e8-bb2c-1bb77a93f7ef\") " pod="openshift-dns/dns-default-59fpc"
Oct 31 17:27:18.479605 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.479545    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/408bcc05-9690-4632-a479-de41718847ad-signing-key\") pod \"service-ca-5fffc97bdf-ncfqp\" (UID: \"408bcc05-9690-4632-a479-de41718847ad\") " pod="openshift-service-ca/service-ca-5fffc97bdf-ncfqp"
Oct 31 17:27:18.479605 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.479569    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/72516605-a894-4a62-84c7-834122a01fb8-system-cni-dir\") pod \"multus-additional-cni-plugins-fhgqw\" (UID: \"72516605-a894-4a62-84c7-834122a01fb8\") " pod="openshift-multus/multus-additional-cni-plugins-fhgqw"
Oct 31 17:27:18.479662 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.479641    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9ee82620-6497-48e8-bb2c-1bb77a93f7ef-tmp-dir\") pod \"dns-default-59fpc\" (UID: \"9ee82620-6497-48e8-bb2c-1bb77a93f7ef\") " pod="openshift-dns/dns-default-59fpc"
Oct 31 17:27:18.479694 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.479669    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-28ft5\" (UniqueName: \"kubernetes.io/projected/47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e-kube-api-access-28ft5\") pod \"multus-rl4hl\" (UID: \"47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e\") " pod="openshift-multus/multus-rl4hl"
Oct 31 17:27:18.479729 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.479703    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/61462533-417a-4fbc-bd85-c56a0112db31-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tg5qk\" (UID: \"61462533-417a-4fbc-bd85-c56a0112db31\") " pod="openshift-insights/insights-runtime-extractor-tg5qk"
Oct 31 17:27:18.479729 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.479719    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/422b9ca9-3e0e-4fe0-860d-7c7d24b179fa-etc-sysctl-d\") pod \"tuned-rvsdp\" (UID: \"422b9ca9-3e0e-4fe0-860d-7c7d24b179fa\") " pod="openshift-cluster-node-tuning-operator/tuned-rvsdp"
Oct 31 17:27:18.479798 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.479735    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/5819fd64-9ac0-4436-b8fe-dcd3a27827ac-host-kubelet\") pod \"ovnkube-node-zpf4d\" (UID: \"5819fd64-9ac0-4436-b8fe-dcd3a27827ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpf4d"
Oct 31 17:27:18.479798 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.479782    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/5819fd64-9ac0-4436-b8fe-dcd3a27827ac-systemd-units\") pod \"ovnkube-node-zpf4d\" (UID: \"5819fd64-9ac0-4436-b8fe-dcd3a27827ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpf4d"
Oct 31 17:27:18.479868 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.479806    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qfvft\" (UniqueName: \"kubernetes.io/projected/0f353547-2feb-42f4-aae4-b0f1a30efa34-kube-api-access-qfvft\") pod \"aws-ebs-csi-driver-node-2v874\" (UID: \"0f353547-2feb-42f4-aae4-b0f1a30efa34\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2v874"
Oct 31 17:27:18.479868 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.479823    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/72516605-a894-4a62-84c7-834122a01fb8-cnibin\") pod \"multus-additional-cni-plugins-fhgqw\" (UID: \"72516605-a894-4a62-84c7-834122a01fb8\") " pod="openshift-multus/multus-additional-cni-plugins-fhgqw"
Oct 31 17:27:18.479868 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.479838    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e-system-cni-dir\") pod \"multus-rl4hl\" (UID: \"47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e\") " pod="openshift-multus/multus-rl4hl"
Oct 31 17:27:18.480022 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.479867    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e-host-var-lib-cni-multus\") pod \"multus-rl4hl\" (UID: \"47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e\") " pod="openshift-multus/multus-rl4hl"
Oct 31 17:27:18.480022 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.479881    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/408bcc05-9690-4632-a479-de41718847ad-signing-key\") pod \"service-ca-5fffc97bdf-ncfqp\" (UID: \"408bcc05-9690-4632-a479-de41718847ad\") " pod="openshift-service-ca/service-ca-5fffc97bdf-ncfqp"
Oct 31 17:27:18.480022 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.479910    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z8zhc\" (UniqueName: \"kubernetes.io/projected/9ee82620-6497-48e8-bb2c-1bb77a93f7ef-kube-api-access-z8zhc\") pod \"dns-default-59fpc\" (UID: \"9ee82620-6497-48e8-bb2c-1bb77a93f7ef\") " pod="openshift-dns/dns-default-59fpc"
Oct 31 17:27:18.480022 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.479964    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/422b9ca9-3e0e-4fe0-860d-7c7d24b179fa-etc-sysconfig\") pod \"tuned-rvsdp\" (UID: \"422b9ca9-3e0e-4fe0-860d-7c7d24b179fa\") " pod="openshift-cluster-node-tuning-operator/tuned-rvsdp"
Oct 31 17:27:18.480022 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.479989    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/5819fd64-9ac0-4436-b8fe-dcd3a27827ac-ovnkube-script-lib\") pod \"ovnkube-node-zpf4d\" (UID: \"5819fd64-9ac0-4436-b8fe-dcd3a27827ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpf4d"
Oct 31 17:27:18.480022 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.480005    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e-cnibin\") pod \"multus-rl4hl\" (UID: \"47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e\") " pod="openshift-multus/multus-rl4hl"
Oct 31 17:27:18.480263 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.480027    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e-multus-socket-dir-parent\") pod \"multus-rl4hl\" (UID: \"47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e\") " pod="openshift-multus/multus-rl4hl"
Oct 31 17:27:18.480263 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.480048    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e-host-run-k8s-cni-cncf-io\") pod \"multus-rl4hl\" (UID: \"47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e\") " pod="openshift-multus/multus-rl4hl"
Oct 31 17:27:18.480263 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.480064    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e-host-var-lib-cni-bin\") pod \"multus-rl4hl\" (UID: \"47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e\") " pod="openshift-multus/multus-rl4hl"
Oct 31 17:27:18.480263 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.480079    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5819fd64-9ac0-4436-b8fe-dcd3a27827ac-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-zpf4d\" (UID: \"5819fd64-9ac0-4436-b8fe-dcd3a27827ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpf4d"
Oct 31 17:27:18.480263 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.480096    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/0f353547-2feb-42f4-aae4-b0f1a30efa34-sys-fs\") pod \"aws-ebs-csi-driver-node-2v874\" (UID: \"0f353547-2feb-42f4-aae4-b0f1a30efa34\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2v874"
Oct 31 17:27:18.480263 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.480130    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e-host-run-multus-certs\") pod \"multus-rl4hl\" (UID: \"47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e\") " pod="openshift-multus/multus-rl4hl"
Oct 31 17:27:18.480263 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.480161    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/72516605-a894-4a62-84c7-834122a01fb8-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-fhgqw\" (UID: \"72516605-a894-4a62-84c7-834122a01fb8\") " pod="openshift-multus/multus-additional-cni-plugins-fhgqw"
Oct 31 17:27:18.480263 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.480196    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e-host-var-lib-kubelet\") pod \"multus-rl4hl\" (UID: \"47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e\") " pod="openshift-multus/multus-rl4hl"
Oct 31 17:27:18.480263 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.480219    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e-hostroot\") pod \"multus-rl4hl\" (UID: \"47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e\") " pod="openshift-multus/multus-rl4hl"
Oct 31 17:27:18.480263 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.480254    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e-multus-daemon-config\") pod \"multus-rl4hl\" (UID: \"47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e\") " pod="openshift-multus/multus-rl4hl"
Oct 31 17:27:18.480599 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.480294    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/5819fd64-9ac0-4436-b8fe-dcd3a27827ac-ovnkube-script-lib\") pod \"ovnkube-node-zpf4d\" (UID: \"5819fd64-9ac0-4436-b8fe-dcd3a27827ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpf4d"
Oct 31 17:27:18.480599 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.480332    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d6ck5\" (UniqueName: \"kubernetes.io/projected/138bfcc8-f199-4ffd-b5d2-8d83ade6199a-kube-api-access-d6ck5\") pod \"network-metrics-daemon-pnjdm\" (UID: \"138bfcc8-f199-4ffd-b5d2-8d83ade6199a\") " pod="openshift-multus/network-metrics-daemon-pnjdm"
Oct 31 17:27:18.480599 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.480482    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/422b9ca9-3e0e-4fe0-860d-7c7d24b179fa-etc-kubernetes\") pod \"tuned-rvsdp\" (UID: \"422b9ca9-3e0e-4fe0-860d-7c7d24b179fa\") " pod="openshift-cluster-node-tuning-operator/tuned-rvsdp"
Oct 31 17:27:18.480599 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.480497    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e-multus-daemon-config\") pod \"multus-rl4hl\" (UID: \"47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e\") " pod="openshift-multus/multus-rl4hl"
Oct 31 17:27:18.480599 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.480504    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/72516605-a894-4a62-84c7-834122a01fb8-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-fhgqw\" (UID: \"72516605-a894-4a62-84c7-834122a01fb8\") " pod="openshift-multus/multus-additional-cni-plugins-fhgqw"
Oct 31 17:27:18.480599 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.480522    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/422b9ca9-3e0e-4fe0-860d-7c7d24b179fa-run\") pod \"tuned-rvsdp\" (UID: \"422b9ca9-3e0e-4fe0-860d-7c7d24b179fa\") " pod="openshift-cluster-node-tuning-operator/tuned-rvsdp"
Oct 31 17:27:18.480599 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.480552    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9ee82620-6497-48e8-bb2c-1bb77a93f7ef-metrics-tls\") pod \"dns-default-59fpc\" (UID: \"9ee82620-6497-48e8-bb2c-1bb77a93f7ef\") " pod="openshift-dns/dns-default-59fpc"
Oct 31 17:27:18.480599 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.480600    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6d7781d6-8f73-4f1d-bb9a-0212e2eb6e4b-dbus\") pod \"global-pull-secret-syncer-kmkdh\" (UID: \"6d7781d6-8f73-4f1d-bb9a-0212e2eb6e4b\") " pod="kube-system/global-pull-secret-syncer-kmkdh"
Oct 31 17:27:18.480942 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.480690    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/422b9ca9-3e0e-4fe0-860d-7c7d24b179fa-sys\") pod \"tuned-rvsdp\" (UID: \"422b9ca9-3e0e-4fe0-860d-7c7d24b179fa\") " pod="openshift-cluster-node-tuning-operator/tuned-rvsdp"
Oct 31 17:27:18.480942 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.480716    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5819fd64-9ac0-4436-b8fe-dcd3a27827ac-host-slash\") pod \"ovnkube-node-zpf4d\" (UID: \"5819fd64-9ac0-4436-b8fe-dcd3a27827ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpf4d"
Oct 31 17:27:18.480942 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.480733    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5819fd64-9ac0-4436-b8fe-dcd3a27827ac-host-run-netns\") pod \"ovnkube-node-zpf4d\" (UID: \"5819fd64-9ac0-4436-b8fe-dcd3a27827ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpf4d"
Oct 31 17:27:18.480942 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.480757    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5819fd64-9ac0-4436-b8fe-dcd3a27827ac-host-cni-bin\") pod \"ovnkube-node-zpf4d\" (UID: \"5819fd64-9ac0-4436-b8fe-dcd3a27827ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpf4d"
Oct 31 17:27:18.480942 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.480768    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9ee82620-6497-48e8-bb2c-1bb77a93f7ef-metrics-tls\") pod \"dns-default-59fpc\" (UID: \"9ee82620-6497-48e8-bb2c-1bb77a93f7ef\") " pod="openshift-dns/dns-default-59fpc"
Oct 31 17:27:18.480942 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.480776    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/87e61dcd-75f6-4d3f-8ed9-7d35b06a9bb3-serviceca\") pod \"node-ca-qxnvq\" (UID: \"87e61dcd-75f6-4d3f-8ed9-7d35b06a9bb3\") " pod="openshift-image-registry/node-ca-qxnvq"
Oct 31 17:27:18.480942 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.480834    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b8bbaf33-7cd9-4a30-86f7-6e20ac45b0a0-tmp-dir\") pod \"node-resolver-7pfr4\" (UID: \"b8bbaf33-7cd9-4a30-86f7-6e20ac45b0a0\") " pod="openshift-dns/node-resolver-7pfr4"
Oct 31 17:27:18.480942 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.480875    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/61462533-417a-4fbc-bd85-c56a0112db31-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tg5qk\" (UID: \"61462533-417a-4fbc-bd85-c56a0112db31\") " pod="openshift-insights/insights-runtime-extractor-tg5qk"
Oct 31 17:27:18.480942 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.480917    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/422b9ca9-3e0e-4fe0-860d-7c7d24b179fa-etc-sysctl-conf\") pod \"tuned-rvsdp\" (UID: \"422b9ca9-3e0e-4fe0-860d-7c7d24b179fa\") " pod="openshift-cluster-node-tuning-operator/tuned-rvsdp"
Oct 31 17:27:18.480942 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.480938    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b8bbaf33-7cd9-4a30-86f7-6e20ac45b0a0-tmp-dir\") pod \"node-resolver-7pfr4\" (UID: \"b8bbaf33-7cd9-4a30-86f7-6e20ac45b0a0\") " pod="openshift-dns/node-resolver-7pfr4"
Oct 31 17:27:18.480942 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.480945    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/422b9ca9-3e0e-4fe0-860d-7c7d24b179fa-etc-systemd\") pod \"tuned-rvsdp\" (UID: \"422b9ca9-3e0e-4fe0-860d-7c7d24b179fa\") " pod="openshift-cluster-node-tuning-operator/tuned-rvsdp"
Oct 31 17:27:18.481379 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.481037    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/422b9ca9-3e0e-4fe0-860d-7c7d24b179fa-tmp\") pod \"tuned-rvsdp\" (UID: \"422b9ca9-3e0e-4fe0-860d-7c7d24b179fa\") " pod="openshift-cluster-node-tuning-operator/tuned-rvsdp"
Oct 31 17:27:18.481379 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.481077    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/61462533-417a-4fbc-bd85-c56a0112db31-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tg5qk\" (UID: \"61462533-417a-4fbc-bd85-c56a0112db31\") " pod="openshift-insights/insights-runtime-extractor-tg5qk"
Oct 31 17:27:18.481379 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.481079    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5819fd64-9ac0-4436-b8fe-dcd3a27827ac-var-lib-openvswitch\") pod \"ovnkube-node-zpf4d\" (UID: \"5819fd64-9ac0-4436-b8fe-dcd3a27827ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpf4d"
Oct 31 17:27:18.481379 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.481130    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/5819fd64-9ac0-4436-b8fe-dcd3a27827ac-run-ovn\") pod \"ovnkube-node-zpf4d\" (UID: \"5819fd64-9ac0-4436-b8fe-dcd3a27827ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpf4d"
Oct 31 17:27:18.481379 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.481137    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/422b9ca9-3e0e-4fe0-860d-7c7d24b179fa-tmp\") pod \"tuned-rvsdp\" (UID: \"422b9ca9-3e0e-4fe0-860d-7c7d24b179fa\") " pod="openshift-cluster-node-tuning-operator/tuned-rvsdp"
Oct 31 17:27:18.481379 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.481167    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/0f353547-2feb-42f4-aae4-b0f1a30efa34-registration-dir\") pod \"aws-ebs-csi-driver-node-2v874\" (UID: \"0f353547-2feb-42f4-aae4-b0f1a30efa34\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2v874"
Oct 31 17:27:18.481379 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.481193    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5dkwn\" (UniqueName: \"kubernetes.io/projected/408bcc05-9690-4632-a479-de41718847ad-kube-api-access-5dkwn\") pod \"service-ca-5fffc97bdf-ncfqp\" (UID: \"408bcc05-9690-4632-a479-de41718847ad\") " pod="openshift-service-ca/service-ca-5fffc97bdf-ncfqp"
Oct 31 17:27:18.481379 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.481245    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/03a15842-25a2-4fbe-b418-8867070c4482-host-slash\") pod \"iptables-alerter-jzcgs\" (UID: \"03a15842-25a2-4fbe-b418-8867070c4482\") " pod="openshift-network-operator/iptables-alerter-jzcgs"
Oct 31 17:27:18.481379 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.481274    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p2gtl\" (UniqueName: \"kubernetes.io/projected/61462533-417a-4fbc-bd85-c56a0112db31-kube-api-access-p2gtl\") pod \"insights-runtime-extractor-tg5qk\" (UID: \"61462533-417a-4fbc-bd85-c56a0112db31\") " pod="openshift-insights/insights-runtime-extractor-tg5qk"
Oct 31 17:27:18.481379 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.481301    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5819fd64-9ac0-4436-b8fe-dcd3a27827ac-host-run-ovn-kubernetes\") pod \"ovnkube-node-zpf4d\" (UID: \"5819fd64-9ac0-4436-b8fe-dcd3a27827ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpf4d"
Oct 31 17:27:18.481379 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.481358    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/0f353547-2feb-42f4-aae4-b0f1a30efa34-device-dir\") pod \"aws-ebs-csi-driver-node-2v874\" (UID: \"0f353547-2feb-42f4-aae4-b0f1a30efa34\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2v874"
Oct 31 17:27:18.481379 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.481375    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/87e61dcd-75f6-4d3f-8ed9-7d35b06a9bb3-serviceca\") pod \"node-ca-qxnvq\" (UID: \"87e61dcd-75f6-4d3f-8ed9-7d35b06a9bb3\") " pod="openshift-image-registry/node-ca-qxnvq"
Oct 31 17:27:18.481379 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.481391    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/72516605-a894-4a62-84c7-834122a01fb8-cni-binary-copy\") pod \"multus-additional-cni-plugins-fhgqw\" (UID: \"72516605-a894-4a62-84c7-834122a01fb8\") " pod="openshift-multus/multus-additional-cni-plugins-fhgqw"
Oct 31 17:27:18.481906 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.481429    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/422b9ca9-3e0e-4fe0-860d-7c7d24b179fa-var-lib-kubelet\") pod \"tuned-rvsdp\" (UID: \"422b9ca9-3e0e-4fe0-860d-7c7d24b179fa\") " pod="openshift-cluster-node-tuning-operator/tuned-rvsdp"
Oct 31 17:27:18.481906 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.481477    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e-multus-cni-dir\") pod \"multus-rl4hl\" (UID: \"47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e\") " pod="openshift-multus/multus-rl4hl"
Oct 31 17:27:18.481906 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.481504    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/9ee82620-6497-48e8-bb2c-1bb77a93f7ef-config-volume\") pod \"dns-default-59fpc\" (UID: \"9ee82620-6497-48e8-bb2c-1bb77a93f7ef\") " pod="openshift-dns/dns-default-59fpc"
Oct 31 17:27:18.481906 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.481528    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/2f646206-1186-4fcf-a9ca-fe92371d4d2b-konnectivity-ca\") pod \"konnectivity-agent-mnbmt\" (UID: \"2f646206-1186-4fcf-a9ca-fe92371d4d2b\") " pod="kube-system/konnectivity-agent-mnbmt"
Oct 31 17:27:18.481906 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.481543    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6d7781d6-8f73-4f1d-bb9a-0212e2eb6e4b-original-pull-secret\") pod \"global-pull-secret-syncer-kmkdh\" (UID: \"6d7781d6-8f73-4f1d-bb9a-0212e2eb6e4b\") " pod="kube-system/global-pull-secret-syncer-kmkdh"
Oct 31 17:27:18.481906 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.481603    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b8bbaf33-7cd9-4a30-86f7-6e20ac45b0a0-hosts-file\") pod \"node-resolver-7pfr4\" (UID: \"b8bbaf33-7cd9-4a30-86f7-6e20ac45b0a0\") " pod="openshift-dns/node-resolver-7pfr4"
Oct 31 17:27:18.481906 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.481618    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/72516605-a894-4a62-84c7-834122a01fb8-cni-binary-copy\") pod \"multus-additional-cni-plugins-fhgqw\" (UID: \"72516605-a894-4a62-84c7-834122a01fb8\") " pod="openshift-multus/multus-additional-cni-plugins-fhgqw"
Oct 31 17:27:18.481906 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.481682    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/5819fd64-9ac0-4436-b8fe-dcd3a27827ac-run-systemd\") pod \"ovnkube-node-zpf4d\" (UID: \"5819fd64-9ac0-4436-b8fe-dcd3a27827ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpf4d"
Oct 31 17:27:18.481906 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.481692    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/9ee82620-6497-48e8-bb2c-1bb77a93f7ef-config-volume\") pod \"dns-default-59fpc\" (UID: \"9ee82620-6497-48e8-bb2c-1bb77a93f7ef\") " pod="openshift-dns/dns-default-59fpc"
Oct 31 17:27:18.481906 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.481721    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/5819fd64-9ac0-4436-b8fe-dcd3a27827ac-log-socket\") pod \"ovnkube-node-zpf4d\" (UID: \"5819fd64-9ac0-4436-b8fe-dcd3a27827ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpf4d"
Oct 31 17:27:18.481906 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.481766    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vv9nf\" (UniqueName: \"kubernetes.io/projected/87e61dcd-75f6-4d3f-8ed9-7d35b06a9bb3-kube-api-access-vv9nf\") pod \"node-ca-qxnvq\" (UID: \"87e61dcd-75f6-4d3f-8ed9-7d35b06a9bb3\") " pod="openshift-image-registry/node-ca-qxnvq"
Oct 31 17:27:18.481906 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.481769    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/2f646206-1186-4fcf-a9ca-fe92371d4d2b-konnectivity-ca\") pod \"konnectivity-agent-mnbmt\" (UID: \"2f646206-1186-4fcf-a9ca-fe92371d4d2b\") " pod="kube-system/konnectivity-agent-mnbmt"
Oct 31 17:27:18.481906 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.481783    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6d7781d6-8f73-4f1d-bb9a-0212e2eb6e4b-original-pull-secret\") pod \"global-pull-secret-syncer-kmkdh\" (UID: \"6d7781d6-8f73-4f1d-bb9a-0212e2eb6e4b\") " pod="kube-system/global-pull-secret-syncer-kmkdh"
Oct 31 17:27:18.481906 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.481799    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-59c8z\" (UniqueName: \"kubernetes.io/projected/422b9ca9-3e0e-4fe0-860d-7c7d24b179fa-kube-api-access-59c8z\") pod \"tuned-rvsdp\" (UID: \"422b9ca9-3e0e-4fe0-860d-7c7d24b179fa\") " pod="openshift-cluster-node-tuning-operator/tuned-rvsdp"
Oct 31 17:27:18.481906 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.481827    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e-multus-conf-dir\") pod \"multus-rl4hl\" (UID: \"47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e\") " pod="openshift-multus/multus-rl4hl"
Oct 31 17:27:18.481906 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.481854    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/2f646206-1186-4fcf-a9ca-fe92371d4d2b-agent-certs\") pod \"konnectivity-agent-mnbmt\" (UID: \"2f646206-1186-4fcf-a9ca-fe92371d4d2b\") " pod="kube-system/konnectivity-agent-mnbmt"
Oct 31 17:27:18.481906 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.481881    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/61462533-417a-4fbc-bd85-c56a0112db31-crio-socket\") pod \"insights-runtime-extractor-tg5qk\" (UID: \"61462533-417a-4fbc-bd85-c56a0112db31\") " pod="openshift-insights/insights-runtime-extractor-tg5qk"
Oct 31 17:27:18.481906 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.481901    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/72516605-a894-4a62-84c7-834122a01fb8-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-fhgqw\" (UID: \"72516605-a894-4a62-84c7-834122a01fb8\") " pod="openshift-multus/multus-additional-cni-plugins-fhgqw"
Oct 31 17:27:18.482587 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.481939    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/408bcc05-9690-4632-a479-de41718847ad-signing-cabundle\") pod \"service-ca-5fffc97bdf-ncfqp\" (UID: \"408bcc05-9690-4632-a479-de41718847ad\") " pod="openshift-service-ca/service-ca-5fffc97bdf-ncfqp"
Oct 31 17:27:18.482587 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.481958    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6d7781d6-8f73-4f1d-bb9a-0212e2eb6e4b-kubelet-config\") pod \"global-pull-secret-syncer-kmkdh\" (UID: \"6d7781d6-8f73-4f1d-bb9a-0212e2eb6e4b\") " pod="kube-system/global-pull-secret-syncer-kmkdh"
Oct 31 17:27:18.482587 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.481974    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/61462533-417a-4fbc-bd85-c56a0112db31-data-volume\") pod \"insights-runtime-extractor-tg5qk\" (UID: \"61462533-417a-4fbc-bd85-c56a0112db31\") " pod="openshift-insights/insights-runtime-extractor-tg5qk"
Oct 31 17:27:18.482587 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.481994    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/138bfcc8-f199-4ffd-b5d2-8d83ade6199a-metrics-certs\") pod \"network-metrics-daemon-pnjdm\" (UID: \"138bfcc8-f199-4ffd-b5d2-8d83ade6199a\") " pod="openshift-multus/network-metrics-daemon-pnjdm"
Oct 31 17:27:18.482587 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.482021    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/422b9ca9-3e0e-4fe0-860d-7c7d24b179fa-host\") pod \"tuned-rvsdp\" (UID: \"422b9ca9-3e0e-4fe0-860d-7c7d24b179fa\") " pod="openshift-cluster-node-tuning-operator/tuned-rvsdp"
Oct 31 17:27:18.482587 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.482042    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/422b9ca9-3e0e-4fe0-860d-7c7d24b179fa-etc-tuned\") pod \"tuned-rvsdp\" (UID: \"422b9ca9-3e0e-4fe0-860d-7c7d24b179fa\") " pod="openshift-cluster-node-tuning-operator/tuned-rvsdp"
Oct 31 17:27:18.482587 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.482084    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/2f646206-1186-4fcf-a9ca-fe92371d4d2b-agent-certs\") pod \"konnectivity-agent-mnbmt\" (UID: \"2f646206-1186-4fcf-a9ca-fe92371d4d2b\") " pod="kube-system/konnectivity-agent-mnbmt"
Oct 31 17:27:18.482587 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.482098    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5819fd64-9ac0-4436-b8fe-dcd3a27827ac-etc-openvswitch\") pod \"ovnkube-node-zpf4d\" (UID: \"5819fd64-9ac0-4436-b8fe-dcd3a27827ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpf4d"
Oct 31 17:27:18.482587 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.482092    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/61462533-417a-4fbc-bd85-c56a0112db31-data-volume\") pod \"insights-runtime-extractor-tg5qk\" (UID: \"61462533-417a-4fbc-bd85-c56a0112db31\") " pod="openshift-insights/insights-runtime-extractor-tg5qk"
Oct 31 17:27:18.482587 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.482128    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/422b9ca9-3e0e-4fe0-860d-7c7d24b179fa-etc-tuned\") pod \"tuned-rvsdp\" (UID: \"422b9ca9-3e0e-4fe0-860d-7c7d24b179fa\") " pod="openshift-cluster-node-tuning-operator/tuned-rvsdp"
Oct 31 17:27:18.482587 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.482142    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xxrgs\" (UniqueName: \"kubernetes.io/projected/72516605-a894-4a62-84c7-834122a01fb8-kube-api-access-xxrgs\") pod \"multus-additional-cni-plugins-fhgqw\" (UID: \"72516605-a894-4a62-84c7-834122a01fb8\") " pod="openshift-multus/multus-additional-cni-plugins-fhgqw"
Oct 31 17:27:18.482587 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.482159    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/5819fd64-9ac0-4436-b8fe-dcd3a27827ac-node-log\") pod \"ovnkube-node-zpf4d\" (UID: \"5819fd64-9ac0-4436-b8fe-dcd3a27827ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpf4d"
Oct 31 17:27:18.482587 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.482160    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/408bcc05-9690-4632-a479-de41718847ad-signing-cabundle\") pod \"service-ca-5fffc97bdf-ncfqp\" (UID: \"408bcc05-9690-4632-a479-de41718847ad\") " pod="openshift-service-ca/service-ca-5fffc97bdf-ncfqp"
Oct 31 17:27:18.482587 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.482160    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/72516605-a894-4a62-84c7-834122a01fb8-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-fhgqw\" (UID: \"72516605-a894-4a62-84c7-834122a01fb8\") " pod="openshift-multus/multus-additional-cni-plugins-fhgqw"
Oct 31 17:27:18.482587 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.482175    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/5819fd64-9ac0-4436-b8fe-dcd3a27827ac-ovnkube-config\") pod \"ovnkube-node-zpf4d\" (UID: \"5819fd64-9ac0-4436-b8fe-dcd3a27827ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpf4d"
Oct 31 17:27:18.482587 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.482205    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/5819fd64-9ac0-4436-b8fe-dcd3a27827ac-ovn-node-metrics-cert\") pod \"ovnkube-node-zpf4d\" (UID: \"5819fd64-9ac0-4436-b8fe-dcd3a27827ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpf4d"
Oct 31 17:27:18.482587 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.482266    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g2n2p\" (UniqueName: \"kubernetes.io/projected/5819fd64-9ac0-4436-b8fe-dcd3a27827ac-kube-api-access-g2n2p\") pod \"ovnkube-node-zpf4d\" (UID: \"5819fd64-9ac0-4436-b8fe-dcd3a27827ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpf4d"
Oct 31 17:27:18.482587 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.482284    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/87e61dcd-75f6-4d3f-8ed9-7d35b06a9bb3-host\") pod \"node-ca-qxnvq\" (UID: \"87e61dcd-75f6-4d3f-8ed9-7d35b06a9bb3\") " pod="openshift-image-registry/node-ca-qxnvq"
Oct 31 17:27:18.482587 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.482298    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/0f353547-2feb-42f4-aae4-b0f1a30efa34-socket-dir\") pod \"aws-ebs-csi-driver-node-2v874\" (UID: \"0f353547-2feb-42f4-aae4-b0f1a30efa34\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2v874"
Oct 31 17:27:18.482587 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.482312    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/0f353547-2feb-42f4-aae4-b0f1a30efa34-etc-selinux\") pod \"aws-ebs-csi-driver-node-2v874\" (UID: \"0f353547-2feb-42f4-aae4-b0f1a30efa34\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2v874"
Oct 31 17:27:18.482587 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.482326    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e-cni-binary-copy\") pod \"multus-rl4hl\" (UID: \"47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e\") " pod="openshift-multus/multus-rl4hl"
Oct 31 17:27:18.482587 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.482337    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/5819fd64-9ac0-4436-b8fe-dcd3a27827ac-ovnkube-config\") pod \"ovnkube-node-zpf4d\" (UID: \"5819fd64-9ac0-4436-b8fe-dcd3a27827ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpf4d"
Oct 31 17:27:18.482587 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.482341    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e-host-run-netns\") pod \"multus-rl4hl\" (UID: \"47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e\") " pod="openshift-multus/multus-rl4hl"
Oct 31 17:27:18.482587 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.482357    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e-etc-kubernetes\") pod \"multus-rl4hl\" (UID: \"47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e\") " pod="openshift-multus/multus-rl4hl"
Oct 31 17:27:18.482587 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.482381    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c6k68\" (UniqueName: \"kubernetes.io/projected/fb9c203a-af67-4df4-88dc-74cf9eab3623-kube-api-access-c6k68\") pod \"network-check-target-tskzw\" (UID: \"fb9c203a-af67-4df4-88dc-74cf9eab3623\") " pod="openshift-network-diagnostics/network-check-target-tskzw"
Oct 31 17:27:18.482587 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.482403    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/5819fd64-9ac0-4436-b8fe-dcd3a27827ac-host-cni-netd\") pod \"ovnkube-node-zpf4d\" (UID: \"5819fd64-9ac0-4436-b8fe-dcd3a27827ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpf4d"
Oct 31 17:27:18.482587 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.482431    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/5819fd64-9ac0-4436-b8fe-dcd3a27827ac-env-overrides\") pod \"ovnkube-node-zpf4d\" (UID: \"5819fd64-9ac0-4436-b8fe-dcd3a27827ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpf4d"
Oct 31 17:27:18.482587 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.482471    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/0f353547-2feb-42f4-aae4-b0f1a30efa34-kubelet-dir\") pod \"aws-ebs-csi-driver-node-2v874\" (UID: \"0f353547-2feb-42f4-aae4-b0f1a30efa34\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2v874"
Oct 31 17:27:18.482587 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.482502    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e-cni-binary-copy\") pod \"multus-rl4hl\" (UID: \"47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e\") " pod="openshift-multus/multus-rl4hl"
Oct 31 17:27:18.483564 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.482612    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/03a15842-25a2-4fbe-b418-8867070c4482-iptables-alerter-script\") pod \"iptables-alerter-jzcgs\" (UID: \"03a15842-25a2-4fbe-b418-8867070c4482\") " pod="openshift-network-operator/iptables-alerter-jzcgs"
Oct 31 17:27:18.483564 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.482641    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lc2r4\" (UniqueName: \"kubernetes.io/projected/03a15842-25a2-4fbe-b418-8867070c4482-kube-api-access-lc2r4\") pod \"iptables-alerter-jzcgs\" (UID: \"03a15842-25a2-4fbe-b418-8867070c4482\") " pod="openshift-network-operator/iptables-alerter-jzcgs"
Oct 31 17:27:18.483564 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.482640    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/5819fd64-9ac0-4436-b8fe-dcd3a27827ac-ovn-node-metrics-cert\") pod \"ovnkube-node-zpf4d\" (UID: \"5819fd64-9ac0-4436-b8fe-dcd3a27827ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpf4d"
Oct 31 17:27:18.483564 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.482699    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/5819fd64-9ac0-4436-b8fe-dcd3a27827ac-env-overrides\") pod \"ovnkube-node-zpf4d\" (UID: \"5819fd64-9ac0-4436-b8fe-dcd3a27827ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpf4d"
Oct 31 17:27:18.483564 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.482708    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-grrfv\" (UniqueName: \"kubernetes.io/projected/b8bbaf33-7cd9-4a30-86f7-6e20ac45b0a0-kube-api-access-grrfv\") pod \"node-resolver-7pfr4\" (UID: \"b8bbaf33-7cd9-4a30-86f7-6e20ac45b0a0\") " pod="openshift-dns/node-resolver-7pfr4"
Oct 31 17:27:18.483564 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.482787    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/22096c98-1b93-4ea4-9a6f-6383df9eafe4-cert\") pod \"ingress-canary-b5ln2\" (UID: \"22096c98-1b93-4ea4-9a6f-6383df9eafe4\") " pod="openshift-ingress-canary/ingress-canary-b5ln2"
Oct 31 17:27:18.483564 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.482819    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-49ggd\" (UniqueName: \"kubernetes.io/projected/22096c98-1b93-4ea4-9a6f-6383df9eafe4-kube-api-access-49ggd\") pod \"ingress-canary-b5ln2\" (UID: \"22096c98-1b93-4ea4-9a6f-6383df9eafe4\") " pod="openshift-ingress-canary/ingress-canary-b5ln2"
Oct 31 17:27:18.483564 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.482847    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/422b9ca9-3e0e-4fe0-860d-7c7d24b179fa-etc-modprobe-d\") pod \"tuned-rvsdp\" (UID: \"422b9ca9-3e0e-4fe0-860d-7c7d24b179fa\") " pod="openshift-cluster-node-tuning-operator/tuned-rvsdp"
Oct 31 17:27:18.483564 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.482852    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/03a15842-25a2-4fbe-b418-8867070c4482-iptables-alerter-script\") pod \"iptables-alerter-jzcgs\" (UID: \"03a15842-25a2-4fbe-b418-8867070c4482\") " pod="openshift-network-operator/iptables-alerter-jzcgs"
Oct 31 17:27:18.483564 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.482889    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/422b9ca9-3e0e-4fe0-860d-7c7d24b179fa-lib-modules\") pod \"tuned-rvsdp\" (UID: \"422b9ca9-3e0e-4fe0-860d-7c7d24b179fa\") " pod="openshift-cluster-node-tuning-operator/tuned-rvsdp"
Oct 31 17:27:18.483564 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.483041    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/22096c98-1b93-4ea4-9a6f-6383df9eafe4-cert\") pod \"ingress-canary-b5ln2\" (UID: \"22096c98-1b93-4ea4-9a6f-6383df9eafe4\") " pod="openshift-ingress-canary/ingress-canary-b5ln2"
Oct 31 17:27:18.498227 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.498200    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qfvft\" (UniqueName: \"kubernetes.io/projected/0f353547-2feb-42f4-aae4-b0f1a30efa34-kube-api-access-qfvft\") pod \"aws-ebs-csi-driver-node-2v874\" (UID: \"0f353547-2feb-42f4-aae4-b0f1a30efa34\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2v874"
Oct 31 17:27:18.498294 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.498262    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z8zhc\" (UniqueName: \"kubernetes.io/projected/9ee82620-6497-48e8-bb2c-1bb77a93f7ef-kube-api-access-z8zhc\") pod \"dns-default-59fpc\" (UID: \"9ee82620-6497-48e8-bb2c-1bb77a93f7ef\") " pod="openshift-dns/dns-default-59fpc"
Oct 31 17:27:18.498369 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.498349    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d6ck5\" (UniqueName: \"kubernetes.io/projected/138bfcc8-f199-4ffd-b5d2-8d83ade6199a-kube-api-access-d6ck5\") pod \"network-metrics-daemon-pnjdm\" (UID: \"138bfcc8-f199-4ffd-b5d2-8d83ade6199a\") " pod="openshift-multus/network-metrics-daemon-pnjdm"
Oct 31 17:27:18.498410 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.498393    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-28ft5\" (UniqueName: \"kubernetes.io/projected/47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e-kube-api-access-28ft5\") pod \"multus-rl4hl\" (UID: \"47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e\") " pod="openshift-multus/multus-rl4hl"
Oct 31 17:27:18.498996 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.498980    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5dkwn\" (UniqueName: \"kubernetes.io/projected/408bcc05-9690-4632-a479-de41718847ad-kube-api-access-5dkwn\") pod \"service-ca-5fffc97bdf-ncfqp\" (UID: \"408bcc05-9690-4632-a479-de41718847ad\") " pod="openshift-service-ca/service-ca-5fffc97bdf-ncfqp"
Oct 31 17:27:18.499032 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.499009    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p2gtl\" (UniqueName: \"kubernetes.io/projected/61462533-417a-4fbc-bd85-c56a0112db31-kube-api-access-p2gtl\") pod \"insights-runtime-extractor-tg5qk\" (UID: \"61462533-417a-4fbc-bd85-c56a0112db31\") " pod="openshift-insights/insights-runtime-extractor-tg5qk"
Oct 31 17:27:18.499576 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.499561    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-59c8z\" (UniqueName: \"kubernetes.io/projected/422b9ca9-3e0e-4fe0-860d-7c7d24b179fa-kube-api-access-59c8z\") pod \"tuned-rvsdp\" (UID: \"422b9ca9-3e0e-4fe0-860d-7c7d24b179fa\") " pod="openshift-cluster-node-tuning-operator/tuned-rvsdp"
Oct 31 17:27:18.500194 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.500173    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vv9nf\" (UniqueName: \"kubernetes.io/projected/87e61dcd-75f6-4d3f-8ed9-7d35b06a9bb3-kube-api-access-vv9nf\") pod \"node-ca-qxnvq\" (UID: \"87e61dcd-75f6-4d3f-8ed9-7d35b06a9bb3\") " pod="openshift-image-registry/node-ca-qxnvq"
Oct 31 17:27:18.500354 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.500337    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xxrgs\" (UniqueName: \"kubernetes.io/projected/72516605-a894-4a62-84c7-834122a01fb8-kube-api-access-xxrgs\") pod \"multus-additional-cni-plugins-fhgqw\" (UID: \"72516605-a894-4a62-84c7-834122a01fb8\") " pod="openshift-multus/multus-additional-cni-plugins-fhgqw"
Oct 31 17:27:18.500553 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.500538    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c6k68\" (UniqueName: \"kubernetes.io/projected/fb9c203a-af67-4df4-88dc-74cf9eab3623-kube-api-access-c6k68\") pod \"network-check-target-tskzw\" (UID: \"fb9c203a-af67-4df4-88dc-74cf9eab3623\") " pod="openshift-network-diagnostics/network-check-target-tskzw"
Oct 31 17:27:18.500594 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.500541    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lc2r4\" (UniqueName: \"kubernetes.io/projected/03a15842-25a2-4fbe-b418-8867070c4482-kube-api-access-lc2r4\") pod \"iptables-alerter-jzcgs\" (UID: \"03a15842-25a2-4fbe-b418-8867070c4482\") " pod="openshift-network-operator/iptables-alerter-jzcgs"
Oct 31 17:27:18.500640 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.500626    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g2n2p\" (UniqueName: \"kubernetes.io/projected/5819fd64-9ac0-4436-b8fe-dcd3a27827ac-kube-api-access-g2n2p\") pod \"ovnkube-node-zpf4d\" (UID: \"5819fd64-9ac0-4436-b8fe-dcd3a27827ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpf4d"
Oct 31 17:27:18.501488 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.501435    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-49ggd\" (UniqueName: \"kubernetes.io/projected/22096c98-1b93-4ea4-9a6f-6383df9eafe4-kube-api-access-49ggd\") pod \"ingress-canary-b5ln2\" (UID: \"22096c98-1b93-4ea4-9a6f-6383df9eafe4\") " pod="openshift-ingress-canary/ingress-canary-b5ln2"
Oct 31 17:27:18.501590 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.501576    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-grrfv\" (UniqueName: \"kubernetes.io/projected/b8bbaf33-7cd9-4a30-86f7-6e20ac45b0a0-kube-api-access-grrfv\") pod \"node-resolver-7pfr4\" (UID: \"b8bbaf33-7cd9-4a30-86f7-6e20ac45b0a0\") " pod="openshift-dns/node-resolver-7pfr4"
Oct 31 17:27:18.583622 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.583570    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/422b9ca9-3e0e-4fe0-860d-7c7d24b179fa-var-lib-kubelet\") pod \"tuned-rvsdp\" (UID: \"422b9ca9-3e0e-4fe0-860d-7c7d24b179fa\") " pod="openshift-cluster-node-tuning-operator/tuned-rvsdp"
Oct 31 17:27:18.583622 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.583594    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e-multus-cni-dir\") pod \"multus-rl4hl\" (UID: \"47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e\") " pod="openshift-multus/multus-rl4hl"
Oct 31 17:27:18.583622 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.583608    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b8bbaf33-7cd9-4a30-86f7-6e20ac45b0a0-hosts-file\") pod \"node-resolver-7pfr4\" (UID: \"b8bbaf33-7cd9-4a30-86f7-6e20ac45b0a0\") " pod="openshift-dns/node-resolver-7pfr4"
Oct 31 17:27:18.583743 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.583624    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/5819fd64-9ac0-4436-b8fe-dcd3a27827ac-run-systemd\") pod \"ovnkube-node-zpf4d\" (UID: \"5819fd64-9ac0-4436-b8fe-dcd3a27827ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpf4d"
Oct 31 17:27:18.583743 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.583675    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b8bbaf33-7cd9-4a30-86f7-6e20ac45b0a0-hosts-file\") pod \"node-resolver-7pfr4\" (UID: \"b8bbaf33-7cd9-4a30-86f7-6e20ac45b0a0\") " pod="openshift-dns/node-resolver-7pfr4"
Oct 31 17:27:18.583743 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.583694    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/422b9ca9-3e0e-4fe0-860d-7c7d24b179fa-var-lib-kubelet\") pod \"tuned-rvsdp\" (UID: \"422b9ca9-3e0e-4fe0-860d-7c7d24b179fa\") " pod="openshift-cluster-node-tuning-operator/tuned-rvsdp"
Oct 31 17:27:18.583743 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.583703    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/5819fd64-9ac0-4436-b8fe-dcd3a27827ac-run-systemd\") pod \"ovnkube-node-zpf4d\" (UID: \"5819fd64-9ac0-4436-b8fe-dcd3a27827ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpf4d"
Oct 31 17:27:18.583743 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.583694    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e-multus-cni-dir\") pod \"multus-rl4hl\" (UID: \"47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e\") " pod="openshift-multus/multus-rl4hl"
Oct 31 17:27:18.583743 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.583717    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/5819fd64-9ac0-4436-b8fe-dcd3a27827ac-log-socket\") pod \"ovnkube-node-zpf4d\" (UID: \"5819fd64-9ac0-4436-b8fe-dcd3a27827ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpf4d"
Oct 31 17:27:18.583743 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.583743    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e-multus-conf-dir\") pod \"multus-rl4hl\" (UID: \"47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e\") " pod="openshift-multus/multus-rl4hl"
Oct 31 17:27:18.584010 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.583758    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/61462533-417a-4fbc-bd85-c56a0112db31-crio-socket\") pod \"insights-runtime-extractor-tg5qk\" (UID: \"61462533-417a-4fbc-bd85-c56a0112db31\") " pod="openshift-insights/insights-runtime-extractor-tg5qk"
Oct 31 17:27:18.584010 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.583772    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6d7781d6-8f73-4f1d-bb9a-0212e2eb6e4b-kubelet-config\") pod \"global-pull-secret-syncer-kmkdh\" (UID: \"6d7781d6-8f73-4f1d-bb9a-0212e2eb6e4b\") " pod="kube-system/global-pull-secret-syncer-kmkdh"
Oct 31 17:27:18.584010 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.583811    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/61462533-417a-4fbc-bd85-c56a0112db31-crio-socket\") pod \"insights-runtime-extractor-tg5qk\" (UID: \"61462533-417a-4fbc-bd85-c56a0112db31\") " pod="openshift-insights/insights-runtime-extractor-tg5qk"
Oct 31 17:27:18.584010 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.583813    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e-multus-conf-dir\") pod \"multus-rl4hl\" (UID: \"47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e\") " pod="openshift-multus/multus-rl4hl"
Oct 31 17:27:18.584010 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.583824    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/5819fd64-9ac0-4436-b8fe-dcd3a27827ac-log-socket\") pod \"ovnkube-node-zpf4d\" (UID: \"5819fd64-9ac0-4436-b8fe-dcd3a27827ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpf4d"
Oct 31 17:27:18.584010 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.583816    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6d7781d6-8f73-4f1d-bb9a-0212e2eb6e4b-kubelet-config\") pod \"global-pull-secret-syncer-kmkdh\" (UID: \"6d7781d6-8f73-4f1d-bb9a-0212e2eb6e4b\") " pod="kube-system/global-pull-secret-syncer-kmkdh"
Oct 31 17:27:18.584010 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.583836    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/138bfcc8-f199-4ffd-b5d2-8d83ade6199a-metrics-certs\") pod \"network-metrics-daemon-pnjdm\" (UID: \"138bfcc8-f199-4ffd-b5d2-8d83ade6199a\") " pod="openshift-multus/network-metrics-daemon-pnjdm"
Oct 31 17:27:18.584010 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.583855    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/422b9ca9-3e0e-4fe0-860d-7c7d24b179fa-host\") pod \"tuned-rvsdp\" (UID: \"422b9ca9-3e0e-4fe0-860d-7c7d24b179fa\") " pod="openshift-cluster-node-tuning-operator/tuned-rvsdp"
Oct 31 17:27:18.584010 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.583869    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5819fd64-9ac0-4436-b8fe-dcd3a27827ac-etc-openvswitch\") pod \"ovnkube-node-zpf4d\" (UID: \"5819fd64-9ac0-4436-b8fe-dcd3a27827ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpf4d"
Oct 31 17:27:18.584010 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.583884    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/5819fd64-9ac0-4436-b8fe-dcd3a27827ac-node-log\") pod \"ovnkube-node-zpf4d\" (UID: \"5819fd64-9ac0-4436-b8fe-dcd3a27827ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpf4d"
Oct 31 17:27:18.584010 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.583902    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/87e61dcd-75f6-4d3f-8ed9-7d35b06a9bb3-host\") pod \"node-ca-qxnvq\" (UID: \"87e61dcd-75f6-4d3f-8ed9-7d35b06a9bb3\") " pod="openshift-image-registry/node-ca-qxnvq"
Oct 31 17:27:18.584010 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.583925    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/0f353547-2feb-42f4-aae4-b0f1a30efa34-socket-dir\") pod \"aws-ebs-csi-driver-node-2v874\" (UID: \"0f353547-2feb-42f4-aae4-b0f1a30efa34\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2v874"
Oct 31 17:27:18.584010 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.583939    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/5819fd64-9ac0-4436-b8fe-dcd3a27827ac-node-log\") pod \"ovnkube-node-zpf4d\" (UID: \"5819fd64-9ac0-4436-b8fe-dcd3a27827ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpf4d"
Oct 31 17:27:18.584010 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.583949    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/87e61dcd-75f6-4d3f-8ed9-7d35b06a9bb3-host\") pod \"node-ca-qxnvq\" (UID: \"87e61dcd-75f6-4d3f-8ed9-7d35b06a9bb3\") " pod="openshift-image-registry/node-ca-qxnvq"
Oct 31 17:27:18.584010 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.583954    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/0f353547-2feb-42f4-aae4-b0f1a30efa34-etc-selinux\") pod \"aws-ebs-csi-driver-node-2v874\" (UID: \"0f353547-2feb-42f4-aae4-b0f1a30efa34\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2v874"
Oct 31 17:27:18.584010 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.583931    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5819fd64-9ac0-4436-b8fe-dcd3a27827ac-etc-openvswitch\") pod \"ovnkube-node-zpf4d\" (UID: \"5819fd64-9ac0-4436-b8fe-dcd3a27827ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpf4d"
Oct 31 17:27:18.584010 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.583972    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e-host-run-netns\") pod \"multus-rl4hl\" (UID: \"47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e\") " pod="openshift-multus/multus-rl4hl"
Oct 31 17:27:18.584010 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.583977    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/422b9ca9-3e0e-4fe0-860d-7c7d24b179fa-host\") pod \"tuned-rvsdp\" (UID: \"422b9ca9-3e0e-4fe0-860d-7c7d24b179fa\") " pod="openshift-cluster-node-tuning-operator/tuned-rvsdp"
Oct 31 17:27:18.584010 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.583986    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e-etc-kubernetes\") pod \"multus-rl4hl\" (UID: \"47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e\") " pod="openshift-multus/multus-rl4hl"
Oct 31 17:27:18.584010 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.583992    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/0f353547-2feb-42f4-aae4-b0f1a30efa34-socket-dir\") pod \"aws-ebs-csi-driver-node-2v874\" (UID: \"0f353547-2feb-42f4-aae4-b0f1a30efa34\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2v874"
Oct 31 17:27:18.584010 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.584018    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/5819fd64-9ac0-4436-b8fe-dcd3a27827ac-host-cni-netd\") pod \"ovnkube-node-zpf4d\" (UID: \"5819fd64-9ac0-4436-b8fe-dcd3a27827ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpf4d"
Oct 31 17:27:18.584010 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.584023    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e-etc-kubernetes\") pod \"multus-rl4hl\" (UID: \"47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e\") " pod="openshift-multus/multus-rl4hl"
Oct 31 17:27:18.584904 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.584036    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e-host-run-netns\") pod \"multus-rl4hl\" (UID: \"47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e\") " pod="openshift-multus/multus-rl4hl"
Oct 31 17:27:18.584904 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.584040    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/0f353547-2feb-42f4-aae4-b0f1a30efa34-kubelet-dir\") pod \"aws-ebs-csi-driver-node-2v874\" (UID: \"0f353547-2feb-42f4-aae4-b0f1a30efa34\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2v874"
Oct 31 17:27:18.584904 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.584041    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/0f353547-2feb-42f4-aae4-b0f1a30efa34-etc-selinux\") pod \"aws-ebs-csi-driver-node-2v874\" (UID: \"0f353547-2feb-42f4-aae4-b0f1a30efa34\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2v874"
Oct 31 17:27:18.584904 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.584052    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/5819fd64-9ac0-4436-b8fe-dcd3a27827ac-host-cni-netd\") pod \"ovnkube-node-zpf4d\" (UID: \"5819fd64-9ac0-4436-b8fe-dcd3a27827ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpf4d"
Oct 31 17:27:18.584904 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.584065    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/0f353547-2feb-42f4-aae4-b0f1a30efa34-kubelet-dir\") pod \"aws-ebs-csi-driver-node-2v874\" (UID: \"0f353547-2feb-42f4-aae4-b0f1a30efa34\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2v874"
Oct 31 17:27:18.584904 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.584071    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/422b9ca9-3e0e-4fe0-860d-7c7d24b179fa-etc-modprobe-d\") pod \"tuned-rvsdp\" (UID: \"422b9ca9-3e0e-4fe0-860d-7c7d24b179fa\") " pod="openshift-cluster-node-tuning-operator/tuned-rvsdp"
Oct 31 17:27:18.584904 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.584090    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/422b9ca9-3e0e-4fe0-860d-7c7d24b179fa-lib-modules\") pod \"tuned-rvsdp\" (UID: \"422b9ca9-3e0e-4fe0-860d-7c7d24b179fa\") " pod="openshift-cluster-node-tuning-operator/tuned-rvsdp"
Oct 31 17:27:18.584904 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.584111    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5819fd64-9ac0-4436-b8fe-dcd3a27827ac-run-openvswitch\") pod \"ovnkube-node-zpf4d\" (UID: \"5819fd64-9ac0-4436-b8fe-dcd3a27827ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpf4d"
Oct 31 17:27:18.584904 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.584132    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/72516605-a894-4a62-84c7-834122a01fb8-tuning-conf-dir\") pod \"multus-additional-cni-plugins-fhgqw\" (UID: \"72516605-a894-4a62-84c7-834122a01fb8\") " pod="openshift-multus/multus-additional-cni-plugins-fhgqw"
Oct 31 17:27:18.584904 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.584153    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/72516605-a894-4a62-84c7-834122a01fb8-system-cni-dir\") pod \"multus-additional-cni-plugins-fhgqw\" (UID: \"72516605-a894-4a62-84c7-834122a01fb8\") " pod="openshift-multus/multus-additional-cni-plugins-fhgqw"
Oct 31 17:27:18.584904 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.584171    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5819fd64-9ac0-4436-b8fe-dcd3a27827ac-run-openvswitch\") pod \"ovnkube-node-zpf4d\" (UID: \"5819fd64-9ac0-4436-b8fe-dcd3a27827ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpf4d"
Oct 31 17:27:18.584904 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.584176    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/61462533-417a-4fbc-bd85-c56a0112db31-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tg5qk\" (UID: \"61462533-417a-4fbc-bd85-c56a0112db31\") " pod="openshift-insights/insights-runtime-extractor-tg5qk"
Oct 31 17:27:18.584904 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.584195    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/72516605-a894-4a62-84c7-834122a01fb8-tuning-conf-dir\") pod \"multus-additional-cni-plugins-fhgqw\" (UID: \"72516605-a894-4a62-84c7-834122a01fb8\") " pod="openshift-multus/multus-additional-cni-plugins-fhgqw"
Oct 31 17:27:18.584904 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.584205    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/422b9ca9-3e0e-4fe0-860d-7c7d24b179fa-etc-sysctl-d\") pod \"tuned-rvsdp\" (UID: \"422b9ca9-3e0e-4fe0-860d-7c7d24b179fa\") " pod="openshift-cluster-node-tuning-operator/tuned-rvsdp"
Oct 31 17:27:18.584904 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.584211    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/72516605-a894-4a62-84c7-834122a01fb8-system-cni-dir\") pod \"multus-additional-cni-plugins-fhgqw\" (UID: \"72516605-a894-4a62-84c7-834122a01fb8\") " pod="openshift-multus/multus-additional-cni-plugins-fhgqw"
Oct 31 17:27:18.584904 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.584211    4276 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Oct 31 17:27:18.584904 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.584228    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/5819fd64-9ac0-4436-b8fe-dcd3a27827ac-host-kubelet\") pod \"ovnkube-node-zpf4d\" (UID: \"5819fd64-9ac0-4436-b8fe-dcd3a27827ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpf4d"
Oct 31 17:27:18.584904 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.584250    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/5819fd64-9ac0-4436-b8fe-dcd3a27827ac-systemd-units\") pod \"ovnkube-node-zpf4d\" (UID: \"5819fd64-9ac0-4436-b8fe-dcd3a27827ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpf4d"
Oct 31 17:27:18.584904 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.584271    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/5819fd64-9ac0-4436-b8fe-dcd3a27827ac-host-kubelet\") pod \"ovnkube-node-zpf4d\" (UID: \"5819fd64-9ac0-4436-b8fe-dcd3a27827ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpf4d"
Oct 31 17:27:18.584904 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.584275    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/72516605-a894-4a62-84c7-834122a01fb8-cnibin\") pod \"multus-additional-cni-plugins-fhgqw\" (UID: \"72516605-a894-4a62-84c7-834122a01fb8\") " pod="openshift-multus/multus-additional-cni-plugins-fhgqw"
Oct 31 17:27:18.584904 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.584250    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/422b9ca9-3e0e-4fe0-860d-7c7d24b179fa-lib-modules\") pod \"tuned-rvsdp\" (UID: \"422b9ca9-3e0e-4fe0-860d-7c7d24b179fa\") " pod="openshift-cluster-node-tuning-operator/tuned-rvsdp"
Oct 31 17:27:18.584904 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.584196    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/422b9ca9-3e0e-4fe0-860d-7c7d24b179fa-etc-modprobe-d\") pod \"tuned-rvsdp\" (UID: \"422b9ca9-3e0e-4fe0-860d-7c7d24b179fa\") " pod="openshift-cluster-node-tuning-operator/tuned-rvsdp"
Oct 31 17:27:18.584904 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.584300    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e-system-cni-dir\") pod \"multus-rl4hl\" (UID: \"47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e\") " pod="openshift-multus/multus-rl4hl"
Oct 31 17:27:18.584904 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.584307    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/72516605-a894-4a62-84c7-834122a01fb8-cnibin\") pod \"multus-additional-cni-plugins-fhgqw\" (UID: \"72516605-a894-4a62-84c7-834122a01fb8\") " pod="openshift-multus/multus-additional-cni-plugins-fhgqw"
Oct 31 17:27:18.584904 ip-10-0-129-127 kubenswrapper[4276]: E1031 17:27:18.584274    4276 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 31 17:27:18.584904 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.584276    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/422b9ca9-3e0e-4fe0-860d-7c7d24b179fa-etc-sysctl-d\") pod \"tuned-rvsdp\" (UID: \"422b9ca9-3e0e-4fe0-860d-7c7d24b179fa\") " pod="openshift-cluster-node-tuning-operator/tuned-rvsdp"
Oct 31 17:27:18.584904 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.584320    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e-host-var-lib-cni-multus\") pod \"multus-rl4hl\" (UID: \"47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e\") " pod="openshift-multus/multus-rl4hl"
Oct 31 17:27:18.584904 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.584347    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e-system-cni-dir\") pod \"multus-rl4hl\" (UID: \"47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e\") " pod="openshift-multus/multus-rl4hl"
Oct 31 17:27:18.584904 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.584355    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/422b9ca9-3e0e-4fe0-860d-7c7d24b179fa-etc-sysconfig\") pod \"tuned-rvsdp\" (UID: \"422b9ca9-3e0e-4fe0-860d-7c7d24b179fa\") " pod="openshift-cluster-node-tuning-operator/tuned-rvsdp"
Oct 31 17:27:18.584904 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.584307    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/5819fd64-9ac0-4436-b8fe-dcd3a27827ac-systemd-units\") pod \"ovnkube-node-zpf4d\" (UID: \"5819fd64-9ac0-4436-b8fe-dcd3a27827ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpf4d"
Oct 31 17:27:18.584904 ip-10-0-129-127 kubenswrapper[4276]: E1031 17:27:18.584383    4276 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/61462533-417a-4fbc-bd85-c56a0112db31-insights-runtime-extractor-tls podName:61462533-417a-4fbc-bd85-c56a0112db31 nodeName:}" failed. No retries permitted until 2026-10-31 17:27:19.084365437 +0000 UTC m=+1.882110366 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/61462533-417a-4fbc-bd85-c56a0112db31-insights-runtime-extractor-tls") pod "insights-runtime-extractor-tg5qk" (UID: "61462533-417a-4fbc-bd85-c56a0112db31") : secret "insights-runtime-extractor-tls" not found
Oct 31 17:27:18.584904 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.584380    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e-host-var-lib-cni-multus\") pod \"multus-rl4hl\" (UID: \"47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e\") " pod="openshift-multus/multus-rl4hl"
Oct 31 17:27:18.584904 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.584401    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/422b9ca9-3e0e-4fe0-860d-7c7d24b179fa-etc-sysconfig\") pod \"tuned-rvsdp\" (UID: \"422b9ca9-3e0e-4fe0-860d-7c7d24b179fa\") " pod="openshift-cluster-node-tuning-operator/tuned-rvsdp"
Oct 31 17:27:18.584904 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.584415    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e-cnibin\") pod \"multus-rl4hl\" (UID: \"47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e\") " pod="openshift-multus/multus-rl4hl"
Oct 31 17:27:18.584904 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.584437    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e-multus-socket-dir-parent\") pod \"multus-rl4hl\" (UID: \"47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e\") " pod="openshift-multus/multus-rl4hl"
Oct 31 17:27:18.584904 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.584483    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e-host-run-k8s-cni-cncf-io\") pod \"multus-rl4hl\" (UID: \"47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e\") " pod="openshift-multus/multus-rl4hl"
Oct 31 17:27:18.584904 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.584492    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e-multus-socket-dir-parent\") pod \"multus-rl4hl\" (UID: \"47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e\") " pod="openshift-multus/multus-rl4hl"
Oct 31 17:27:18.584904 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.584499    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e-cnibin\") pod \"multus-rl4hl\" (UID: \"47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e\") " pod="openshift-multus/multus-rl4hl"
Oct 31 17:27:18.584904 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.584505    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e-host-var-lib-cni-bin\") pod \"multus-rl4hl\" (UID: \"47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e\") " pod="openshift-multus/multus-rl4hl"
Oct 31 17:27:18.584904 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.584521    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e-host-run-k8s-cni-cncf-io\") pod \"multus-rl4hl\" (UID: \"47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e\") " pod="openshift-multus/multus-rl4hl"
Oct 31 17:27:18.584904 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.584529    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5819fd64-9ac0-4436-b8fe-dcd3a27827ac-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-zpf4d\" (UID: \"5819fd64-9ac0-4436-b8fe-dcd3a27827ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpf4d"
Oct 31 17:27:18.584904 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.584562    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5819fd64-9ac0-4436-b8fe-dcd3a27827ac-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-zpf4d\" (UID: \"5819fd64-9ac0-4436-b8fe-dcd3a27827ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpf4d"
Oct 31 17:27:18.584904 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.584570    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e-host-var-lib-cni-bin\") pod \"multus-rl4hl\" (UID: \"47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e\") " pod="openshift-multus/multus-rl4hl"
Oct 31 17:27:18.584904 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.584578    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/0f353547-2feb-42f4-aae4-b0f1a30efa34-sys-fs\") pod \"aws-ebs-csi-driver-node-2v874\" (UID: \"0f353547-2feb-42f4-aae4-b0f1a30efa34\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2v874"
Oct 31 17:27:18.584904 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.584601    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e-host-run-multus-certs\") pod \"multus-rl4hl\" (UID: \"47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e\") " pod="openshift-multus/multus-rl4hl"
Oct 31 17:27:18.584904 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.584623    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e-host-var-lib-kubelet\") pod \"multus-rl4hl\" (UID: \"47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e\") " pod="openshift-multus/multus-rl4hl"
Oct 31 17:27:18.584904 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.584646    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e-hostroot\") pod \"multus-rl4hl\" (UID: \"47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e\") " pod="openshift-multus/multus-rl4hl"
Oct 31 17:27:18.584904 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.584670    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/422b9ca9-3e0e-4fe0-860d-7c7d24b179fa-etc-kubernetes\") pod \"tuned-rvsdp\" (UID: \"422b9ca9-3e0e-4fe0-860d-7c7d24b179fa\") " pod="openshift-cluster-node-tuning-operator/tuned-rvsdp"
Oct 31 17:27:18.584904 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.584679    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e-host-run-multus-certs\") pod \"multus-rl4hl\" (UID: \"47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e\") " pod="openshift-multus/multus-rl4hl"
Oct 31 17:27:18.584904 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.584685    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/0f353547-2feb-42f4-aae4-b0f1a30efa34-sys-fs\") pod \"aws-ebs-csi-driver-node-2v874\" (UID: \"0f353547-2feb-42f4-aae4-b0f1a30efa34\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2v874"
Oct 31 17:27:18.584904 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.584673    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e-host-var-lib-kubelet\") pod \"multus-rl4hl\" (UID: \"47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e\") " pod="openshift-multus/multus-rl4hl"
Oct 31 17:27:18.584904 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.584681    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e-hostroot\") pod \"multus-rl4hl\" (UID: \"47e8b7f2-09d2-4ac3-90c4-d0cee4f3e58e\") " pod="openshift-multus/multus-rl4hl"
Oct 31 17:27:18.584904 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.584689    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/422b9ca9-3e0e-4fe0-860d-7c7d24b179fa-run\") pod \"tuned-rvsdp\" (UID: \"422b9ca9-3e0e-4fe0-860d-7c7d24b179fa\") " pod="openshift-cluster-node-tuning-operator/tuned-rvsdp"
Oct 31 17:27:18.584904 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.584716    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/422b9ca9-3e0e-4fe0-860d-7c7d24b179fa-etc-kubernetes\") pod \"tuned-rvsdp\" (UID: \"422b9ca9-3e0e-4fe0-860d-7c7d24b179fa\") " pod="openshift-cluster-node-tuning-operator/tuned-rvsdp"
Oct 31 17:27:18.584904 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.584730    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/422b9ca9-3e0e-4fe0-860d-7c7d24b179fa-run\") pod \"tuned-rvsdp\" (UID: \"422b9ca9-3e0e-4fe0-860d-7c7d24b179fa\") " pod="openshift-cluster-node-tuning-operator/tuned-rvsdp"
Oct 31 17:27:18.584904 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.584735    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6d7781d6-8f73-4f1d-bb9a-0212e2eb6e4b-dbus\") pod \"global-pull-secret-syncer-kmkdh\" (UID: \"6d7781d6-8f73-4f1d-bb9a-0212e2eb6e4b\") " pod="kube-system/global-pull-secret-syncer-kmkdh"
Oct 31 17:27:18.584904 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.584754    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/422b9ca9-3e0e-4fe0-860d-7c7d24b179fa-sys\") pod \"tuned-rvsdp\" (UID: \"422b9ca9-3e0e-4fe0-860d-7c7d24b179fa\") " pod="openshift-cluster-node-tuning-operator/tuned-rvsdp"
Oct 31 17:27:18.584904 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.584770    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5819fd64-9ac0-4436-b8fe-dcd3a27827ac-host-slash\") pod \"ovnkube-node-zpf4d\" (UID: \"5819fd64-9ac0-4436-b8fe-dcd3a27827ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpf4d"
Oct 31 17:27:18.584904 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.584788    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5819fd64-9ac0-4436-b8fe-dcd3a27827ac-host-run-netns\") pod \"ovnkube-node-zpf4d\" (UID: \"5819fd64-9ac0-4436-b8fe-dcd3a27827ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpf4d"
Oct 31 17:27:18.584904 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.584772    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6d7781d6-8f73-4f1d-bb9a-0212e2eb6e4b-dbus\") pod \"global-pull-secret-syncer-kmkdh\" (UID: \"6d7781d6-8f73-4f1d-bb9a-0212e2eb6e4b\") " pod="kube-system/global-pull-secret-syncer-kmkdh"
Oct 31 17:27:18.584904 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.584813    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5819fd64-9ac0-4436-b8fe-dcd3a27827ac-host-cni-bin\") pod \"ovnkube-node-zpf4d\" (UID: \"5819fd64-9ac0-4436-b8fe-dcd3a27827ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpf4d"
Oct 31 17:27:18.584904 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.584817    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5819fd64-9ac0-4436-b8fe-dcd3a27827ac-host-slash\") pod \"ovnkube-node-zpf4d\" (UID: \"5819fd64-9ac0-4436-b8fe-dcd3a27827ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpf4d"
Oct 31 17:27:18.584904 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.584829    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5819fd64-9ac0-4436-b8fe-dcd3a27827ac-host-run-netns\") pod \"ovnkube-node-zpf4d\" (UID: \"5819fd64-9ac0-4436-b8fe-dcd3a27827ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpf4d"
Oct 31 17:27:18.584904 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.584802    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/422b9ca9-3e0e-4fe0-860d-7c7d24b179fa-sys\") pod \"tuned-rvsdp\" (UID: \"422b9ca9-3e0e-4fe0-860d-7c7d24b179fa\") " pod="openshift-cluster-node-tuning-operator/tuned-rvsdp"
Oct 31 17:27:18.584904 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.584839    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/422b9ca9-3e0e-4fe0-860d-7c7d24b179fa-etc-sysctl-conf\") pod \"tuned-rvsdp\" (UID: \"422b9ca9-3e0e-4fe0-860d-7c7d24b179fa\") " pod="openshift-cluster-node-tuning-operator/tuned-rvsdp"
Oct 31 17:27:18.584904 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.584858    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5819fd64-9ac0-4436-b8fe-dcd3a27827ac-host-cni-bin\") pod \"ovnkube-node-zpf4d\" (UID: \"5819fd64-9ac0-4436-b8fe-dcd3a27827ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpf4d"
Oct 31 17:27:18.584904 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.584866    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/422b9ca9-3e0e-4fe0-860d-7c7d24b179fa-etc-systemd\") pod \"tuned-rvsdp\" (UID: \"422b9ca9-3e0e-4fe0-860d-7c7d24b179fa\") " pod="openshift-cluster-node-tuning-operator/tuned-rvsdp"
Oct 31 17:27:18.584904 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.584885    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5819fd64-9ac0-4436-b8fe-dcd3a27827ac-var-lib-openvswitch\") pod \"ovnkube-node-zpf4d\" (UID: \"5819fd64-9ac0-4436-b8fe-dcd3a27827ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpf4d"
Oct 31 17:27:18.584904 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.584898    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/5819fd64-9ac0-4436-b8fe-dcd3a27827ac-run-ovn\") pod \"ovnkube-node-zpf4d\" (UID: \"5819fd64-9ac0-4436-b8fe-dcd3a27827ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpf4d"
Oct 31 17:27:18.584904 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.584911    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/0f353547-2feb-42f4-aae4-b0f1a30efa34-registration-dir\") pod \"aws-ebs-csi-driver-node-2v874\" (UID: \"0f353547-2feb-42f4-aae4-b0f1a30efa34\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2v874"
Oct 31 17:27:18.584904 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.584920    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/422b9ca9-3e0e-4fe0-860d-7c7d24b179fa-etc-systemd\") pod \"tuned-rvsdp\" (UID: \"422b9ca9-3e0e-4fe0-860d-7c7d24b179fa\") " pod="openshift-cluster-node-tuning-operator/tuned-rvsdp"
Oct 31 17:27:18.584904 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.584932    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/03a15842-25a2-4fbe-b418-8867070c4482-host-slash\") pod \"iptables-alerter-jzcgs\" (UID: \"03a15842-25a2-4fbe-b418-8867070c4482\") " pod="openshift-network-operator/iptables-alerter-jzcgs"
Oct 31 17:27:18.584904 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.584944    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5819fd64-9ac0-4436-b8fe-dcd3a27827ac-var-lib-openvswitch\") pod \"ovnkube-node-zpf4d\" (UID: \"5819fd64-9ac0-4436-b8fe-dcd3a27827ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpf4d"
Oct 31 17:27:18.584904 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.584953    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/5819fd64-9ac0-4436-b8fe-dcd3a27827ac-run-ovn\") pod \"ovnkube-node-zpf4d\" (UID: \"5819fd64-9ac0-4436-b8fe-dcd3a27827ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpf4d"
Oct 31 17:27:18.584904 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.584957    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5819fd64-9ac0-4436-b8fe-dcd3a27827ac-host-run-ovn-kubernetes\") pod \"ovnkube-node-zpf4d\" (UID: \"5819fd64-9ac0-4436-b8fe-dcd3a27827ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpf4d"
Oct 31 17:27:18.584904 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.584959    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/422b9ca9-3e0e-4fe0-860d-7c7d24b179fa-etc-sysctl-conf\") pod \"tuned-rvsdp\" (UID: \"422b9ca9-3e0e-4fe0-860d-7c7d24b179fa\") " pod="openshift-cluster-node-tuning-operator/tuned-rvsdp"
Oct 31 17:27:18.586984 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.584984    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5819fd64-9ac0-4436-b8fe-dcd3a27827ac-host-run-ovn-kubernetes\") pod \"ovnkube-node-zpf4d\" (UID: \"5819fd64-9ac0-4436-b8fe-dcd3a27827ac\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpf4d"
Oct 31 17:27:18.586984 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.584989    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/0f353547-2feb-42f4-aae4-b0f1a30efa34-device-dir\") pod \"aws-ebs-csi-driver-node-2v874\" (UID: \"0f353547-2feb-42f4-aae4-b0f1a30efa34\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2v874"
Oct 31 17:27:18.586984 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.584993    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/0f353547-2feb-42f4-aae4-b0f1a30efa34-registration-dir\") pod \"aws-ebs-csi-driver-node-2v874\" (UID: \"0f353547-2feb-42f4-aae4-b0f1a30efa34\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2v874"
Oct 31 17:27:18.586984 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.584996    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/03a15842-25a2-4fbe-b418-8867070c4482-host-slash\") pod \"iptables-alerter-jzcgs\" (UID: \"03a15842-25a2-4fbe-b418-8867070c4482\") " pod="openshift-network-operator/iptables-alerter-jzcgs"
Oct 31 17:27:18.586984 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.585027    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/0f353547-2feb-42f4-aae4-b0f1a30efa34-device-dir\") pod \"aws-ebs-csi-driver-node-2v874\" (UID: \"0f353547-2feb-42f4-aae4-b0f1a30efa34\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2v874"
Oct 31 17:27:18.587298 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.587281    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/138bfcc8-f199-4ffd-b5d2-8d83ade6199a-metrics-certs\") pod \"network-metrics-daemon-pnjdm\" (UID: \"138bfcc8-f199-4ffd-b5d2-8d83ade6199a\") " pod="openshift-multus/network-metrics-daemon-pnjdm"
Oct 31 17:27:18.661829 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.661801    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pnjdm"
Oct 31 17:27:18.663146 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.663055    4276 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 31 17:27:18.999166 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:18.998880    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-pnjdm"]
Oct 31 17:27:19.002277 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:19.002247    4276 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod138bfcc8_f199_4ffd_b5d2_8d83ade6199a.slice/crio-2f57805c9939ecc5ba3caaf49e1ecd29910ad65ffb39cbe5cd80303ab8b3a842 WatchSource:0}: Error finding container 2f57805c9939ecc5ba3caaf49e1ecd29910ad65ffb39cbe5cd80303ab8b3a842: Status 404 returned error can't find the container with id 2f57805c9939ecc5ba3caaf49e1ecd29910ad65ffb39cbe5cd80303ab8b3a842
Oct 31 17:27:19.090494 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:19.090470    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/61462533-417a-4fbc-bd85-c56a0112db31-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tg5qk\" (UID: \"61462533-417a-4fbc-bd85-c56a0112db31\") " pod="openshift-insights/insights-runtime-extractor-tg5qk"
Oct 31 17:27:19.090613 ip-10-0-129-127 kubenswrapper[4276]: E1031 17:27:19.090598    4276 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 31 17:27:19.090654 ip-10-0-129-127 kubenswrapper[4276]: E1031 17:27:19.090645    4276 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/61462533-417a-4fbc-bd85-c56a0112db31-insights-runtime-extractor-tls podName:61462533-417a-4fbc-bd85-c56a0112db31 nodeName:}" failed. No retries permitted until 2026-10-31 17:27:20.090631501 +0000 UTC m=+2.888376407 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/61462533-417a-4fbc-bd85-c56a0112db31-insights-runtime-extractor-tls") pod "insights-runtime-extractor-tg5qk" (UID: "61462533-417a-4fbc-bd85-c56a0112db31") : secret "insights-runtime-extractor-tls" not found
Oct 31 17:27:19.401884 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:19.401854    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-59fpc" event={"ID":"9ee82620-6497-48e8-bb2c-1bb77a93f7ef","Type":"ContainerStarted","Data":"f99c44cad303b7854e401c1b831d0c7b1d0ac8c10017936c5cef5f316a377c61"}
Oct 31 17:27:19.403174 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:19.403147    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-pnjdm" event={"ID":"138bfcc8-f199-4ffd-b5d2-8d83ade6199a","Type":"ContainerStarted","Data":"2f57805c9939ecc5ba3caaf49e1ecd29910ad65ffb39cbe5cd80303ab8b3a842"}
Oct 31 17:27:19.544235 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:19.544138    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:27:19.671466 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:19.671422    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-mnbmt"
Oct 31 17:27:19.736281 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:19.736243    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:27:19.948692 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:19.948654    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:27:20.098274 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:20.098238    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/61462533-417a-4fbc-bd85-c56a0112db31-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tg5qk\" (UID: \"61462533-417a-4fbc-bd85-c56a0112db31\") " pod="openshift-insights/insights-runtime-extractor-tg5qk"
Oct 31 17:27:20.098428 ip-10-0-129-127 kubenswrapper[4276]: E1031 17:27:20.098398    4276 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 31 17:27:20.098512 ip-10-0-129-127 kubenswrapper[4276]: E1031 17:27:20.098490    4276 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/61462533-417a-4fbc-bd85-c56a0112db31-insights-runtime-extractor-tls podName:61462533-417a-4fbc-bd85-c56a0112db31 nodeName:}" failed. No retries permitted until 2026-10-31 17:27:22.098469406 +0000 UTC m=+4.896214329 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/61462533-417a-4fbc-bd85-c56a0112db31-insights-runtime-extractor-tls") pod "insights-runtime-extractor-tg5qk" (UID: "61462533-417a-4fbc-bd85-c56a0112db31") : secret "insights-runtime-extractor-tls" not found
Oct 31 17:27:20.142243 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:20.142201    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:27:20.338485 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:20.338430    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:27:20.409996 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:20.409968    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-59fpc" event={"ID":"9ee82620-6497-48e8-bb2c-1bb77a93f7ef","Type":"ContainerStarted","Data":"41d5fefe020ef0b2b1175eb9c2d3255817b46e0029678eae30d74d32540dca2c"}
Oct 31 17:27:21.273825 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:21.273601    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:27:21.386818 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:21.386790    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:27:21.415386 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:21.415364    4276 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 31 17:27:21.415750 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:21.415381    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-pnjdm" event={"ID":"138bfcc8-f199-4ffd-b5d2-8d83ade6199a","Type":"ContainerStarted","Data":"9af0108fabd724c33281022c0e5c206316248f8a2260e622ccf5de342a0c4a8b"}
Oct 31 17:27:21.415750 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:21.415434    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-pnjdm" event={"ID":"138bfcc8-f199-4ffd-b5d2-8d83ade6199a","Type":"ContainerStarted","Data":"b76229105cf69ca40a226c961a5c7f35715e500f8bdc291c19cdc608ae6cbe50"}
Oct 31 17:27:21.575954 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:21.575921    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-59fpc"
Oct 31 17:27:21.744846 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:21.744811    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:27:21.819287 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:21.819265    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-zpf4d"
Oct 31 17:27:21.950655 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:21.950588    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:27:22.112497 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:22.112470    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/61462533-417a-4fbc-bd85-c56a0112db31-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tg5qk\" (UID: \"61462533-417a-4fbc-bd85-c56a0112db31\") " pod="openshift-insights/insights-runtime-extractor-tg5qk"
Oct 31 17:27:22.116290 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:22.116259    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/61462533-417a-4fbc-bd85-c56a0112db31-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tg5qk\" (UID: \"61462533-417a-4fbc-bd85-c56a0112db31\") " pod="openshift-insights/insights-runtime-extractor-tg5qk"
Oct 31 17:27:22.137317 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:22.137293    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:27:22.247457 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:22.247381    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:27:22.263333 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:22.263314    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-tg5qk"
Oct 31 17:27:22.544226 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:22.544137    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:27:22.572517 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:22.572278    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-tg5qk"]
Oct 31 17:27:22.575631 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:22.575599    4276 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61462533_417a_4fbc_bd85_c56a0112db31.slice/crio-1fd018cc361d109427b4017cad43b229134cd15a3a7e492cda187cd05f43f54b WatchSource:0}: Error finding container 1fd018cc361d109427b4017cad43b229134cd15a3a7e492cda187cd05f43f54b: Status 404 returned error can't find the container with id 1fd018cc361d109427b4017cad43b229134cd15a3a7e492cda187cd05f43f54b
Oct 31 17:27:22.747709 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:22.747677    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:27:23.320127 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:23.320101    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-zpf4d"
Oct 31 17:27:23.320329 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:23.320312    4276 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 31 17:27:23.428280 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:23.428251    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-tg5qk" event={"ID":"61462533-417a-4fbc-bd85-c56a0112db31","Type":"ContainerStarted","Data":"023d97d2e161d5b4b4ffceefb0c6a1daddf43edab146cea2f0d2493f938de3cf"}
Oct 31 17:27:23.428408 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:23.428299    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-tg5qk" event={"ID":"61462533-417a-4fbc-bd85-c56a0112db31","Type":"ContainerStarted","Data":"2a75fde0053cd6d418af1ddf839dd30deb9043e877535d08b9ee5d25be4522b8"}
Oct 31 17:27:23.428408 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:23.428319    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-tg5qk" event={"ID":"61462533-417a-4fbc-bd85-c56a0112db31","Type":"ContainerStarted","Data":"1fd018cc361d109427b4017cad43b229134cd15a3a7e492cda187cd05f43f54b"}
Oct 31 17:27:24.089087 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:24.089048    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:27:24.194751 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:24.194683    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:27:24.315222 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:24.315193    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-tskzw"
Oct 31 17:27:25.436914 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:25.436871    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-tg5qk" event={"ID":"61462533-417a-4fbc-bd85-c56a0112db31","Type":"ContainerStarted","Data":"5d3b7a51a1002862d1344d6cb9af10052c43f8ff609dec740f8bf55baaaa2bdb"}
Oct 31 17:27:25.549915 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:25.549882    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:27:25.662388 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:25.662358    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:27:25.767730 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:25.767657    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:27:25.884649 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:25.884616    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:27:25.998308 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:25.998281    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:27:26.110988 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:26.110958    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:27:26.266343 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:26.266312    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:27:26.377662 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:26.377584    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:27:27.717671 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:27.717634    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:27:29.323307 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:29.323276    4276 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-v54cn"]
Oct 31 17:27:29.326244 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:29.326222    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-v54cn"
Oct 31 17:27:29.336716 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:29.336698    4276 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-6dd789c8b7-zgfbl"]
Oct 31 17:27:29.339262 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:29.339246    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-6dd789c8b7-zgfbl"
Oct 31 17:27:29.347225 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:29.347199    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-f4hkh\""
Oct 31 17:27:29.350047 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:29.349872    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Oct 31 17:27:29.350647 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:29.350627    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-v54cn"]
Oct 31 17:27:29.360865 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:29.360847    4276 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Oct 31 17:27:29.360971 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:29.360920    4276 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Oct 31 17:27:29.364080 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:29.364064    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-6dd789c8b7-zgfbl"]
Oct 31 17:27:29.389233 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:29.389206    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:27:29.417400 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:29.417378    4276 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-865894c87d-kktxm"]
Oct 31 17:27:29.420191 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:29.420176    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-865894c87d-kktxm"
Oct 31 17:27:29.452363 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:29.452340    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Oct 31 17:27:29.452503 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:29.452377    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Oct 31 17:27:29.452503 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:29.452422    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-6krd7\""
Oct 31 17:27:29.452503 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:29.452429    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Oct 31 17:27:29.452626 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:29.452601    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"downloads-dockercfg-dd7hr\""
Oct 31 17:27:29.452768 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:29.452754    4276 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Oct 31 17:27:29.453049 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:29.453035    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-865894c87d-kktxm"]
Oct 31 17:27:29.458788 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:29.458770    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/45619685-5030-4e78-8c16-09f401ca1692-tls-certificates\") pod \"prometheus-operator-admission-webhook-6489d8bf89-v54cn\" (UID: \"45619685-5030-4e78-8c16-09f401ca1692\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-v54cn"
Oct 31 17:27:29.458859 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:29.458795    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d7460c3b-edb4-40ce-9b5f-89e46950a7c0-tmp\") pod \"downloads-6dd789c8b7-zgfbl\" (UID: \"d7460c3b-edb4-40ce-9b5f-89e46950a7c0\") " pod="openshift-console/downloads-6dd789c8b7-zgfbl"
Oct 31 17:27:29.458859 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:29.458829    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nr6vk\" (UniqueName: \"kubernetes.io/projected/d7460c3b-edb4-40ce-9b5f-89e46950a7c0-kube-api-access-nr6vk\") pod \"downloads-6dd789c8b7-zgfbl\" (UID: \"d7460c3b-edb4-40ce-9b5f-89e46950a7c0\") " pod="openshift-console/downloads-6dd789c8b7-zgfbl"
Oct 31 17:27:29.544196 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:29.544167    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:27:29.559694 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:29.559674    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/765a023c-9406-4774-95f5-5af640086496-registry-tls\") pod \"image-registry-865894c87d-kktxm\" (UID: \"765a023c-9406-4774-95f5-5af640086496\") " pod="openshift-image-registry/image-registry-865894c87d-kktxm"
Oct 31 17:27:29.559787 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:29.559700    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/765a023c-9406-4774-95f5-5af640086496-trusted-ca\") pod \"image-registry-865894c87d-kktxm\" (UID: \"765a023c-9406-4774-95f5-5af640086496\") " pod="openshift-image-registry/image-registry-865894c87d-kktxm"
Oct 31 17:27:29.559787 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:29.559720    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t9nf6\" (UniqueName: \"kubernetes.io/projected/765a023c-9406-4774-95f5-5af640086496-kube-api-access-t9nf6\") pod \"image-registry-865894c87d-kktxm\" (UID: \"765a023c-9406-4774-95f5-5af640086496\") " pod="openshift-image-registry/image-registry-865894c87d-kktxm"
Oct 31 17:27:29.559787 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:29.559758    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/765a023c-9406-4774-95f5-5af640086496-image-registry-private-configuration\") pod \"image-registry-865894c87d-kktxm\" (UID: \"765a023c-9406-4774-95f5-5af640086496\") " pod="openshift-image-registry/image-registry-865894c87d-kktxm"
Oct 31 17:27:29.559887 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:29.559873    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nr6vk\" (UniqueName: \"kubernetes.io/projected/d7460c3b-edb4-40ce-9b5f-89e46950a7c0-kube-api-access-nr6vk\") pod \"downloads-6dd789c8b7-zgfbl\" (UID: \"d7460c3b-edb4-40ce-9b5f-89e46950a7c0\") " pod="openshift-console/downloads-6dd789c8b7-zgfbl"
Oct 31 17:27:29.559920 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:29.559903    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/765a023c-9406-4774-95f5-5af640086496-bound-sa-token\") pod \"image-registry-865894c87d-kktxm\" (UID: \"765a023c-9406-4774-95f5-5af640086496\") " pod="openshift-image-registry/image-registry-865894c87d-kktxm"
Oct 31 17:27:29.559952 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:29.559935    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/765a023c-9406-4774-95f5-5af640086496-registry-certificates\") pod \"image-registry-865894c87d-kktxm\" (UID: \"765a023c-9406-4774-95f5-5af640086496\") " pod="openshift-image-registry/image-registry-865894c87d-kktxm"
Oct 31 17:27:29.560038 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:29.560021    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/765a023c-9406-4774-95f5-5af640086496-installation-pull-secrets\") pod \"image-registry-865894c87d-kktxm\" (UID: \"765a023c-9406-4774-95f5-5af640086496\") " pod="openshift-image-registry/image-registry-865894c87d-kktxm"
Oct 31 17:27:29.560075 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:29.560060    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/45619685-5030-4e78-8c16-09f401ca1692-tls-certificates\") pod \"prometheus-operator-admission-webhook-6489d8bf89-v54cn\" (UID: \"45619685-5030-4e78-8c16-09f401ca1692\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-v54cn"
Oct 31 17:27:29.560110 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:29.560084    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/765a023c-9406-4774-95f5-5af640086496-ca-trust-extracted\") pod \"image-registry-865894c87d-kktxm\" (UID: \"765a023c-9406-4774-95f5-5af640086496\") " pod="openshift-image-registry/image-registry-865894c87d-kktxm"
Oct 31 17:27:29.560142 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:29.560131    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d7460c3b-edb4-40ce-9b5f-89e46950a7c0-tmp\") pod \"downloads-6dd789c8b7-zgfbl\" (UID: \"d7460c3b-edb4-40ce-9b5f-89e46950a7c0\") " pod="openshift-console/downloads-6dd789c8b7-zgfbl"
Oct 31 17:27:29.560434 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:29.560418    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d7460c3b-edb4-40ce-9b5f-89e46950a7c0-tmp\") pod \"downloads-6dd789c8b7-zgfbl\" (UID: \"d7460c3b-edb4-40ce-9b5f-89e46950a7c0\") " pod="openshift-console/downloads-6dd789c8b7-zgfbl"
Oct 31 17:27:29.563662 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:29.563642    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/45619685-5030-4e78-8c16-09f401ca1692-tls-certificates\") pod \"prometheus-operator-admission-webhook-6489d8bf89-v54cn\" (UID: \"45619685-5030-4e78-8c16-09f401ca1692\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-v54cn"
Oct 31 17:27:29.582266 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:29.582215    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nr6vk\" (UniqueName: \"kubernetes.io/projected/d7460c3b-edb4-40ce-9b5f-89e46950a7c0-kube-api-access-nr6vk\") pod \"downloads-6dd789c8b7-zgfbl\" (UID: \"d7460c3b-edb4-40ce-9b5f-89e46950a7c0\") " pod="openshift-console/downloads-6dd789c8b7-zgfbl"
Oct 31 17:27:29.636743 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:29.636720    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-v54cn"
Oct 31 17:27:29.649705 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:29.649687    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-6dd789c8b7-zgfbl"
Oct 31 17:27:29.654246 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:29.654224    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:27:29.661246 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:29.661222    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/765a023c-9406-4774-95f5-5af640086496-registry-tls\") pod \"image-registry-865894c87d-kktxm\" (UID: \"765a023c-9406-4774-95f5-5af640086496\") " pod="openshift-image-registry/image-registry-865894c87d-kktxm"
Oct 31 17:27:29.661838 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:29.661259    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/765a023c-9406-4774-95f5-5af640086496-trusted-ca\") pod \"image-registry-865894c87d-kktxm\" (UID: \"765a023c-9406-4774-95f5-5af640086496\") " pod="openshift-image-registry/image-registry-865894c87d-kktxm"
Oct 31 17:27:29.661838 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:29.661284    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t9nf6\" (UniqueName: \"kubernetes.io/projected/765a023c-9406-4774-95f5-5af640086496-kube-api-access-t9nf6\") pod \"image-registry-865894c87d-kktxm\" (UID: \"765a023c-9406-4774-95f5-5af640086496\") " pod="openshift-image-registry/image-registry-865894c87d-kktxm"
Oct 31 17:27:29.661838 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:29.661350    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/765a023c-9406-4774-95f5-5af640086496-image-registry-private-configuration\") pod \"image-registry-865894c87d-kktxm\" (UID: \"765a023c-9406-4774-95f5-5af640086496\") " pod="openshift-image-registry/image-registry-865894c87d-kktxm"
Oct 31 17:27:29.661838 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:29.661389    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/765a023c-9406-4774-95f5-5af640086496-bound-sa-token\") pod \"image-registry-865894c87d-kktxm\" (UID: \"765a023c-9406-4774-95f5-5af640086496\") " pod="openshift-image-registry/image-registry-865894c87d-kktxm"
Oct 31 17:27:29.661838 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:29.661424    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/765a023c-9406-4774-95f5-5af640086496-registry-certificates\") pod \"image-registry-865894c87d-kktxm\" (UID: \"765a023c-9406-4774-95f5-5af640086496\") " pod="openshift-image-registry/image-registry-865894c87d-kktxm"
Oct 31 17:27:29.661838 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:29.661489    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/765a023c-9406-4774-95f5-5af640086496-installation-pull-secrets\") pod \"image-registry-865894c87d-kktxm\" (UID: \"765a023c-9406-4774-95f5-5af640086496\") " pod="openshift-image-registry/image-registry-865894c87d-kktxm"
Oct 31 17:27:29.661838 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:29.661523    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/765a023c-9406-4774-95f5-5af640086496-ca-trust-extracted\") pod \"image-registry-865894c87d-kktxm\" (UID: \"765a023c-9406-4774-95f5-5af640086496\") " pod="openshift-image-registry/image-registry-865894c87d-kktxm"
Oct 31 17:27:29.662223 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:29.661886    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/765a023c-9406-4774-95f5-5af640086496-ca-trust-extracted\") pod \"image-registry-865894c87d-kktxm\" (UID: \"765a023c-9406-4774-95f5-5af640086496\") " pod="openshift-image-registry/image-registry-865894c87d-kktxm"
Oct 31 17:27:29.663052 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:29.663015    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/765a023c-9406-4774-95f5-5af640086496-trusted-ca\") pod \"image-registry-865894c87d-kktxm\" (UID: \"765a023c-9406-4774-95f5-5af640086496\") " pod="openshift-image-registry/image-registry-865894c87d-kktxm"
Oct 31 17:27:29.663148 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:29.663073    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/765a023c-9406-4774-95f5-5af640086496-registry-certificates\") pod \"image-registry-865894c87d-kktxm\" (UID: \"765a023c-9406-4774-95f5-5af640086496\") " pod="openshift-image-registry/image-registry-865894c87d-kktxm"
Oct 31 17:27:29.668061 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:29.665159    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/765a023c-9406-4774-95f5-5af640086496-image-registry-private-configuration\") pod \"image-registry-865894c87d-kktxm\" (UID: \"765a023c-9406-4774-95f5-5af640086496\") " pod="openshift-image-registry/image-registry-865894c87d-kktxm"
Oct 31 17:27:29.668061 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:29.665630    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/765a023c-9406-4774-95f5-5af640086496-registry-tls\") pod \"image-registry-865894c87d-kktxm\" (UID: \"765a023c-9406-4774-95f5-5af640086496\") " pod="openshift-image-registry/image-registry-865894c87d-kktxm"
Oct 31 17:27:29.670104 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:29.670080    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/765a023c-9406-4774-95f5-5af640086496-installation-pull-secrets\") pod \"image-registry-865894c87d-kktxm\" (UID: \"765a023c-9406-4774-95f5-5af640086496\") " pod="openshift-image-registry/image-registry-865894c87d-kktxm"
Oct 31 17:27:29.681423 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:29.681404    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/765a023c-9406-4774-95f5-5af640086496-bound-sa-token\") pod \"image-registry-865894c87d-kktxm\" (UID: \"765a023c-9406-4774-95f5-5af640086496\") " pod="openshift-image-registry/image-registry-865894c87d-kktxm"
Oct 31 17:27:29.681924 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:29.681902    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t9nf6\" (UniqueName: \"kubernetes.io/projected/765a023c-9406-4774-95f5-5af640086496-kube-api-access-t9nf6\") pod \"image-registry-865894c87d-kktxm\" (UID: \"765a023c-9406-4774-95f5-5af640086496\") " pod="openshift-image-registry/image-registry-865894c87d-kktxm"
Oct 31 17:27:29.731415 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:29.731387    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-865894c87d-kktxm"
Oct 31 17:27:29.934632 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:29.934563    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-v54cn"]
Oct 31 17:27:29.937139 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:29.937117    4276 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod45619685_5030_4e78_8c16_09f401ca1692.slice/crio-a22ec58ce8599fb385843f7b1da764b629c226ec897e2c22cb29e8de7f092f19 WatchSource:0}: Error finding container a22ec58ce8599fb385843f7b1da764b629c226ec897e2c22cb29e8de7f092f19: Status 404 returned error can't find the container with id a22ec58ce8599fb385843f7b1da764b629c226ec897e2c22cb29e8de7f092f19
Oct 31 17:27:29.971590 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:29.971562    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-6dd789c8b7-zgfbl"]
Oct 31 17:27:29.974108 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:29.974085    4276 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7460c3b_edb4_40ce_9b5f_89e46950a7c0.slice/crio-8cab67a0fc9edd8aa05e8fce2fe7db17018b7cfcb7c7d9d655c6be72f0ab6711 WatchSource:0}: Error finding container 8cab67a0fc9edd8aa05e8fce2fe7db17018b7cfcb7c7d9d655c6be72f0ab6711: Status 404 returned error can't find the container with id 8cab67a0fc9edd8aa05e8fce2fe7db17018b7cfcb7c7d9d655c6be72f0ab6711
Oct 31 17:27:30.023345 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:30.023323    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-865894c87d-kktxm"]
Oct 31 17:27:30.025740 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:30.025719    4276 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod765a023c_9406_4774_95f5_5af640086496.slice/crio-17b8b71a23e085c2636c18a4bcd9aca3181a511dd6efb55faa28ccc02628f006 WatchSource:0}: Error finding container 17b8b71a23e085c2636c18a4bcd9aca3181a511dd6efb55faa28ccc02628f006: Status 404 returned error can't find the container with id 17b8b71a23e085c2636c18a4bcd9aca3181a511dd6efb55faa28ccc02628f006
Oct 31 17:27:30.455888 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:30.455747    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-865894c87d-kktxm" event={"ID":"765a023c-9406-4774-95f5-5af640086496","Type":"ContainerStarted","Data":"bac8e54f02a17fb576dbee8c37e927b83bee1f7d96f75ac9d28223225e92232d"}
Oct 31 17:27:30.455888 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:30.455804    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-865894c87d-kktxm" event={"ID":"765a023c-9406-4774-95f5-5af640086496","Type":"ContainerStarted","Data":"17b8b71a23e085c2636c18a4bcd9aca3181a511dd6efb55faa28ccc02628f006"}
Oct 31 17:27:30.455888 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:30.455839    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-865894c87d-kktxm"
Oct 31 17:27:30.457178 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:30.457151    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-6dd789c8b7-zgfbl" event={"ID":"d7460c3b-edb4-40ce-9b5f-89e46950a7c0","Type":"ContainerStarted","Data":"8cab67a0fc9edd8aa05e8fce2fe7db17018b7cfcb7c7d9d655c6be72f0ab6711"}
Oct 31 17:27:30.458730 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:30.458700    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-v54cn" event={"ID":"45619685-5030-4e78-8c16-09f401ca1692","Type":"ContainerStarted","Data":"a22ec58ce8599fb385843f7b1da764b629c226ec897e2c22cb29e8de7f092f19"}
Oct 31 17:27:31.359064 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:31.359037    4276 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-59fpc_9ee82620-6497-48e8-bb2c-1bb77a93f7ef/dns/0.log"
Oct 31 17:27:31.423798 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:31.423771    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-59fpc"
Oct 31 17:27:31.465309 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:31.465271    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-v54cn" event={"ID":"45619685-5030-4e78-8c16-09f401ca1692","Type":"ContainerStarted","Data":"3285c6402c5371dc932edcfabac6eb25acbd01d7fa9827e7a1d188f95d803a20"}
Oct 31 17:27:31.465309 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:31.465316    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-v54cn"
Oct 31 17:27:31.465839 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:31.465706    4276 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-image-registry/image-registry-865894c87d-kktxm" podStartSLOduration=2.46568984 podStartE2EDuration="2.46568984s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 17:27:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 17:27:30.499458458 +0000 UTC m=+13.297203380" watchObservedRunningTime="2026-10-31 17:27:31.46568984 +0000 UTC m=+14.263434749"
Oct 31 17:27:31.471715 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:31.471689    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-v54cn"
Oct 31 17:27:31.543490 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:31.543428    4276 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-v54cn" podStartSLOduration=1.531688325 podStartE2EDuration="2.543410643s" totalImagesPullingTime="1.011722318s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 17:27:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 17:27:31.504474339 +0000 UTC m=+14.302219270" watchObservedRunningTime="2026-10-31 17:27:31.543410643 +0000 UTC m=+14.341155564"
Oct 31 17:27:31.564100 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:31.564069    4276 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-59fpc_9ee82620-6497-48e8-bb2c-1bb77a93f7ef/kube-rbac-proxy/0.log"
Oct 31 17:27:31.706418 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:31.706345    4276 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-598df4d98d-2vp6w"]
Oct 31 17:27:31.709642 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:31.709620    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-598df4d98d-2vp6w"
Oct 31 17:27:31.726839 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:31.726814    4276 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Oct 31 17:27:31.727023 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:31.727008    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Oct 31 17:27:31.727109 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:31.727083    4276 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Oct 31 17:27:31.727213 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:31.727185    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-ht46x\""
Oct 31 17:27:31.727346 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:31.727227    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Oct 31 17:27:31.727346 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:31.727331    4276 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Oct 31 17:27:31.727621 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:31.727604    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-598df4d98d-2vp6w"]
Oct 31 17:27:31.880864 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:31.880832    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/92766ec3-7459-4dd1-894c-e15ad29b98ed-service-ca\") pod \"console-598df4d98d-2vp6w\" (UID: \"92766ec3-7459-4dd1-894c-e15ad29b98ed\") " pod="openshift-console/console-598df4d98d-2vp6w"
Oct 31 17:27:31.881036 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:31.880882    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/92766ec3-7459-4dd1-894c-e15ad29b98ed-console-serving-cert\") pod \"console-598df4d98d-2vp6w\" (UID: \"92766ec3-7459-4dd1-894c-e15ad29b98ed\") " pod="openshift-console/console-598df4d98d-2vp6w"
Oct 31 17:27:31.881036 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:31.880989    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/92766ec3-7459-4dd1-894c-e15ad29b98ed-console-oauth-config\") pod \"console-598df4d98d-2vp6w\" (UID: \"92766ec3-7459-4dd1-894c-e15ad29b98ed\") " pod="openshift-console/console-598df4d98d-2vp6w"
Oct 31 17:27:31.881036 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:31.881018    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9plzn\" (UniqueName: \"kubernetes.io/projected/92766ec3-7459-4dd1-894c-e15ad29b98ed-kube-api-access-9plzn\") pod \"console-598df4d98d-2vp6w\" (UID: \"92766ec3-7459-4dd1-894c-e15ad29b98ed\") " pod="openshift-console/console-598df4d98d-2vp6w"
Oct 31 17:27:31.881179 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:31.881061    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/92766ec3-7459-4dd1-894c-e15ad29b98ed-tmp\") pod \"console-598df4d98d-2vp6w\" (UID: \"92766ec3-7459-4dd1-894c-e15ad29b98ed\") " pod="openshift-console/console-598df4d98d-2vp6w"
Oct 31 17:27:31.881179 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:31.881114    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/92766ec3-7459-4dd1-894c-e15ad29b98ed-oauth-serving-cert\") pod \"console-598df4d98d-2vp6w\" (UID: \"92766ec3-7459-4dd1-894c-e15ad29b98ed\") " pod="openshift-console/console-598df4d98d-2vp6w"
Oct 31 17:27:31.881261 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:31.881247    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/92766ec3-7459-4dd1-894c-e15ad29b98ed-console-config\") pod \"console-598df4d98d-2vp6w\" (UID: \"92766ec3-7459-4dd1-894c-e15ad29b98ed\") " pod="openshift-console/console-598df4d98d-2vp6w"
Oct 31 17:27:31.982218 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:31.982139    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/92766ec3-7459-4dd1-894c-e15ad29b98ed-service-ca\") pod \"console-598df4d98d-2vp6w\" (UID: \"92766ec3-7459-4dd1-894c-e15ad29b98ed\") " pod="openshift-console/console-598df4d98d-2vp6w"
Oct 31 17:27:31.982218 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:31.982181    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/92766ec3-7459-4dd1-894c-e15ad29b98ed-console-serving-cert\") pod \"console-598df4d98d-2vp6w\" (UID: \"92766ec3-7459-4dd1-894c-e15ad29b98ed\") " pod="openshift-console/console-598df4d98d-2vp6w"
Oct 31 17:27:31.982218 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:31.982212    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/92766ec3-7459-4dd1-894c-e15ad29b98ed-console-oauth-config\") pod \"console-598df4d98d-2vp6w\" (UID: \"92766ec3-7459-4dd1-894c-e15ad29b98ed\") " pod="openshift-console/console-598df4d98d-2vp6w"
Oct 31 17:27:31.982474 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:31.982238    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9plzn\" (UniqueName: \"kubernetes.io/projected/92766ec3-7459-4dd1-894c-e15ad29b98ed-kube-api-access-9plzn\") pod \"console-598df4d98d-2vp6w\" (UID: \"92766ec3-7459-4dd1-894c-e15ad29b98ed\") " pod="openshift-console/console-598df4d98d-2vp6w"
Oct 31 17:27:31.982474 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:31.982279    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/92766ec3-7459-4dd1-894c-e15ad29b98ed-tmp\") pod \"console-598df4d98d-2vp6w\" (UID: \"92766ec3-7459-4dd1-894c-e15ad29b98ed\") " pod="openshift-console/console-598df4d98d-2vp6w"
Oct 31 17:27:31.982474 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:31.982303    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/92766ec3-7459-4dd1-894c-e15ad29b98ed-oauth-serving-cert\") pod \"console-598df4d98d-2vp6w\" (UID: \"92766ec3-7459-4dd1-894c-e15ad29b98ed\") " pod="openshift-console/console-598df4d98d-2vp6w"
Oct 31 17:27:31.982474 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:31.982337    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/92766ec3-7459-4dd1-894c-e15ad29b98ed-console-config\") pod \"console-598df4d98d-2vp6w\" (UID: \"92766ec3-7459-4dd1-894c-e15ad29b98ed\") " pod="openshift-console/console-598df4d98d-2vp6w"
Oct 31 17:27:31.983021 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:31.982984    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/92766ec3-7459-4dd1-894c-e15ad29b98ed-tmp\") pod \"console-598df4d98d-2vp6w\" (UID: \"92766ec3-7459-4dd1-894c-e15ad29b98ed\") " pod="openshift-console/console-598df4d98d-2vp6w"
Oct 31 17:27:31.983021 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:31.983021    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/92766ec3-7459-4dd1-894c-e15ad29b98ed-service-ca\") pod \"console-598df4d98d-2vp6w\" (UID: \"92766ec3-7459-4dd1-894c-e15ad29b98ed\") " pod="openshift-console/console-598df4d98d-2vp6w"
Oct 31 17:27:31.983209 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:31.983021    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/92766ec3-7459-4dd1-894c-e15ad29b98ed-console-config\") pod \"console-598df4d98d-2vp6w\" (UID: \"92766ec3-7459-4dd1-894c-e15ad29b98ed\") " pod="openshift-console/console-598df4d98d-2vp6w"
Oct 31 17:27:31.983567 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:31.983544    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/92766ec3-7459-4dd1-894c-e15ad29b98ed-oauth-serving-cert\") pod \"console-598df4d98d-2vp6w\" (UID: \"92766ec3-7459-4dd1-894c-e15ad29b98ed\") " pod="openshift-console/console-598df4d98d-2vp6w"
Oct 31 17:27:31.985068 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:31.985048    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/92766ec3-7459-4dd1-894c-e15ad29b98ed-console-oauth-config\") pod \"console-598df4d98d-2vp6w\" (UID: \"92766ec3-7459-4dd1-894c-e15ad29b98ed\") " pod="openshift-console/console-598df4d98d-2vp6w"
Oct 31 17:27:31.986705 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:31.986670    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/92766ec3-7459-4dd1-894c-e15ad29b98ed-console-serving-cert\") pod \"console-598df4d98d-2vp6w\" (UID: \"92766ec3-7459-4dd1-894c-e15ad29b98ed\") " pod="openshift-console/console-598df4d98d-2vp6w"
Oct 31 17:27:32.003006 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:32.002977    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9plzn\" (UniqueName: \"kubernetes.io/projected/92766ec3-7459-4dd1-894c-e15ad29b98ed-kube-api-access-9plzn\") pod \"console-598df4d98d-2vp6w\" (UID: \"92766ec3-7459-4dd1-894c-e15ad29b98ed\") " pod="openshift-console/console-598df4d98d-2vp6w"
Oct 31 17:27:32.022808 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:32.022783    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-598df4d98d-2vp6w"
Oct 31 17:27:32.326839 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:32.326811    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-598df4d98d-2vp6w"]
Oct 31 17:27:32.334792 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:32.334763    4276 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92766ec3_7459_4dd1_894c_e15ad29b98ed.slice/crio-3bc4e9ba11777481151cc2dd04c9602472c65e04da2b692a838f75c5d5fe7dce WatchSource:0}: Error finding container 3bc4e9ba11777481151cc2dd04c9602472c65e04da2b692a838f75c5d5fe7dce: Status 404 returned error can't find the container with id 3bc4e9ba11777481151cc2dd04c9602472c65e04da2b692a838f75c5d5fe7dce
Oct 31 17:27:32.358324 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:32.358298    4276 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-7pfr4_b8bbaf33-7cd9-4a30-86f7-6e20ac45b0a0/dns-node-resolver/0.log"
Oct 31 17:27:32.404403 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:32.404376    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:27:32.469515 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:32.469079    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-598df4d98d-2vp6w" event={"ID":"92766ec3-7459-4dd1-894c-e15ad29b98ed","Type":"ContainerStarted","Data":"3bc4e9ba11777481151cc2dd04c9602472c65e04da2b692a838f75c5d5fe7dce"}
Oct 31 17:27:32.490125 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:32.490093    4276 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-9cd89b74c-kfd5z"]
Oct 31 17:27:32.494763 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:32.494743    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-9cd89b74c-kfd5z"
Oct 31 17:27:32.511039 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:32.510998    4276 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Oct 31 17:27:32.514139 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:32.514112    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Oct 31 17:27:32.514240 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:32.514119    4276 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Oct 31 17:27:32.518083 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:32.518059    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:27:32.524749 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:32.524728    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Oct 31 17:27:32.527317 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:32.527297    4276 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Oct 31 17:27:32.527317 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:32.527309    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-dhv4b\""
Oct 31 17:27:32.528138 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:32.528119    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-9cd89b74c-kfd5z"]
Oct 31 17:27:32.587278 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:32.586721    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/54a68758-d202-43a5-b6e4-cb4287494abf-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-kfd5z\" (UID: \"54a68758-d202-43a5-b6e4-cb4287494abf\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-kfd5z"
Oct 31 17:27:32.587278 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:32.586771    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/54a68758-d202-43a5-b6e4-cb4287494abf-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-kfd5z\" (UID: \"54a68758-d202-43a5-b6e4-cb4287494abf\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-kfd5z"
Oct 31 17:27:32.587278 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:32.586902    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/54a68758-d202-43a5-b6e4-cb4287494abf-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-kfd5z\" (UID: \"54a68758-d202-43a5-b6e4-cb4287494abf\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-kfd5z"
Oct 31 17:27:32.587278 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:32.586969    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2xbhs\" (UniqueName: \"kubernetes.io/projected/54a68758-d202-43a5-b6e4-cb4287494abf-kube-api-access-2xbhs\") pod \"prometheus-operator-9cd89b74c-kfd5z\" (UID: \"54a68758-d202-43a5-b6e4-cb4287494abf\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-kfd5z"
Oct 31 17:27:32.623668 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:32.623642    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:27:32.687310 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:32.687282    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/54a68758-d202-43a5-b6e4-cb4287494abf-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-kfd5z\" (UID: \"54a68758-d202-43a5-b6e4-cb4287494abf\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-kfd5z"
Oct 31 17:27:32.687436 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:32.687347    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/54a68758-d202-43a5-b6e4-cb4287494abf-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-kfd5z\" (UID: \"54a68758-d202-43a5-b6e4-cb4287494abf\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-kfd5z"
Oct 31 17:27:32.687436 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:32.687389    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2xbhs\" (UniqueName: \"kubernetes.io/projected/54a68758-d202-43a5-b6e4-cb4287494abf-kube-api-access-2xbhs\") pod \"prometheus-operator-9cd89b74c-kfd5z\" (UID: \"54a68758-d202-43a5-b6e4-cb4287494abf\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-kfd5z"
Oct 31 17:27:32.687574 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:32.687438    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/54a68758-d202-43a5-b6e4-cb4287494abf-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-kfd5z\" (UID: \"54a68758-d202-43a5-b6e4-cb4287494abf\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-kfd5z"
Oct 31 17:27:32.688210 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:32.688181    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/54a68758-d202-43a5-b6e4-cb4287494abf-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-kfd5z\" (UID: \"54a68758-d202-43a5-b6e4-cb4287494abf\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-kfd5z"
Oct 31 17:27:32.690127 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:32.690109    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/54a68758-d202-43a5-b6e4-cb4287494abf-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-kfd5z\" (UID: \"54a68758-d202-43a5-b6e4-cb4287494abf\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-kfd5z"
Oct 31 17:27:32.690231 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:32.690209    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/54a68758-d202-43a5-b6e4-cb4287494abf-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-kfd5z\" (UID: \"54a68758-d202-43a5-b6e4-cb4287494abf\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-kfd5z"
Oct 31 17:27:32.707527 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:32.707508    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2xbhs\" (UniqueName: \"kubernetes.io/projected/54a68758-d202-43a5-b6e4-cb4287494abf-kube-api-access-2xbhs\") pod \"prometheus-operator-9cd89b74c-kfd5z\" (UID: \"54a68758-d202-43a5-b6e4-cb4287494abf\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-kfd5z"
Oct 31 17:27:32.810392 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:32.810356    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-9cd89b74c-kfd5z"
Oct 31 17:27:32.958582 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:32.958553    4276 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-865894c87d-kktxm_765a023c-9406-4774-95f5-5af640086496/registry/0.log"
Oct 31 17:27:33.107827 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:33.107795    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-9cd89b74c-kfd5z"]
Oct 31 17:27:33.111599 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:33.111565    4276 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54a68758_d202_43a5_b6e4_cb4287494abf.slice/crio-4627f551a7aa2b37b925a7a1a0ea21e2db04cbc27faec6d1bcc82d2aecbd0f83 WatchSource:0}: Error finding container 4627f551a7aa2b37b925a7a1a0ea21e2db04cbc27faec6d1bcc82d2aecbd0f83: Status 404 returned error can't find the container with id 4627f551a7aa2b37b925a7a1a0ea21e2db04cbc27faec6d1bcc82d2aecbd0f83
Oct 31 17:27:33.473866 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:33.473789    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-9cd89b74c-kfd5z" event={"ID":"54a68758-d202-43a5-b6e4-cb4287494abf","Type":"ContainerStarted","Data":"4627f551a7aa2b37b925a7a1a0ea21e2db04cbc27faec6d1bcc82d2aecbd0f83"}
Oct 31 17:27:33.758669 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:33.758596    4276 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-qxnvq_87e61dcd-75f6-4d3f-8ed9-7d35b06a9bb3/node-ca/0.log"
Oct 31 17:27:34.158594 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:34.158565    4276 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-b5ln2_22096c98-1b93-4ea4-9a6f-6383df9eafe4/serve-healthcheck-canary/0.log"
Oct 31 17:27:34.160963 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:34.160932    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:27:34.275554 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:34.275528    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:27:35.435314 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:35.435277    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:27:35.484767 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:35.484733    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-9cd89b74c-kfd5z" event={"ID":"54a68758-d202-43a5-b6e4-cb4287494abf","Type":"ContainerStarted","Data":"a695477ca52e2288fbc33b797b091b5783c77a9ce1cb5e0f7f802193cde1d5af"}
Oct 31 17:27:35.484941 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:35.484794    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-9cd89b74c-kfd5z" event={"ID":"54a68758-d202-43a5-b6e4-cb4287494abf","Type":"ContainerStarted","Data":"f67c228c4f0dd574b94c35691df735c790e702065642d0b555db536b49addef9"}
Oct 31 17:27:35.525281 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:35.525235    4276 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-9cd89b74c-kfd5z" podStartSLOduration=2.11037831 podStartE2EDuration="3.5252214s" totalImagesPullingTime="1.41484309s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 17:27:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 17:27:35.524538084 +0000 UTC m=+18.322283009" watchObservedRunningTime="2026-10-31 17:27:35.5252214 +0000 UTC m=+18.322966328"
Oct 31 17:27:36.490199 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:36.490159    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-598df4d98d-2vp6w" event={"ID":"92766ec3-7459-4dd1-894c-e15ad29b98ed","Type":"ContainerStarted","Data":"86bc69a323a9af0e2726ecda1e36b29dcedcac578bf32ac14c68bb4a9052a313"}
Oct 31 17:27:36.531030 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:36.530982    4276 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-598df4d98d-2vp6w" podStartSLOduration=2.321964828 podStartE2EDuration="5.530969122s" totalImagesPullingTime="3.209004294s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 17:27:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 17:27:36.530677878 +0000 UTC m=+19.328422806" watchObservedRunningTime="2026-10-31 17:27:36.530969122 +0000 UTC m=+19.328714049"
Oct 31 17:27:37.853837 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:37.853799    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:27:38.004768 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:38.004688    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:27:38.126526 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:38.126435    4276 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-mlwjd"]
Oct 31 17:27:38.130098 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:38.130075    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-mlwjd"
Oct 31 17:27:38.145227 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:38.145195    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:27:38.163640 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:38.163609    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Oct 31 17:27:38.163756 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:38.163698    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Oct 31 17:27:38.163822 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:38.163806    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-mcfpr\""
Oct 31 17:27:38.196255 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:38.196232    4276 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Oct 31 17:27:38.231610 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:38.231575    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2ca5233b-53f1-4251-b43f-fc9c0a14747a-sys\") pod \"node-exporter-mlwjd\" (UID: \"2ca5233b-53f1-4251-b43f-fc9c0a14747a\") " pod="openshift-monitoring/node-exporter-mlwjd"
Oct 31 17:27:38.231739 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:38.231624    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/2ca5233b-53f1-4251-b43f-fc9c0a14747a-node-exporter-textfile\") pod \"node-exporter-mlwjd\" (UID: \"2ca5233b-53f1-4251-b43f-fc9c0a14747a\") " pod="openshift-monitoring/node-exporter-mlwjd"
Oct 31 17:27:38.231739 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:38.231673    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2ca5233b-53f1-4251-b43f-fc9c0a14747a-node-exporter-tls\") pod \"node-exporter-mlwjd\" (UID: \"2ca5233b-53f1-4251-b43f-fc9c0a14747a\") " pod="openshift-monitoring/node-exporter-mlwjd"
Oct 31 17:27:38.231857 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:38.231752    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/2ca5233b-53f1-4251-b43f-fc9c0a14747a-node-exporter-wtmp\") pod \"node-exporter-mlwjd\" (UID: \"2ca5233b-53f1-4251-b43f-fc9c0a14747a\") " pod="openshift-monitoring/node-exporter-mlwjd"
Oct 31 17:27:38.231857 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:38.231833    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2ca5233b-53f1-4251-b43f-fc9c0a14747a-metrics-client-ca\") pod \"node-exporter-mlwjd\" (UID: \"2ca5233b-53f1-4251-b43f-fc9c0a14747a\") " pod="openshift-monitoring/node-exporter-mlwjd"
Oct 31 17:27:38.231937 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:38.231892    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sc5pp\" (UniqueName: \"kubernetes.io/projected/2ca5233b-53f1-4251-b43f-fc9c0a14747a-kube-api-access-sc5pp\") pod \"node-exporter-mlwjd\" (UID: \"2ca5233b-53f1-4251-b43f-fc9c0a14747a\") " pod="openshift-monitoring/node-exporter-mlwjd"
Oct 31 17:27:38.231937 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:38.231926    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2ca5233b-53f1-4251-b43f-fc9c0a14747a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-mlwjd\" (UID: \"2ca5233b-53f1-4251-b43f-fc9c0a14747a\") " pod="openshift-monitoring/node-exporter-mlwjd"
Oct 31 17:27:38.232019 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:38.231943    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/2ca5233b-53f1-4251-b43f-fc9c0a14747a-root\") pod \"node-exporter-mlwjd\" (UID: \"2ca5233b-53f1-4251-b43f-fc9c0a14747a\") " pod="openshift-monitoring/node-exporter-mlwjd"
Oct 31 17:27:38.232019 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:38.231964    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/2ca5233b-53f1-4251-b43f-fc9c0a14747a-node-exporter-accelerators-collector-config\") pod \"node-exporter-mlwjd\" (UID: \"2ca5233b-53f1-4251-b43f-fc9c0a14747a\") " pod="openshift-monitoring/node-exporter-mlwjd"
Oct 31 17:27:38.262804 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:38.262760    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:27:38.332555 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:38.332528    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2ca5233b-53f1-4251-b43f-fc9c0a14747a-metrics-client-ca\") pod \"node-exporter-mlwjd\" (UID: \"2ca5233b-53f1-4251-b43f-fc9c0a14747a\") " pod="openshift-monitoring/node-exporter-mlwjd"
Oct 31 17:27:38.332677 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:38.332562    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sc5pp\" (UniqueName: \"kubernetes.io/projected/2ca5233b-53f1-4251-b43f-fc9c0a14747a-kube-api-access-sc5pp\") pod \"node-exporter-mlwjd\" (UID: \"2ca5233b-53f1-4251-b43f-fc9c0a14747a\") " pod="openshift-monitoring/node-exporter-mlwjd"
Oct 31 17:27:38.332677 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:38.332587    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2ca5233b-53f1-4251-b43f-fc9c0a14747a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-mlwjd\" (UID: \"2ca5233b-53f1-4251-b43f-fc9c0a14747a\") " pod="openshift-monitoring/node-exporter-mlwjd"
Oct 31 17:27:38.332677 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:38.332669    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/2ca5233b-53f1-4251-b43f-fc9c0a14747a-root\") pod \"node-exporter-mlwjd\" (UID: \"2ca5233b-53f1-4251-b43f-fc9c0a14747a\") " pod="openshift-monitoring/node-exporter-mlwjd"
Oct 31 17:27:38.332830 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:38.332708    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/2ca5233b-53f1-4251-b43f-fc9c0a14747a-node-exporter-accelerators-collector-config\") pod \"node-exporter-mlwjd\" (UID: \"2ca5233b-53f1-4251-b43f-fc9c0a14747a\") " pod="openshift-monitoring/node-exporter-mlwjd"
Oct 31 17:27:38.332830 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:38.332737    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2ca5233b-53f1-4251-b43f-fc9c0a14747a-sys\") pod \"node-exporter-mlwjd\" (UID: \"2ca5233b-53f1-4251-b43f-fc9c0a14747a\") " pod="openshift-monitoring/node-exporter-mlwjd"
Oct 31 17:27:38.332830 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:38.332756    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/2ca5233b-53f1-4251-b43f-fc9c0a14747a-node-exporter-textfile\") pod \"node-exporter-mlwjd\" (UID: \"2ca5233b-53f1-4251-b43f-fc9c0a14747a\") " pod="openshift-monitoring/node-exporter-mlwjd"
Oct 31 17:27:38.332830 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:38.332786    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2ca5233b-53f1-4251-b43f-fc9c0a14747a-node-exporter-tls\") pod \"node-exporter-mlwjd\" (UID: \"2ca5233b-53f1-4251-b43f-fc9c0a14747a\") " pod="openshift-monitoring/node-exporter-mlwjd"
Oct 31 17:27:38.332830 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:38.332809    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/2ca5233b-53f1-4251-b43f-fc9c0a14747a-node-exporter-wtmp\") pod \"node-exporter-mlwjd\" (UID: \"2ca5233b-53f1-4251-b43f-fc9c0a14747a\") " pod="openshift-monitoring/node-exporter-mlwjd"
Oct 31 17:27:38.333065 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:38.332847    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/2ca5233b-53f1-4251-b43f-fc9c0a14747a-root\") pod \"node-exporter-mlwjd\" (UID: \"2ca5233b-53f1-4251-b43f-fc9c0a14747a\") " pod="openshift-monitoring/node-exporter-mlwjd"
Oct 31 17:27:38.333065 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:38.332845    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2ca5233b-53f1-4251-b43f-fc9c0a14747a-sys\") pod \"node-exporter-mlwjd\" (UID: \"2ca5233b-53f1-4251-b43f-fc9c0a14747a\") " pod="openshift-monitoring/node-exporter-mlwjd"
Oct 31 17:27:38.333065 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:38.332987    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/2ca5233b-53f1-4251-b43f-fc9c0a14747a-node-exporter-wtmp\") pod \"node-exporter-mlwjd\" (UID: \"2ca5233b-53f1-4251-b43f-fc9c0a14747a\") " pod="openshift-monitoring/node-exporter-mlwjd"
Oct 31 17:27:38.333216 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:38.333166    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/2ca5233b-53f1-4251-b43f-fc9c0a14747a-node-exporter-textfile\") pod \"node-exporter-mlwjd\" (UID: \"2ca5233b-53f1-4251-b43f-fc9c0a14747a\") " pod="openshift-monitoring/node-exporter-mlwjd"
Oct 31 17:27:38.333330 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:38.333285    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2ca5233b-53f1-4251-b43f-fc9c0a14747a-metrics-client-ca\") pod \"node-exporter-mlwjd\" (UID: \"2ca5233b-53f1-4251-b43f-fc9c0a14747a\") " pod="openshift-monitoring/node-exporter-mlwjd"
Oct 31 17:27:38.333330 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:38.333303    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/2ca5233b-53f1-4251-b43f-fc9c0a14747a-node-exporter-accelerators-collector-config\") pod \"node-exporter-mlwjd\" (UID: \"2ca5233b-53f1-4251-b43f-fc9c0a14747a\") " pod="openshift-monitoring/node-exporter-mlwjd"
Oct 31 17:27:38.335152 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:38.335133    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2ca5233b-53f1-4251-b43f-fc9c0a14747a-node-exporter-tls\") pod \"node-exporter-mlwjd\" (UID: \"2ca5233b-53f1-4251-b43f-fc9c0a14747a\") " pod="openshift-monitoring/node-exporter-mlwjd"
Oct 31 17:27:38.335345 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:38.335329    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2ca5233b-53f1-4251-b43f-fc9c0a14747a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-mlwjd\" (UID: \"2ca5233b-53f1-4251-b43f-fc9c0a14747a\") " pod="openshift-monitoring/node-exporter-mlwjd"
Oct 31 17:27:38.352581 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:38.352563    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sc5pp\" (UniqueName: \"kubernetes.io/projected/2ca5233b-53f1-4251-b43f-fc9c0a14747a-kube-api-access-sc5pp\") pod \"node-exporter-mlwjd\" (UID: \"2ca5233b-53f1-4251-b43f-fc9c0a14747a\") " pod="openshift-monitoring/node-exporter-mlwjd"
Oct 31 17:27:38.377508 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:38.377414    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:27:38.447382 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:38.447356    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-mlwjd"
Oct 31 17:27:38.460652 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:38.460615    4276 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ca5233b_53f1_4251_b43f_fc9c0a14747a.slice/crio-6e2d8cc001944766b9bcfbf661d309ae84e442f1b1f478dc66a0f6b8695f3eeb WatchSource:0}: Error finding container 6e2d8cc001944766b9bcfbf661d309ae84e442f1b1f478dc66a0f6b8695f3eeb: Status 404 returned error can't find the container with id 6e2d8cc001944766b9bcfbf661d309ae84e442f1b1f478dc66a0f6b8695f3eeb
Oct 31 17:27:38.488111 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:38.488069    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:27:38.498361 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:38.498337    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mlwjd" event={"ID":"2ca5233b-53f1-4251-b43f-fc9c0a14747a","Type":"ContainerStarted","Data":"6e2d8cc001944766b9bcfbf661d309ae84e442f1b1f478dc66a0f6b8695f3eeb"}
Oct 31 17:27:38.625316 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:38.625280    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:27:38.996201 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:38.996170    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:27:39.128152 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:39.128114    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:27:39.231380 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:39.231345    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:27:39.345606 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:39.345561    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:27:39.504467 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:39.504400    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mlwjd" event={"ID":"2ca5233b-53f1-4251-b43f-fc9c0a14747a","Type":"ContainerStarted","Data":"6a0fd51b5d8f1f2f4cd347d03cc1f1c27a04401997eba39f3025e79974fc26d9"}
Oct 31 17:27:39.552239 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:39.552210    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:27:39.706242 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:39.706155    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:27:40.510460 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:40.510406    4276 generic.go:356] "Generic (PLEG): container finished" podID="2ca5233b-53f1-4251-b43f-fc9c0a14747a" containerID="6a0fd51b5d8f1f2f4cd347d03cc1f1c27a04401997eba39f3025e79974fc26d9" exitCode=0
Oct 31 17:27:40.510903 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:40.510492    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mlwjd" event={"ID":"2ca5233b-53f1-4251-b43f-fc9c0a14747a","Type":"ContainerDied","Data":"6a0fd51b5d8f1f2f4cd347d03cc1f1c27a04401997eba39f3025e79974fc26d9"}
Oct 31 17:27:41.017271 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:41.017228    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:27:41.134734 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:41.134700    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:27:41.250191 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:41.250151    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:27:41.359655 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:41.359609    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:27:41.474657 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:41.474626    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:27:41.518194 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:41.518159    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mlwjd" event={"ID":"2ca5233b-53f1-4251-b43f-fc9c0a14747a","Type":"ContainerStarted","Data":"c6411d079621eebb6a96f15ead0572624b06790ca6dd33484281d7d987c1f7c4"}
Oct 31 17:27:41.518659 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:41.518219    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mlwjd" event={"ID":"2ca5233b-53f1-4251-b43f-fc9c0a14747a","Type":"ContainerStarted","Data":"b7d8a0e8639e44d2832f40eb2703c6d21a6765ca341e47cf3013fdc6ea562a38"}
Oct 31 17:27:41.580899 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:41.580867    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:27:41.591074 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:41.591022    4276 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-mlwjd" podStartSLOduration=2.584992695 podStartE2EDuration="3.591006874s" totalImagesPullingTime="929.951933ms" totalInitContainerRuntime="76.062246ms" isStatefulPod=true podCreationTimestamp="2026-10-31 17:27:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 17:27:41.57730106 +0000 UTC m=+24.375045988" watchObservedRunningTime="2026-10-31 17:27:41.591006874 +0000 UTC m=+24.388751802"
Oct 31 17:27:42.023714 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:42.023673    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-598df4d98d-2vp6w"
Oct 31 17:27:42.023916 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:42.023820    4276 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-598df4d98d-2vp6w"
Oct 31 17:27:42.029366 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:42.029345    4276 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-598df4d98d-2vp6w"
Oct 31 17:27:42.236951 ip-10-0-129-127 kubenswrapper[4276]: E1031 17:27:42.236902    4276 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tg5qk" containerName="extractor"
Oct 31 17:27:42.526105 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:42.526060    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-598df4d98d-2vp6w"
Oct 31 17:27:42.945149 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:42.945112    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:27:43.058139 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:43.058109    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:27:43.178762 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:43.178727    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:27:45.116005 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:45.115970    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:27:47.438573 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:47.438520    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:27:47.542127 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:47.542092    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-6dd789c8b7-zgfbl" event={"ID":"d7460c3b-edb4-40ce-9b5f-89e46950a7c0","Type":"ContainerStarted","Data":"002115a92d77cbed9659680727c35c21799ea194e068b93d44b01130f88e88d0"}
Oct 31 17:27:47.542389 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:47.542366    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-6dd789c8b7-zgfbl"
Oct 31 17:27:47.549630 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:47.549588    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:27:47.556558 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:47.556539    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-6dd789c8b7-zgfbl"
Oct 31 17:27:47.596653 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:47.596599    4276 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/downloads-6dd789c8b7-zgfbl" podStartSLOduration=1.988726193 podStartE2EDuration="18.596580272s" totalImagesPullingTime="16.607854079s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 17:27:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 17:27:47.595991678 +0000 UTC m=+30.393736606" watchObservedRunningTime="2026-10-31 17:27:47.596580272 +0000 UTC m=+30.394325201"
Oct 31 17:27:48.032581 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:48.032549    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-zpf4d"
Oct 31 17:27:48.365469 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:48.365425    4276 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-7cc89f4ff-7cckv"]
Oct 31 17:27:48.399476 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:48.396994    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7cc89f4ff-7cckv"]
Oct 31 17:27:48.399476 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:48.397121    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7cc89f4ff-7cckv"
Oct 31 17:27:48.427153 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:48.427119    4276 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Oct 31 17:27:48.513099 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:48.513060    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9f054444-3b18-4c57-89bc-1844e5dd830c-trusted-ca-bundle\") pod \"console-7cc89f4ff-7cckv\" (UID: \"9f054444-3b18-4c57-89bc-1844e5dd830c\") " pod="openshift-console/console-7cc89f4ff-7cckv"
Oct 31 17:27:48.513543 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:48.513116    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9f054444-3b18-4c57-89bc-1844e5dd830c-service-ca\") pod \"console-7cc89f4ff-7cckv\" (UID: \"9f054444-3b18-4c57-89bc-1844e5dd830c\") " pod="openshift-console/console-7cc89f4ff-7cckv"
Oct 31 17:27:48.513543 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:48.513148    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9f054444-3b18-4c57-89bc-1844e5dd830c-console-serving-cert\") pod \"console-7cc89f4ff-7cckv\" (UID: \"9f054444-3b18-4c57-89bc-1844e5dd830c\") " pod="openshift-console/console-7cc89f4ff-7cckv"
Oct 31 17:27:48.513543 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:48.513253    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9f054444-3b18-4c57-89bc-1844e5dd830c-console-oauth-config\") pod \"console-7cc89f4ff-7cckv\" (UID: \"9f054444-3b18-4c57-89bc-1844e5dd830c\") " pod="openshift-console/console-7cc89f4ff-7cckv"
Oct 31 17:27:48.513543 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:48.513282    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9f054444-3b18-4c57-89bc-1844e5dd830c-tmp\") pod \"console-7cc89f4ff-7cckv\" (UID: \"9f054444-3b18-4c57-89bc-1844e5dd830c\") " pod="openshift-console/console-7cc89f4ff-7cckv"
Oct 31 17:27:48.513543 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:48.513310    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k985r\" (UniqueName: \"kubernetes.io/projected/9f054444-3b18-4c57-89bc-1844e5dd830c-kube-api-access-k985r\") pod \"console-7cc89f4ff-7cckv\" (UID: \"9f054444-3b18-4c57-89bc-1844e5dd830c\") " pod="openshift-console/console-7cc89f4ff-7cckv"
Oct 31 17:27:48.513543 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:48.513378    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9f054444-3b18-4c57-89bc-1844e5dd830c-oauth-serving-cert\") pod \"console-7cc89f4ff-7cckv\" (UID: \"9f054444-3b18-4c57-89bc-1844e5dd830c\") " pod="openshift-console/console-7cc89f4ff-7cckv"
Oct 31 17:27:48.513543 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:48.513423    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9f054444-3b18-4c57-89bc-1844e5dd830c-console-config\") pod \"console-7cc89f4ff-7cckv\" (UID: \"9f054444-3b18-4c57-89bc-1844e5dd830c\") " pod="openshift-console/console-7cc89f4ff-7cckv"
Oct 31 17:27:48.614572 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:48.614532    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9f054444-3b18-4c57-89bc-1844e5dd830c-trusted-ca-bundle\") pod \"console-7cc89f4ff-7cckv\" (UID: \"9f054444-3b18-4c57-89bc-1844e5dd830c\") " pod="openshift-console/console-7cc89f4ff-7cckv"
Oct 31 17:27:48.614764 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:48.614582    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9f054444-3b18-4c57-89bc-1844e5dd830c-service-ca\") pod \"console-7cc89f4ff-7cckv\" (UID: \"9f054444-3b18-4c57-89bc-1844e5dd830c\") " pod="openshift-console/console-7cc89f4ff-7cckv"
Oct 31 17:27:48.614764 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:48.614614    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9f054444-3b18-4c57-89bc-1844e5dd830c-console-serving-cert\") pod \"console-7cc89f4ff-7cckv\" (UID: \"9f054444-3b18-4c57-89bc-1844e5dd830c\") " pod="openshift-console/console-7cc89f4ff-7cckv"
Oct 31 17:27:48.614764 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:48.614649    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9f054444-3b18-4c57-89bc-1844e5dd830c-console-oauth-config\") pod \"console-7cc89f4ff-7cckv\" (UID: \"9f054444-3b18-4c57-89bc-1844e5dd830c\") " pod="openshift-console/console-7cc89f4ff-7cckv"
Oct 31 17:27:48.614764 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:48.614676    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9f054444-3b18-4c57-89bc-1844e5dd830c-tmp\") pod \"console-7cc89f4ff-7cckv\" (UID: \"9f054444-3b18-4c57-89bc-1844e5dd830c\") " pod="openshift-console/console-7cc89f4ff-7cckv"
Oct 31 17:27:48.614764 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:48.614705    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k985r\" (UniqueName: \"kubernetes.io/projected/9f054444-3b18-4c57-89bc-1844e5dd830c-kube-api-access-k985r\") pod \"console-7cc89f4ff-7cckv\" (UID: \"9f054444-3b18-4c57-89bc-1844e5dd830c\") " pod="openshift-console/console-7cc89f4ff-7cckv"
Oct 31 17:27:48.614764 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:48.614734    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9f054444-3b18-4c57-89bc-1844e5dd830c-oauth-serving-cert\") pod \"console-7cc89f4ff-7cckv\" (UID: \"9f054444-3b18-4c57-89bc-1844e5dd830c\") " pod="openshift-console/console-7cc89f4ff-7cckv"
Oct 31 17:27:48.614764 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:48.614765    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9f054444-3b18-4c57-89bc-1844e5dd830c-console-config\") pod \"console-7cc89f4ff-7cckv\" (UID: \"9f054444-3b18-4c57-89bc-1844e5dd830c\") " pod="openshift-console/console-7cc89f4ff-7cckv"
Oct 31 17:27:48.615464 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:48.615413    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9f054444-3b18-4c57-89bc-1844e5dd830c-service-ca\") pod \"console-7cc89f4ff-7cckv\" (UID: \"9f054444-3b18-4c57-89bc-1844e5dd830c\") " pod="openshift-console/console-7cc89f4ff-7cckv"
Oct 31 17:27:48.616280 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:48.615436    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9f054444-3b18-4c57-89bc-1844e5dd830c-tmp\") pod \"console-7cc89f4ff-7cckv\" (UID: \"9f054444-3b18-4c57-89bc-1844e5dd830c\") " pod="openshift-console/console-7cc89f4ff-7cckv"
Oct 31 17:27:48.616280 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:48.615571    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9f054444-3b18-4c57-89bc-1844e5dd830c-console-config\") pod \"console-7cc89f4ff-7cckv\" (UID: \"9f054444-3b18-4c57-89bc-1844e5dd830c\") " pod="openshift-console/console-7cc89f4ff-7cckv"
Oct 31 17:27:48.616280 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:48.615645    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9f054444-3b18-4c57-89bc-1844e5dd830c-trusted-ca-bundle\") pod \"console-7cc89f4ff-7cckv\" (UID: \"9f054444-3b18-4c57-89bc-1844e5dd830c\") " pod="openshift-console/console-7cc89f4ff-7cckv"
Oct 31 17:27:48.616280 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:48.616185    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9f054444-3b18-4c57-89bc-1844e5dd830c-oauth-serving-cert\") pod \"console-7cc89f4ff-7cckv\" (UID: \"9f054444-3b18-4c57-89bc-1844e5dd830c\") " pod="openshift-console/console-7cc89f4ff-7cckv"
Oct 31 17:27:48.618146 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:48.618126    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9f054444-3b18-4c57-89bc-1844e5dd830c-console-serving-cert\") pod \"console-7cc89f4ff-7cckv\" (UID: \"9f054444-3b18-4c57-89bc-1844e5dd830c\") " pod="openshift-console/console-7cc89f4ff-7cckv"
Oct 31 17:27:48.618255 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:48.618185    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9f054444-3b18-4c57-89bc-1844e5dd830c-console-oauth-config\") pod \"console-7cc89f4ff-7cckv\" (UID: \"9f054444-3b18-4c57-89bc-1844e5dd830c\") " pod="openshift-console/console-7cc89f4ff-7cckv"
Oct 31 17:27:48.635916 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:48.635894    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k985r\" (UniqueName: \"kubernetes.io/projected/9f054444-3b18-4c57-89bc-1844e5dd830c-kube-api-access-k985r\") pod \"console-7cc89f4ff-7cckv\" (UID: \"9f054444-3b18-4c57-89bc-1844e5dd830c\") " pod="openshift-console/console-7cc89f4ff-7cckv"
Oct 31 17:27:48.725959 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:48.725926    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7cc89f4ff-7cckv"
Oct 31 17:27:48.776012 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:48.775975    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:27:48.884301 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:48.884204    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:27:49.067801 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:49.067770    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7cc89f4ff-7cckv"]
Oct 31 17:27:49.071428 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:27:49.071396    4276 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f054444_3b18_4c57_89bc_1844e5dd830c.slice/crio-82c1f6e870e8e8ed6b50154e5b3694cb53ab04c098fe81e39f49ede062a6ae21 WatchSource:0}: Error finding container 82c1f6e870e8e8ed6b50154e5b3694cb53ab04c098fe81e39f49ede062a6ae21: Status 404 returned error can't find the container with id 82c1f6e870e8e8ed6b50154e5b3694cb53ab04c098fe81e39f49ede062a6ae21
Oct 31 17:27:49.551834 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:49.551784    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7cc89f4ff-7cckv" event={"ID":"9f054444-3b18-4c57-89bc-1844e5dd830c","Type":"ContainerStarted","Data":"58becb04518e7bfbf7b64ab84958abc97ace18d2c55f54a22a258be7d3bd90d2"}
Oct 31 17:27:49.552302 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:49.551857    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7cc89f4ff-7cckv" event={"ID":"9f054444-3b18-4c57-89bc-1844e5dd830c","Type":"ContainerStarted","Data":"82c1f6e870e8e8ed6b50154e5b3694cb53ab04c098fe81e39f49ede062a6ae21"}
Oct 31 17:27:49.594206 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:49.594150    4276 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-7cc89f4ff-7cckv" podStartSLOduration=1.594136195 podStartE2EDuration="1.594136195s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 17:27:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 17:27:49.593192177 +0000 UTC m=+32.390937105" watchObservedRunningTime="2026-10-31 17:27:49.594136195 +0000 UTC m=+32.391881123"
Oct 31 17:27:51.355144 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:51.355109    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:27:51.467133 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:51.467039    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:27:51.471082 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:51.471061    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-865894c87d-kktxm"
Oct 31 17:27:51.681506 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:51.681423    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:27:51.784800 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:51.784745    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:27:53.163703 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:53.163650    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:27:53.271577 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:53.271543    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:27:54.317989 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:54.317959    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-tskzw"
Oct 31 17:27:55.332122 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:55.332063    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:27:55.443197 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:55.443159    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:27:55.996571 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:55.996541    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:27:56.101148 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:56.101116    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:27:57.533388 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:57.533357    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:27:58.726459 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:58.726411    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7cc89f4ff-7cckv"
Oct 31 17:27:58.726907 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:58.726489    4276 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7cc89f4ff-7cckv"
Oct 31 17:27:58.732046 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:58.732023    4276 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7cc89f4ff-7cckv"
Oct 31 17:27:58.934029 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:58.933980    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:27:59.046710 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:59.046598    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:27:59.596918 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:59.596892    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7cc89f4ff-7cckv"
Oct 31 17:27:59.663779 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:27:59.663750    4276 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-598df4d98d-2vp6w"]
Oct 31 17:28:01.319935 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:28:01.319770    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:28:01.531573 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:28:01.531539    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:28:02.805713 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:28:02.805676    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:28:10.994318 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:28:10.994277    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:28:11.105347 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:28:11.105314    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:28:12.233276 ip-10-0-129-127 kubenswrapper[4276]: E1031 17:28:12.233245    4276 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tg5qk" containerName="extractor"
Oct 31 17:28:19.883147 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:28:19.883114    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:28:19.999951 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:28:19.999919    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:28:20.929434 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:28:20.929400    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:28:21.040350 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:28:21.040319    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:28:23.815149 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:28:23.815114    4276 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-dbf8bf5bf-j2wb6"]
Oct 31 17:28:23.849718 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:28:23.849690    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-dbf8bf5bf-j2wb6"
Oct 31 17:28:23.855608 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:28:23.855572    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-dbf8bf5bf-j2wb6"]
Oct 31 17:28:23.968392 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:28:23.968367    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/20547067-6a08-4783-84e1-de23febc6893-trusted-ca-bundle\") pod \"console-dbf8bf5bf-j2wb6\" (UID: \"20547067-6a08-4783-84e1-de23febc6893\") " pod="openshift-console/console-dbf8bf5bf-j2wb6"
Oct 31 17:28:23.968392 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:28:23.968399    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/20547067-6a08-4783-84e1-de23febc6893-console-oauth-config\") pod \"console-dbf8bf5bf-j2wb6\" (UID: \"20547067-6a08-4783-84e1-de23febc6893\") " pod="openshift-console/console-dbf8bf5bf-j2wb6"
Oct 31 17:28:23.968567 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:28:23.968423    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k6n8d\" (UniqueName: \"kubernetes.io/projected/20547067-6a08-4783-84e1-de23febc6893-kube-api-access-k6n8d\") pod \"console-dbf8bf5bf-j2wb6\" (UID: \"20547067-6a08-4783-84e1-de23febc6893\") " pod="openshift-console/console-dbf8bf5bf-j2wb6"
Oct 31 17:28:23.968567 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:28:23.968543    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/20547067-6a08-4783-84e1-de23febc6893-console-serving-cert\") pod \"console-dbf8bf5bf-j2wb6\" (UID: \"20547067-6a08-4783-84e1-de23febc6893\") " pod="openshift-console/console-dbf8bf5bf-j2wb6"
Oct 31 17:28:23.968634 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:28:23.968584    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/20547067-6a08-4783-84e1-de23febc6893-console-config\") pod \"console-dbf8bf5bf-j2wb6\" (UID: \"20547067-6a08-4783-84e1-de23febc6893\") " pod="openshift-console/console-dbf8bf5bf-j2wb6"
Oct 31 17:28:23.968634 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:28:23.968606    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/20547067-6a08-4783-84e1-de23febc6893-tmp\") pod \"console-dbf8bf5bf-j2wb6\" (UID: \"20547067-6a08-4783-84e1-de23febc6893\") " pod="openshift-console/console-dbf8bf5bf-j2wb6"
Oct 31 17:28:23.968693 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:28:23.968641    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/20547067-6a08-4783-84e1-de23febc6893-oauth-serving-cert\") pod \"console-dbf8bf5bf-j2wb6\" (UID: \"20547067-6a08-4783-84e1-de23febc6893\") " pod="openshift-console/console-dbf8bf5bf-j2wb6"
Oct 31 17:28:23.968727 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:28:23.968696    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/20547067-6a08-4783-84e1-de23febc6893-service-ca\") pod \"console-dbf8bf5bf-j2wb6\" (UID: \"20547067-6a08-4783-84e1-de23febc6893\") " pod="openshift-console/console-dbf8bf5bf-j2wb6"
Oct 31 17:28:23.979788 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:28:23.979764    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:28:24.069558 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:28:24.069509    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/20547067-6a08-4783-84e1-de23febc6893-trusted-ca-bundle\") pod \"console-dbf8bf5bf-j2wb6\" (UID: \"20547067-6a08-4783-84e1-de23febc6893\") " pod="openshift-console/console-dbf8bf5bf-j2wb6"
Oct 31 17:28:24.069558 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:28:24.069539    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/20547067-6a08-4783-84e1-de23febc6893-console-oauth-config\") pod \"console-dbf8bf5bf-j2wb6\" (UID: \"20547067-6a08-4783-84e1-de23febc6893\") " pod="openshift-console/console-dbf8bf5bf-j2wb6"
Oct 31 17:28:24.069558 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:28:24.069555    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k6n8d\" (UniqueName: \"kubernetes.io/projected/20547067-6a08-4783-84e1-de23febc6893-kube-api-access-k6n8d\") pod \"console-dbf8bf5bf-j2wb6\" (UID: \"20547067-6a08-4783-84e1-de23febc6893\") " pod="openshift-console/console-dbf8bf5bf-j2wb6"
Oct 31 17:28:24.069735 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:28:24.069575    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/20547067-6a08-4783-84e1-de23febc6893-console-serving-cert\") pod \"console-dbf8bf5bf-j2wb6\" (UID: \"20547067-6a08-4783-84e1-de23febc6893\") " pod="openshift-console/console-dbf8bf5bf-j2wb6"
Oct 31 17:28:24.069735 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:28:24.069600    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/20547067-6a08-4783-84e1-de23febc6893-console-config\") pod \"console-dbf8bf5bf-j2wb6\" (UID: \"20547067-6a08-4783-84e1-de23febc6893\") " pod="openshift-console/console-dbf8bf5bf-j2wb6"
Oct 31 17:28:24.069735 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:28:24.069618    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/20547067-6a08-4783-84e1-de23febc6893-tmp\") pod \"console-dbf8bf5bf-j2wb6\" (UID: \"20547067-6a08-4783-84e1-de23febc6893\") " pod="openshift-console/console-dbf8bf5bf-j2wb6"
Oct 31 17:28:24.069735 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:28:24.069638    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/20547067-6a08-4783-84e1-de23febc6893-oauth-serving-cert\") pod \"console-dbf8bf5bf-j2wb6\" (UID: \"20547067-6a08-4783-84e1-de23febc6893\") " pod="openshift-console/console-dbf8bf5bf-j2wb6"
Oct 31 17:28:24.069735 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:28:24.069674    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/20547067-6a08-4783-84e1-de23febc6893-service-ca\") pod \"console-dbf8bf5bf-j2wb6\" (UID: \"20547067-6a08-4783-84e1-de23febc6893\") " pod="openshift-console/console-dbf8bf5bf-j2wb6"
Oct 31 17:28:24.070020 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:28:24.069998    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/20547067-6a08-4783-84e1-de23febc6893-tmp\") pod \"console-dbf8bf5bf-j2wb6\" (UID: \"20547067-6a08-4783-84e1-de23febc6893\") " pod="openshift-console/console-dbf8bf5bf-j2wb6"
Oct 31 17:28:24.070344 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:28:24.070324    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/20547067-6a08-4783-84e1-de23febc6893-console-config\") pod \"console-dbf8bf5bf-j2wb6\" (UID: \"20547067-6a08-4783-84e1-de23febc6893\") " pod="openshift-console/console-dbf8bf5bf-j2wb6"
Oct 31 17:28:24.070432 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:28:24.070409    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/20547067-6a08-4783-84e1-de23febc6893-service-ca\") pod \"console-dbf8bf5bf-j2wb6\" (UID: \"20547067-6a08-4783-84e1-de23febc6893\") " pod="openshift-console/console-dbf8bf5bf-j2wb6"
Oct 31 17:28:24.070544 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:28:24.070525    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/20547067-6a08-4783-84e1-de23febc6893-oauth-serving-cert\") pod \"console-dbf8bf5bf-j2wb6\" (UID: \"20547067-6a08-4783-84e1-de23febc6893\") " pod="openshift-console/console-dbf8bf5bf-j2wb6"
Oct 31 17:28:24.070618 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:28:24.070601    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/20547067-6a08-4783-84e1-de23febc6893-trusted-ca-bundle\") pod \"console-dbf8bf5bf-j2wb6\" (UID: \"20547067-6a08-4783-84e1-de23febc6893\") " pod="openshift-console/console-dbf8bf5bf-j2wb6"
Oct 31 17:28:24.072550 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:28:24.072528    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/20547067-6a08-4783-84e1-de23febc6893-console-serving-cert\") pod \"console-dbf8bf5bf-j2wb6\" (UID: \"20547067-6a08-4783-84e1-de23febc6893\") " pod="openshift-console/console-dbf8bf5bf-j2wb6"
Oct 31 17:28:24.072696 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:28:24.072674    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/20547067-6a08-4783-84e1-de23febc6893-console-oauth-config\") pod \"console-dbf8bf5bf-j2wb6\" (UID: \"20547067-6a08-4783-84e1-de23febc6893\") " pod="openshift-console/console-dbf8bf5bf-j2wb6"
Oct 31 17:28:24.089872 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:28:24.089852    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k6n8d\" (UniqueName: \"kubernetes.io/projected/20547067-6a08-4783-84e1-de23febc6893-kube-api-access-k6n8d\") pod \"console-dbf8bf5bf-j2wb6\" (UID: \"20547067-6a08-4783-84e1-de23febc6893\") " pod="openshift-console/console-dbf8bf5bf-j2wb6"
Oct 31 17:28:24.093130 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:28:24.093100    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:28:24.179854 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:28:24.179831    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-dbf8bf5bf-j2wb6"
Oct 31 17:28:24.274549 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:28:24.274482    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:28:24.383139 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:28:24.383081    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:28:24.516865 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:28:24.516837    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-dbf8bf5bf-j2wb6"]
Oct 31 17:28:24.520337 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:28:24.520313    4276 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20547067_6a08_4783_84e1_de23febc6893.slice/crio-70331ba2054b68c8ffb859de198ce8b9493e23d0f4efc17a5e2a32cda552d565 WatchSource:0}: Error finding container 70331ba2054b68c8ffb859de198ce8b9493e23d0f4efc17a5e2a32cda552d565: Status 404 returned error can't find the container with id 70331ba2054b68c8ffb859de198ce8b9493e23d0f4efc17a5e2a32cda552d565
Oct 31 17:28:24.674235 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:28:24.674158    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-dbf8bf5bf-j2wb6" event={"ID":"20547067-6a08-4783-84e1-de23febc6893","Type":"ContainerStarted","Data":"6efed365ae617ef414f62be6f745e059d90b66124b48743b3cf5014e69a92e0b"}
Oct 31 17:28:24.674235 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:28:24.674215    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-dbf8bf5bf-j2wb6" event={"ID":"20547067-6a08-4783-84e1-de23febc6893","Type":"ContainerStarted","Data":"70331ba2054b68c8ffb859de198ce8b9493e23d0f4efc17a5e2a32cda552d565"}
Oct 31 17:28:24.683594 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:28:24.683550    4276 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-console/console-598df4d98d-2vp6w" podUID="92766ec3-7459-4dd1-894c-e15ad29b98ed" containerName="console" containerID="cri-o://86bc69a323a9af0e2726ecda1e36b29dcedcac578bf32ac14c68bb4a9052a313" gracePeriod=15
Oct 31 17:28:24.718359 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:28:24.718320    4276 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-dbf8bf5bf-j2wb6" podStartSLOduration=1.718307821 podStartE2EDuration="1.718307821s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 17:28:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 17:28:24.718027774 +0000 UTC m=+67.515772702" watchObservedRunningTime="2026-10-31 17:28:24.718307821 +0000 UTC m=+67.516052788"
Oct 31 17:28:24.943879 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:28:24.943823    4276 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-598df4d98d-2vp6w"
Oct 31 17:28:24.977486 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:28:24.977458    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/92766ec3-7459-4dd1-894c-e15ad29b98ed-console-oauth-config\" (UniqueName: \"kubernetes.io/secret/92766ec3-7459-4dd1-894c-e15ad29b98ed-console-oauth-config\") pod \"92766ec3-7459-4dd1-894c-e15ad29b98ed\" (UID: \"92766ec3-7459-4dd1-894c-e15ad29b98ed\") "
Oct 31 17:28:24.977609 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:28:24.977499    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/92766ec3-7459-4dd1-894c-e15ad29b98ed-console-config\" (UniqueName: \"kubernetes.io/configmap/92766ec3-7459-4dd1-894c-e15ad29b98ed-console-config\") pod \"92766ec3-7459-4dd1-894c-e15ad29b98ed\" (UID: \"92766ec3-7459-4dd1-894c-e15ad29b98ed\") "
Oct 31 17:28:24.977655 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:28:24.977610    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/92766ec3-7459-4dd1-894c-e15ad29b98ed-oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/92766ec3-7459-4dd1-894c-e15ad29b98ed-oauth-serving-cert\") pod \"92766ec3-7459-4dd1-894c-e15ad29b98ed\" (UID: \"92766ec3-7459-4dd1-894c-e15ad29b98ed\") "
Oct 31 17:28:24.977697 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:28:24.977656    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/92766ec3-7459-4dd1-894c-e15ad29b98ed-kube-api-access-9plzn\" (UniqueName: \"kubernetes.io/projected/92766ec3-7459-4dd1-894c-e15ad29b98ed-kube-api-access-9plzn\") pod \"92766ec3-7459-4dd1-894c-e15ad29b98ed\" (UID: \"92766ec3-7459-4dd1-894c-e15ad29b98ed\") "
Oct 31 17:28:24.977697 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:28:24.977689    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/92766ec3-7459-4dd1-894c-e15ad29b98ed-console-serving-cert\" (UniqueName: \"kubernetes.io/secret/92766ec3-7459-4dd1-894c-e15ad29b98ed-console-serving-cert\") pod \"92766ec3-7459-4dd1-894c-e15ad29b98ed\" (UID: \"92766ec3-7459-4dd1-894c-e15ad29b98ed\") "
Oct 31 17:28:24.977780 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:28:24.977728    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/92766ec3-7459-4dd1-894c-e15ad29b98ed-service-ca\" (UniqueName: \"kubernetes.io/configmap/92766ec3-7459-4dd1-894c-e15ad29b98ed-service-ca\") pod \"92766ec3-7459-4dd1-894c-e15ad29b98ed\" (UID: \"92766ec3-7459-4dd1-894c-e15ad29b98ed\") "
Oct 31 17:28:24.977780 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:28:24.977769    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/92766ec3-7459-4dd1-894c-e15ad29b98ed-tmp\" (UniqueName: \"kubernetes.io/empty-dir/92766ec3-7459-4dd1-894c-e15ad29b98ed-tmp\") pod \"92766ec3-7459-4dd1-894c-e15ad29b98ed\" (UID: \"92766ec3-7459-4dd1-894c-e15ad29b98ed\") "
Oct 31 17:28:24.977947 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:28:24.977924    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/92766ec3-7459-4dd1-894c-e15ad29b98ed-console-config" pod "92766ec3-7459-4dd1-894c-e15ad29b98ed" (UID: "92766ec3-7459-4dd1-894c-e15ad29b98ed"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 17:28:24.978076 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:28:24.978036    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/92766ec3-7459-4dd1-894c-e15ad29b98ed-oauth-serving-cert" pod "92766ec3-7459-4dd1-894c-e15ad29b98ed" (UID: "92766ec3-7459-4dd1-894c-e15ad29b98ed"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 17:28:24.978154 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:28:24.978137    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/92766ec3-7459-4dd1-894c-e15ad29b98ed-tmp" pod "92766ec3-7459-4dd1-894c-e15ad29b98ed" (UID: "92766ec3-7459-4dd1-894c-e15ad29b98ed"). InnerVolumeSpecName "tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:28:24.978204 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:28:24.978157    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/92766ec3-7459-4dd1-894c-e15ad29b98ed-service-ca" pod "92766ec3-7459-4dd1-894c-e15ad29b98ed" (UID: "92766ec3-7459-4dd1-894c-e15ad29b98ed"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 17:28:24.979719 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:28:24.979699    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/92766ec3-7459-4dd1-894c-e15ad29b98ed-console-oauth-config" pod "92766ec3-7459-4dd1-894c-e15ad29b98ed" (UID: "92766ec3-7459-4dd1-894c-e15ad29b98ed"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 17:28:24.980175 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:28:24.980154    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/92766ec3-7459-4dd1-894c-e15ad29b98ed-console-serving-cert" pod "92766ec3-7459-4dd1-894c-e15ad29b98ed" (UID: "92766ec3-7459-4dd1-894c-e15ad29b98ed"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 17:28:24.980175 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:28:24.980168    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/92766ec3-7459-4dd1-894c-e15ad29b98ed-kube-api-access-9plzn" pod "92766ec3-7459-4dd1-894c-e15ad29b98ed" (UID: "92766ec3-7459-4dd1-894c-e15ad29b98ed"). InnerVolumeSpecName "kube-api-access-9plzn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 17:28:25.079211 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:28:25.079187    4276 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/92766ec3-7459-4dd1-894c-e15ad29b98ed-console-serving-cert\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:28:25.079211 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:28:25.079210    4276 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/92766ec3-7459-4dd1-894c-e15ad29b98ed-service-ca\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:28:25.079211 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:28:25.079222    4276 reconciler_common.go:299] "Volume detached for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/92766ec3-7459-4dd1-894c-e15ad29b98ed-tmp\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:28:25.079435 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:28:25.079236    4276 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/92766ec3-7459-4dd1-894c-e15ad29b98ed-console-oauth-config\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:28:25.079435 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:28:25.079245    4276 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/92766ec3-7459-4dd1-894c-e15ad29b98ed-console-config\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:28:25.079435 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:28:25.079253    4276 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/92766ec3-7459-4dd1-894c-e15ad29b98ed-oauth-serving-cert\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:28:25.079435 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:28:25.079266    4276 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9plzn\" (UniqueName: \"kubernetes.io/projected/92766ec3-7459-4dd1-894c-e15ad29b98ed-kube-api-access-9plzn\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:28:25.678702 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:28:25.678623    4276 generic.go:356] "Generic (PLEG): container finished" podID="92766ec3-7459-4dd1-894c-e15ad29b98ed" containerID="86bc69a323a9af0e2726ecda1e36b29dcedcac578bf32ac14c68bb4a9052a313" exitCode=0
Oct 31 17:28:25.678702 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:28:25.678674    4276 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-598df4d98d-2vp6w"
Oct 31 17:28:25.679044 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:28:25.678707    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-598df4d98d-2vp6w" event={"ID":"92766ec3-7459-4dd1-894c-e15ad29b98ed","Type":"ContainerDied","Data":"86bc69a323a9af0e2726ecda1e36b29dcedcac578bf32ac14c68bb4a9052a313"}
Oct 31 17:28:25.679044 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:28:25.678774    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-598df4d98d-2vp6w" event={"ID":"92766ec3-7459-4dd1-894c-e15ad29b98ed","Type":"ContainerDied","Data":"3bc4e9ba11777481151cc2dd04c9602472c65e04da2b692a838f75c5d5fe7dce"}
Oct 31 17:28:25.679044 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:28:25.678794    4276 scope.go:122] "RemoveContainer" containerID="86bc69a323a9af0e2726ecda1e36b29dcedcac578bf32ac14c68bb4a9052a313"
Oct 31 17:28:25.686538 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:28:25.686521    4276 scope.go:122] "RemoveContainer" containerID="86bc69a323a9af0e2726ecda1e36b29dcedcac578bf32ac14c68bb4a9052a313"
Oct 31 17:28:25.686797 ip-10-0-129-127 kubenswrapper[4276]: E1031 17:28:25.686778    4276 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"86bc69a323a9af0e2726ecda1e36b29dcedcac578bf32ac14c68bb4a9052a313\": container with ID starting with 86bc69a323a9af0e2726ecda1e36b29dcedcac578bf32ac14c68bb4a9052a313 not found: ID does not exist" containerID="86bc69a323a9af0e2726ecda1e36b29dcedcac578bf32ac14c68bb4a9052a313"
Oct 31 17:28:25.686869 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:28:25.686807    4276 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"86bc69a323a9af0e2726ecda1e36b29dcedcac578bf32ac14c68bb4a9052a313"} err="failed to get container status \"86bc69a323a9af0e2726ecda1e36b29dcedcac578bf32ac14c68bb4a9052a313\": rpc error: code = NotFound desc = could not find container \"86bc69a323a9af0e2726ecda1e36b29dcedcac578bf32ac14c68bb4a9052a313\": container with ID starting with 86bc69a323a9af0e2726ecda1e36b29dcedcac578bf32ac14c68bb4a9052a313 not found: ID does not exist"
Oct 31 17:28:25.732056 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:28:25.732034    4276 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-598df4d98d-2vp6w"]
Oct 31 17:28:25.745917 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:28:25.745895    4276 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-598df4d98d-2vp6w"]
Oct 31 17:28:27.364740 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:28:27.364706    4276 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="92766ec3-7459-4dd1-894c-e15ad29b98ed" path="/var/lib/kubelet/pods/92766ec3-7459-4dd1-894c-e15ad29b98ed/volumes"
Oct 31 17:28:33.963792 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:28:33.963756    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:28:34.074529 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:28:34.074498    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:28:34.180261 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:28:34.180237    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-dbf8bf5bf-j2wb6"
Oct 31 17:28:34.180378 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:28:34.180288    4276 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-dbf8bf5bf-j2wb6"
Oct 31 17:28:34.184981 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:28:34.184962    4276 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-dbf8bf5bf-j2wb6"
Oct 31 17:28:34.717433 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:28:34.717397    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-dbf8bf5bf-j2wb6"
Oct 31 17:28:34.781682 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:28:34.781653    4276 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7cc89f4ff-7cckv"]
Oct 31 17:28:40.992489 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:28:40.992456    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:28:41.097962 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:28:41.097938    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:28:42.233200 ip-10-0-129-127 kubenswrapper[4276]: E1031 17:28:42.233167    4276 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tg5qk" containerName="extractor"
Oct 31 17:28:59.800317 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:28:59.800281    4276 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-console/console-7cc89f4ff-7cckv" podUID="9f054444-3b18-4c57-89bc-1844e5dd830c" containerName="console" containerID="cri-o://58becb04518e7bfbf7b64ab84958abc97ace18d2c55f54a22a258be7d3bd90d2" gracePeriod=15
Oct 31 17:29:00.042637 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:29:00.042611    4276 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7cc89f4ff-7cckv"
Oct 31 17:29:00.123957 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:29:00.123893    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9f054444-3b18-4c57-89bc-1844e5dd830c-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9f054444-3b18-4c57-89bc-1844e5dd830c-trusted-ca-bundle\") pod \"9f054444-3b18-4c57-89bc-1844e5dd830c\" (UID: \"9f054444-3b18-4c57-89bc-1844e5dd830c\") "
Oct 31 17:29:00.124075 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:29:00.123962    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9f054444-3b18-4c57-89bc-1844e5dd830c-kube-api-access-k985r\" (UniqueName: \"kubernetes.io/projected/9f054444-3b18-4c57-89bc-1844e5dd830c-kube-api-access-k985r\") pod \"9f054444-3b18-4c57-89bc-1844e5dd830c\" (UID: \"9f054444-3b18-4c57-89bc-1844e5dd830c\") "
Oct 31 17:29:00.124075 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:29:00.124004    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9f054444-3b18-4c57-89bc-1844e5dd830c-oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9f054444-3b18-4c57-89bc-1844e5dd830c-oauth-serving-cert\") pod \"9f054444-3b18-4c57-89bc-1844e5dd830c\" (UID: \"9f054444-3b18-4c57-89bc-1844e5dd830c\") "
Oct 31 17:29:00.124075 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:29:00.124048    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9f054444-3b18-4c57-89bc-1844e5dd830c-console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9f054444-3b18-4c57-89bc-1844e5dd830c-console-oauth-config\") pod \"9f054444-3b18-4c57-89bc-1844e5dd830c\" (UID: \"9f054444-3b18-4c57-89bc-1844e5dd830c\") "
Oct 31 17:29:00.124170 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:29:00.124123    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9f054444-3b18-4c57-89bc-1844e5dd830c-service-ca\" (UniqueName: \"kubernetes.io/configmap/9f054444-3b18-4c57-89bc-1844e5dd830c-service-ca\") pod \"9f054444-3b18-4c57-89bc-1844e5dd830c\" (UID: \"9f054444-3b18-4c57-89bc-1844e5dd830c\") "
Oct 31 17:29:00.124170 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:29:00.124149    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9f054444-3b18-4c57-89bc-1844e5dd830c-tmp\" (UniqueName: \"kubernetes.io/empty-dir/9f054444-3b18-4c57-89bc-1844e5dd830c-tmp\") pod \"9f054444-3b18-4c57-89bc-1844e5dd830c\" (UID: \"9f054444-3b18-4c57-89bc-1844e5dd830c\") "
Oct 31 17:29:00.124268 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:29:00.124177    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9f054444-3b18-4c57-89bc-1844e5dd830c-console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9f054444-3b18-4c57-89bc-1844e5dd830c-console-serving-cert\") pod \"9f054444-3b18-4c57-89bc-1844e5dd830c\" (UID: \"9f054444-3b18-4c57-89bc-1844e5dd830c\") "
Oct 31 17:29:00.124268 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:29:00.124217    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9f054444-3b18-4c57-89bc-1844e5dd830c-console-config\" (UniqueName: \"kubernetes.io/configmap/9f054444-3b18-4c57-89bc-1844e5dd830c-console-config\") pod \"9f054444-3b18-4c57-89bc-1844e5dd830c\" (UID: \"9f054444-3b18-4c57-89bc-1844e5dd830c\") "
Oct 31 17:29:00.124370 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:29:00.124337    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9f054444-3b18-4c57-89bc-1844e5dd830c-trusted-ca-bundle" pod "9f054444-3b18-4c57-89bc-1844e5dd830c" (UID: "9f054444-3b18-4c57-89bc-1844e5dd830c"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 17:29:00.124562 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:29:00.124531    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f054444-3b18-4c57-89bc-1844e5dd830c-tmp" pod "9f054444-3b18-4c57-89bc-1844e5dd830c" (UID: "9f054444-3b18-4c57-89bc-1844e5dd830c"). InnerVolumeSpecName "tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:29:00.124562 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:29:00.124557    4276 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9f054444-3b18-4c57-89bc-1844e5dd830c-trusted-ca-bundle\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:29:00.124712 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:29:00.124560    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9f054444-3b18-4c57-89bc-1844e5dd830c-oauth-serving-cert" pod "9f054444-3b18-4c57-89bc-1844e5dd830c" (UID: "9f054444-3b18-4c57-89bc-1844e5dd830c"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 17:29:00.124712 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:29:00.124660    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9f054444-3b18-4c57-89bc-1844e5dd830c-console-config" pod "9f054444-3b18-4c57-89bc-1844e5dd830c" (UID: "9f054444-3b18-4c57-89bc-1844e5dd830c"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 17:29:00.124849 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:29:00.124829    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9f054444-3b18-4c57-89bc-1844e5dd830c-service-ca" pod "9f054444-3b18-4c57-89bc-1844e5dd830c" (UID: "9f054444-3b18-4c57-89bc-1844e5dd830c"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 17:29:00.126156 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:29:00.126136    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9f054444-3b18-4c57-89bc-1844e5dd830c-console-oauth-config" pod "9f054444-3b18-4c57-89bc-1844e5dd830c" (UID: "9f054444-3b18-4c57-89bc-1844e5dd830c"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 17:29:00.126267 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:29:00.126242    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9f054444-3b18-4c57-89bc-1844e5dd830c-kube-api-access-k985r" pod "9f054444-3b18-4c57-89bc-1844e5dd830c" (UID: "9f054444-3b18-4c57-89bc-1844e5dd830c"). InnerVolumeSpecName "kube-api-access-k985r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 17:29:00.126325 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:29:00.126303    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9f054444-3b18-4c57-89bc-1844e5dd830c-console-serving-cert" pod "9f054444-3b18-4c57-89bc-1844e5dd830c" (UID: "9f054444-3b18-4c57-89bc-1844e5dd830c"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 17:29:00.225134 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:29:00.225105    4276 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9f054444-3b18-4c57-89bc-1844e5dd830c-service-ca\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:29:00.225134 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:29:00.225127    4276 reconciler_common.go:299] "Volume detached for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9f054444-3b18-4c57-89bc-1844e5dd830c-tmp\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:29:00.225134 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:29:00.225136    4276 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9f054444-3b18-4c57-89bc-1844e5dd830c-console-serving-cert\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:29:00.225313 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:29:00.225144    4276 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9f054444-3b18-4c57-89bc-1844e5dd830c-console-config\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:29:00.225313 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:29:00.225153    4276 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k985r\" (UniqueName: \"kubernetes.io/projected/9f054444-3b18-4c57-89bc-1844e5dd830c-kube-api-access-k985r\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:29:00.225313 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:29:00.225161    4276 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9f054444-3b18-4c57-89bc-1844e5dd830c-oauth-serving-cert\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:29:00.225313 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:29:00.225170    4276 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9f054444-3b18-4c57-89bc-1844e5dd830c-console-oauth-config\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:29:00.796942 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:29:00.796911    4276 generic.go:356] "Generic (PLEG): container finished" podID="9f054444-3b18-4c57-89bc-1844e5dd830c" containerID="58becb04518e7bfbf7b64ab84958abc97ace18d2c55f54a22a258be7d3bd90d2" exitCode=0
Oct 31 17:29:00.797104 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:29:00.796979    4276 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7cc89f4ff-7cckv"
Oct 31 17:29:00.797104 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:29:00.796998    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7cc89f4ff-7cckv" event={"ID":"9f054444-3b18-4c57-89bc-1844e5dd830c","Type":"ContainerDied","Data":"58becb04518e7bfbf7b64ab84958abc97ace18d2c55f54a22a258be7d3bd90d2"}
Oct 31 17:29:00.797104 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:29:00.797058    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7cc89f4ff-7cckv" event={"ID":"9f054444-3b18-4c57-89bc-1844e5dd830c","Type":"ContainerDied","Data":"82c1f6e870e8e8ed6b50154e5b3694cb53ab04c098fe81e39f49ede062a6ae21"}
Oct 31 17:29:00.797104 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:29:00.797084    4276 scope.go:122] "RemoveContainer" containerID="58becb04518e7bfbf7b64ab84958abc97ace18d2c55f54a22a258be7d3bd90d2"
Oct 31 17:29:00.805424 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:29:00.805263    4276 scope.go:122] "RemoveContainer" containerID="58becb04518e7bfbf7b64ab84958abc97ace18d2c55f54a22a258be7d3bd90d2"
Oct 31 17:29:00.805685 ip-10-0-129-127 kubenswrapper[4276]: E1031 17:29:00.805540    4276 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"58becb04518e7bfbf7b64ab84958abc97ace18d2c55f54a22a258be7d3bd90d2\": container with ID starting with 58becb04518e7bfbf7b64ab84958abc97ace18d2c55f54a22a258be7d3bd90d2 not found: ID does not exist" containerID="58becb04518e7bfbf7b64ab84958abc97ace18d2c55f54a22a258be7d3bd90d2"
Oct 31 17:29:00.805685 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:29:00.805563    4276 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"58becb04518e7bfbf7b64ab84958abc97ace18d2c55f54a22a258be7d3bd90d2"} err="failed to get container status \"58becb04518e7bfbf7b64ab84958abc97ace18d2c55f54a22a258be7d3bd90d2\": rpc error: code = NotFound desc = could not find container \"58becb04518e7bfbf7b64ab84958abc97ace18d2c55f54a22a258be7d3bd90d2\": container with ID starting with 58becb04518e7bfbf7b64ab84958abc97ace18d2c55f54a22a258be7d3bd90d2 not found: ID does not exist"
Oct 31 17:29:00.850936 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:29:00.850914    4276 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7cc89f4ff-7cckv"]
Oct 31 17:29:00.864178 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:29:00.864157    4276 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7cc89f4ff-7cckv"]
Oct 31 17:29:01.364612 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:29:01.364582    4276 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9f054444-3b18-4c57-89bc-1844e5dd830c" path="/var/lib/kubelet/pods/9f054444-3b18-4c57-89bc-1844e5dd830c/volumes"
Oct 31 17:29:01.426992 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:29:01.426958    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:29:05.415401 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:29:05.415362    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:29:12.232719 ip-10-0-129-127 kubenswrapper[4276]: E1031 17:29:12.232675    4276 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tg5qk" containerName="extractor"
Oct 31 17:29:42.233256 ip-10-0-129-127 kubenswrapper[4276]: E1031 17:29:42.233176    4276 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tg5qk" containerName="extractor"
Oct 31 17:30:01.529549 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:30:01.529512    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:30:05.406650 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:30:05.406612    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:30:05.552493 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:30:05.552457    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:30:05.696483 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:30:05.696368    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:30:06.323693 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:30:06.323666    4276 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644rq55d"]
Oct 31 17:30:06.323940 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:30:06.323927    4276 cpu_manager.go:414] "containerMap: removing container" podUID="92766ec3-7459-4dd1-894c-e15ad29b98ed" containerName="console"
Oct 31 17:30:06.323988 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:30:06.323942    4276 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="92766ec3-7459-4dd1-894c-e15ad29b98ed" containerName="console"
Oct 31 17:30:06.323988 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:30:06.323959    4276 cpu_manager.go:414] "containerMap: removing container" podUID="9f054444-3b18-4c57-89bc-1844e5dd830c" containerName="console"
Oct 31 17:30:06.323988 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:30:06.323964    4276 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9f054444-3b18-4c57-89bc-1844e5dd830c" containerName="console"
Oct 31 17:30:06.324076 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:30:06.324002    4276 memory_manager.go:365] "RemoveStaleState removing state" podUID="92766ec3-7459-4dd1-894c-e15ad29b98ed" containerName="console"
Oct 31 17:30:06.324076 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:30:06.324011    4276 memory_manager.go:365] "RemoveStaleState removing state" podUID="9f054444-3b18-4c57-89bc-1844e5dd830c" containerName="console"
Oct 31 17:30:06.326823 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:30:06.326806    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644rq55d"
Oct 31 17:30:06.359377 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:30:06.359352    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"c4959aeb960c8e20c88ac2de924a35b8221494-145dcd5a-dockercfg-rd8lp\""
Oct 31 17:30:06.359377 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:30:06.359364    4276 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Oct 31 17:30:06.359582 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:30:06.359403    4276 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Oct 31 17:30:06.360540 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:30:06.360519    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d6b94c15-6e6e-4fa1-9f92-3b6046d2e973-util\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644rq55d\" (UID: \"d6b94c15-6e6e-4fa1-9f92-3b6046d2e973\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644rq55d"
Oct 31 17:30:06.360633 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:30:06.360548    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d6b94c15-6e6e-4fa1-9f92-3b6046d2e973-bundle\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644rq55d\" (UID: \"d6b94c15-6e6e-4fa1-9f92-3b6046d2e973\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644rq55d"
Oct 31 17:30:06.360633 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:30:06.360574    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pr8kn\" (UniqueName: \"kubernetes.io/projected/d6b94c15-6e6e-4fa1-9f92-3b6046d2e973-kube-api-access-pr8kn\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644rq55d\" (UID: \"d6b94c15-6e6e-4fa1-9f92-3b6046d2e973\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644rq55d"
Oct 31 17:30:06.360849 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:30:06.360825    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644rq55d"]
Oct 31 17:30:06.461677 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:30:06.461636    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d6b94c15-6e6e-4fa1-9f92-3b6046d2e973-util\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644rq55d\" (UID: \"d6b94c15-6e6e-4fa1-9f92-3b6046d2e973\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644rq55d"
Oct 31 17:30:06.461677 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:30:06.461680    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d6b94c15-6e6e-4fa1-9f92-3b6046d2e973-bundle\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644rq55d\" (UID: \"d6b94c15-6e6e-4fa1-9f92-3b6046d2e973\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644rq55d"
Oct 31 17:30:06.462058 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:30:06.461710    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pr8kn\" (UniqueName: \"kubernetes.io/projected/d6b94c15-6e6e-4fa1-9f92-3b6046d2e973-kube-api-access-pr8kn\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644rq55d\" (UID: \"d6b94c15-6e6e-4fa1-9f92-3b6046d2e973\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644rq55d"
Oct 31 17:30:06.462058 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:30:06.461985    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d6b94c15-6e6e-4fa1-9f92-3b6046d2e973-util\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644rq55d\" (UID: \"d6b94c15-6e6e-4fa1-9f92-3b6046d2e973\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644rq55d"
Oct 31 17:30:06.462058 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:30:06.462017    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d6b94c15-6e6e-4fa1-9f92-3b6046d2e973-bundle\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644rq55d\" (UID: \"d6b94c15-6e6e-4fa1-9f92-3b6046d2e973\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644rq55d"
Oct 31 17:30:06.482390 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:30:06.482362    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pr8kn\" (UniqueName: \"kubernetes.io/projected/d6b94c15-6e6e-4fa1-9f92-3b6046d2e973-kube-api-access-pr8kn\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644rq55d\" (UID: \"d6b94c15-6e6e-4fa1-9f92-3b6046d2e973\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644rq55d"
Oct 31 17:30:06.642340 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:30:06.642274    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644rq55d"
Oct 31 17:30:06.650037 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:30:06.650012    4276 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-9pf62"]
Oct 31 17:30:06.653028 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:30:06.653008    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-9pf62"
Oct 31 17:30:06.663438 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:30:06.663410    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7327c156-422f-4691-a3f2-84ead2505894-istio-kubeconfig\") pod \"istiod-openshift-gateway-7876bfcb74-9pf62\" (UID: \"7327c156-422f-4691-a3f2-84ead2505894\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-9pf62"
Oct 31 17:30:06.663551 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:30:06.663464    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/7327c156-422f-4691-a3f2-84ead2505894-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7876bfcb74-9pf62\" (UID: \"7327c156-422f-4691-a3f2-84ead2505894\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-9pf62"
Oct 31 17:30:06.663609 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:30:06.663553    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jmtcp\" (UniqueName: \"kubernetes.io/projected/7327c156-422f-4691-a3f2-84ead2505894-kube-api-access-jmtcp\") pod \"istiod-openshift-gateway-7876bfcb74-9pf62\" (UID: \"7327c156-422f-4691-a3f2-84ead2505894\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-9pf62"
Oct 31 17:30:06.663609 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:30:06.663591    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7327c156-422f-4691-a3f2-84ead2505894-istio-token\") pod \"istiod-openshift-gateway-7876bfcb74-9pf62\" (UID: \"7327c156-422f-4691-a3f2-84ead2505894\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-9pf62"
Oct 31 17:30:06.663744 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:30:06.663725    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/7327c156-422f-4691-a3f2-84ead2505894-local-certs\") pod \"istiod-openshift-gateway-7876bfcb74-9pf62\" (UID: \"7327c156-422f-4691-a3f2-84ead2505894\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-9pf62"
Oct 31 17:30:06.663803 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:30:06.663759    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/7327c156-422f-4691-a3f2-84ead2505894-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7876bfcb74-9pf62\" (UID: \"7327c156-422f-4691-a3f2-84ead2505894\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-9pf62"
Oct 31 17:30:06.663858 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:30:06.663815    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/7327c156-422f-4691-a3f2-84ead2505894-cacerts\") pod \"istiod-openshift-gateway-7876bfcb74-9pf62\" (UID: \"7327c156-422f-4691-a3f2-84ead2505894\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-9pf62"
Oct 31 17:30:06.675604 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:30:06.674414    4276 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Oct 31 17:30:06.675604 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:30:06.674620    4276 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Oct 31 17:30:06.675604 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:30:06.674801    4276 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Oct 31 17:30:06.676007 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:30:06.675986    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-9pf62"]
Oct 31 17:30:06.687219 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:30:06.687191    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:30:06.755958 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:30:06.755926    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Oct 31 17:30:06.755958 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:30:06.755926    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Oct 31 17:30:06.756163 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:30:06.755998    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Oct 31 17:30:06.756163 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:30:06.756073    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-fm5qc\""
Oct 31 17:30:06.764507 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:30:06.764473    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/7327c156-422f-4691-a3f2-84ead2505894-local-certs\") pod \"istiod-openshift-gateway-7876bfcb74-9pf62\" (UID: \"7327c156-422f-4691-a3f2-84ead2505894\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-9pf62"
Oct 31 17:30:06.764570 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:30:06.764513    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/7327c156-422f-4691-a3f2-84ead2505894-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7876bfcb74-9pf62\" (UID: \"7327c156-422f-4691-a3f2-84ead2505894\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-9pf62"
Oct 31 17:30:06.764570 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:30:06.764540    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/7327c156-422f-4691-a3f2-84ead2505894-cacerts\") pod \"istiod-openshift-gateway-7876bfcb74-9pf62\" (UID: \"7327c156-422f-4691-a3f2-84ead2505894\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-9pf62"
Oct 31 17:30:06.764647 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:30:06.764632    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7327c156-422f-4691-a3f2-84ead2505894-istio-kubeconfig\") pod \"istiod-openshift-gateway-7876bfcb74-9pf62\" (UID: \"7327c156-422f-4691-a3f2-84ead2505894\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-9pf62"
Oct 31 17:30:06.764687 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:30:06.764659    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/7327c156-422f-4691-a3f2-84ead2505894-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7876bfcb74-9pf62\" (UID: \"7327c156-422f-4691-a3f2-84ead2505894\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-9pf62"
Oct 31 17:30:06.764731 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:30:06.764686    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jmtcp\" (UniqueName: \"kubernetes.io/projected/7327c156-422f-4691-a3f2-84ead2505894-kube-api-access-jmtcp\") pod \"istiod-openshift-gateway-7876bfcb74-9pf62\" (UID: \"7327c156-422f-4691-a3f2-84ead2505894\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-9pf62"
Oct 31 17:30:06.764731 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:30:06.764711    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7327c156-422f-4691-a3f2-84ead2505894-istio-token\") pod \"istiod-openshift-gateway-7876bfcb74-9pf62\" (UID: \"7327c156-422f-4691-a3f2-84ead2505894\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-9pf62"
Oct 31 17:30:06.765225 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:30:06.765194    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/7327c156-422f-4691-a3f2-84ead2505894-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7876bfcb74-9pf62\" (UID: \"7327c156-422f-4691-a3f2-84ead2505894\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-9pf62"
Oct 31 17:30:06.766891 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:30:06.766869    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/7327c156-422f-4691-a3f2-84ead2505894-cacerts\") pod \"istiod-openshift-gateway-7876bfcb74-9pf62\" (UID: \"7327c156-422f-4691-a3f2-84ead2505894\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-9pf62"
Oct 31 17:30:06.767009 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:30:06.766925    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/7327c156-422f-4691-a3f2-84ead2505894-local-certs\") pod \"istiod-openshift-gateway-7876bfcb74-9pf62\" (UID: \"7327c156-422f-4691-a3f2-84ead2505894\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-9pf62"
Oct 31 17:30:06.767009 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:30:06.766956    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7327c156-422f-4691-a3f2-84ead2505894-istio-kubeconfig\") pod \"istiod-openshift-gateway-7876bfcb74-9pf62\" (UID: \"7327c156-422f-4691-a3f2-84ead2505894\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-9pf62"
Oct 31 17:30:06.767106 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:30:06.767084    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/7327c156-422f-4691-a3f2-84ead2505894-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7876bfcb74-9pf62\" (UID: \"7327c156-422f-4691-a3f2-84ead2505894\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-9pf62"
Oct 31 17:30:06.784512 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:30:06.784483    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jmtcp\" (UniqueName: \"kubernetes.io/projected/7327c156-422f-4691-a3f2-84ead2505894-kube-api-access-jmtcp\") pod \"istiod-openshift-gateway-7876bfcb74-9pf62\" (UID: \"7327c156-422f-4691-a3f2-84ead2505894\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-9pf62"
Oct 31 17:30:06.784602 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:30:06.784513    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7327c156-422f-4691-a3f2-84ead2505894-istio-token\") pod \"istiod-openshift-gateway-7876bfcb74-9pf62\" (UID: \"7327c156-422f-4691-a3f2-84ead2505894\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-9pf62"
Oct 31 17:30:06.807664 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:30:06.807637    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:30:06.939421 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:30:06.939402    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644rq55d"]
Oct 31 17:30:06.941691 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:30:06.941661    4276 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6b94c15_6e6e_4fa1_9f92_3b6046d2e973.slice/crio-303e51bd2305b9ef8724b5c6926d26f0a3a08b20e811e0d47cae261beeb6db7a WatchSource:0}: Error finding container 303e51bd2305b9ef8724b5c6926d26f0a3a08b20e811e0d47cae261beeb6db7a: Status 404 returned error can't find the container with id 303e51bd2305b9ef8724b5c6926d26f0a3a08b20e811e0d47cae261beeb6db7a
Oct 31 17:30:06.971028 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:30:06.971002    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-9pf62"
Oct 31 17:30:07.006152 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:30:07.006119    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644rq55d" event={"ID":"d6b94c15-6e6e-4fa1-9f92-3b6046d2e973","Type":"ContainerStarted","Data":"303e51bd2305b9ef8724b5c6926d26f0a3a08b20e811e0d47cae261beeb6db7a"}
Oct 31 17:30:07.280207 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:30:07.280135    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-9pf62"]
Oct 31 17:30:07.283141 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:30:07.283118    4276 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7327c156_422f_4691_a3f2_84ead2505894.slice/crio-3285b22d44023ebd6959c13e04fca5a2916fb21e85cedf0706e8853609ba51bd WatchSource:0}: Error finding container 3285b22d44023ebd6959c13e04fca5a2916fb21e85cedf0706e8853609ba51bd: Status 404 returned error can't find the container with id 3285b22d44023ebd6959c13e04fca5a2916fb21e85cedf0706e8853609ba51bd
Oct 31 17:30:07.676558 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:30:07.676521    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:30:08.010655 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:30:08.010578    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-9pf62" event={"ID":"7327c156-422f-4691-a3f2-84ead2505894","Type":"ContainerStarted","Data":"3285b22d44023ebd6959c13e04fca5a2916fb21e85cedf0706e8853609ba51bd"}
Oct 31 17:30:10.133216 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:30:10.133179    4276 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 17:30:10.133578 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:30:10.133259    4276 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 17:30:10.821676 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:30:10.821635    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:30:10.959706 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:30:10.959674    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:30:11.023902 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:30:11.023867    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-9pf62" event={"ID":"7327c156-422f-4691-a3f2-84ead2505894","Type":"ContainerStarted","Data":"2b162f1d4f871220f1658d2ce23b506f8a37f019191cdbb20e87a22d2f245b62"}
Oct 31 17:30:11.024062 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:30:11.023962    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-9pf62"
Oct 31 17:30:11.025481 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:30:11.025453    4276 patch_prober.go:29] interesting pod/istiod-openshift-gateway-7876bfcb74-9pf62 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 31 17:30:11.025585 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:30:11.025509    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-9pf62" podUID="7327c156-422f-4691-a3f2-84ead2505894" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 17:30:11.094360 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:30:11.094276    4276 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-9pf62" podStartSLOduration=2.247009718 podStartE2EDuration="5.094262488s" totalImagesPullingTime="2.84725277s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 17:30:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 17:30:11.080928173 +0000 UTC m=+173.878673104" watchObservedRunningTime="2026-10-31 17:30:11.094262488 +0000 UTC m=+173.892007416"
Oct 31 17:30:11.565200 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:30:11.565159    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:30:11.731122 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:30:11.731084    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:30:11.841129 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:30:11.841054    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:30:12.028166 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:30:12.028135    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-9pf62"
Oct 31 17:30:12.236952 ip-10-0-129-127 kubenswrapper[4276]: E1031 17:30:12.236871    4276 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tg5qk" containerName="extractor"
Oct 31 17:30:14.034931 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:30:14.034898    4276 generic.go:356] "Generic (PLEG): container finished" podID="d6b94c15-6e6e-4fa1-9f92-3b6046d2e973" containerID="4ef20a44d00371deea284e5b5ee3e116f8d453ac88aab6222d53db42ef6f9694" exitCode=0
Oct 31 17:30:14.035330 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:30:14.034951    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644rq55d" event={"ID":"d6b94c15-6e6e-4fa1-9f92-3b6046d2e973","Type":"ContainerDied","Data":"4ef20a44d00371deea284e5b5ee3e116f8d453ac88aab6222d53db42ef6f9694"}
Oct 31 17:30:14.711697 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:30:14.711661    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:30:18.051960 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:30:18.051903    4276 generic.go:356] "Generic (PLEG): container finished" podID="d6b94c15-6e6e-4fa1-9f92-3b6046d2e973" containerID="e0024c4b5c2a71a5aadb59566af80d796a78837f83977d9e59c68bc4ce739dff" exitCode=0
Oct 31 17:30:18.052351 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:30:18.051987    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644rq55d" event={"ID":"d6b94c15-6e6e-4fa1-9f92-3b6046d2e973","Type":"ContainerDied","Data":"e0024c4b5c2a71a5aadb59566af80d796a78837f83977d9e59c68bc4ce739dff"}
Oct 31 17:30:24.076215 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:30:24.076164    4276 generic.go:356] "Generic (PLEG): container finished" podID="d6b94c15-6e6e-4fa1-9f92-3b6046d2e973" containerID="c38718ed7fdab20eb5531c7028d09fa8fbd393b1ffa22e814153f5c61b6fab61" exitCode=0
Oct 31 17:30:24.076574 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:30:24.076241    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644rq55d" event={"ID":"d6b94c15-6e6e-4fa1-9f92-3b6046d2e973","Type":"ContainerDied","Data":"c38718ed7fdab20eb5531c7028d09fa8fbd393b1ffa22e814153f5c61b6fab61"}
Oct 31 17:30:25.199291 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:30:25.199268    4276 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644rq55d"
Oct 31 17:30:25.301345 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:30:25.301304    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d6b94c15-6e6e-4fa1-9f92-3b6046d2e973-bundle\" (UniqueName: \"kubernetes.io/empty-dir/d6b94c15-6e6e-4fa1-9f92-3b6046d2e973-bundle\") pod \"d6b94c15-6e6e-4fa1-9f92-3b6046d2e973\" (UID: \"d6b94c15-6e6e-4fa1-9f92-3b6046d2e973\") "
Oct 31 17:30:25.301345 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:30:25.301349    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d6b94c15-6e6e-4fa1-9f92-3b6046d2e973-kube-api-access-pr8kn\" (UniqueName: \"kubernetes.io/projected/d6b94c15-6e6e-4fa1-9f92-3b6046d2e973-kube-api-access-pr8kn\") pod \"d6b94c15-6e6e-4fa1-9f92-3b6046d2e973\" (UID: \"d6b94c15-6e6e-4fa1-9f92-3b6046d2e973\") "
Oct 31 17:30:25.301600 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:30:25.301377    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d6b94c15-6e6e-4fa1-9f92-3b6046d2e973-util\" (UniqueName: \"kubernetes.io/empty-dir/d6b94c15-6e6e-4fa1-9f92-3b6046d2e973-util\") pod \"d6b94c15-6e6e-4fa1-9f92-3b6046d2e973\" (UID: \"d6b94c15-6e6e-4fa1-9f92-3b6046d2e973\") "
Oct 31 17:30:25.301820 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:30:25.301797    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d6b94c15-6e6e-4fa1-9f92-3b6046d2e973-bundle" pod "d6b94c15-6e6e-4fa1-9f92-3b6046d2e973" (UID: "d6b94c15-6e6e-4fa1-9f92-3b6046d2e973"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:30:25.303606 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:30:25.303581    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d6b94c15-6e6e-4fa1-9f92-3b6046d2e973-kube-api-access-pr8kn" pod "d6b94c15-6e6e-4fa1-9f92-3b6046d2e973" (UID: "d6b94c15-6e6e-4fa1-9f92-3b6046d2e973"). InnerVolumeSpecName "kube-api-access-pr8kn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 17:30:25.306142 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:30:25.306116    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d6b94c15-6e6e-4fa1-9f92-3b6046d2e973-util" pod "d6b94c15-6e6e-4fa1-9f92-3b6046d2e973" (UID: "d6b94c15-6e6e-4fa1-9f92-3b6046d2e973"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:30:25.401981 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:30:25.401950    4276 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d6b94c15-6e6e-4fa1-9f92-3b6046d2e973-bundle\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:30:25.401981 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:30:25.401977    4276 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pr8kn\" (UniqueName: \"kubernetes.io/projected/d6b94c15-6e6e-4fa1-9f92-3b6046d2e973-kube-api-access-pr8kn\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:30:25.401981 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:30:25.401987    4276 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d6b94c15-6e6e-4fa1-9f92-3b6046d2e973-util\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:30:26.085182 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:30:26.085145    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644rq55d" event={"ID":"d6b94c15-6e6e-4fa1-9f92-3b6046d2e973","Type":"ContainerDied","Data":"303e51bd2305b9ef8724b5c6926d26f0a3a08b20e811e0d47cae261beeb6db7a"}
Oct 31 17:30:26.085355 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:30:26.085188    4276 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644rq55d"
Oct 31 17:30:26.085355 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:30:26.085218    4276 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="303e51bd2305b9ef8724b5c6926d26f0a3a08b20e811e0d47cae261beeb6db7a"
Oct 31 17:30:37.655513 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:30:37.655477    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:30:37.767315 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:30:37.767283    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:30:42.233892 ip-10-0-129-127 kubenswrapper[4276]: E1031 17:30:42.233855    4276 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tg5qk" containerName="extractor"
Oct 31 17:30:56.217434 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:30:56.217396    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:30:59.619099 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:30:59.619053    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:30:59.788432 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:30:59.788399    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:31:00.001084 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:00.001014    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:31:00.896938 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:00.896869    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:31:01.039071 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:01.039034    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:31:01.052128 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:01.052104    4276 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-75767875db-vnrd6"]
Oct 31 17:31:01.052346 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:01.052335    4276 cpu_manager.go:414] "containerMap: removing container" podUID="d6b94c15-6e6e-4fa1-9f92-3b6046d2e973" containerName="pull"
Oct 31 17:31:01.052394 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:01.052347    4276 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d6b94c15-6e6e-4fa1-9f92-3b6046d2e973" containerName="pull"
Oct 31 17:31:01.052394 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:01.052365    4276 cpu_manager.go:414] "containerMap: removing container" podUID="d6b94c15-6e6e-4fa1-9f92-3b6046d2e973" containerName="util"
Oct 31 17:31:01.052394 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:01.052370    4276 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d6b94c15-6e6e-4fa1-9f92-3b6046d2e973" containerName="util"
Oct 31 17:31:01.052394 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:01.052376    4276 cpu_manager.go:414] "containerMap: removing container" podUID="d6b94c15-6e6e-4fa1-9f92-3b6046d2e973" containerName="extract"
Oct 31 17:31:01.052394 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:01.052381    4276 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d6b94c15-6e6e-4fa1-9f92-3b6046d2e973" containerName="extract"
Oct 31 17:31:01.052558 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:01.052429    4276 memory_manager.go:365] "RemoveStaleState removing state" podUID="d6b94c15-6e6e-4fa1-9f92-3b6046d2e973" containerName="extract"
Oct 31 17:31:01.055279 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:01.055263    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-75767875db-vnrd6"
Oct 31 17:31:01.084903 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:01.084881    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-75767875db-vnrd6"]
Oct 31 17:31:01.145018 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:01.144992    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-manager-dockercfg-wsnj7\""
Oct 31 17:31:01.145152 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:01.145091    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-webhook-cert\""
Oct 31 17:31:01.145201 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:01.145155    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-manager-service-cert\""
Oct 31 17:31:01.145201 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:01.145194    4276 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub-operator-system\"/\"openshift-service-ca.crt\""
Oct 31 17:31:01.145379 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:01.145365    4276 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub-operator-system\"/\"kube-root-ca.crt\""
Oct 31 17:31:01.145463 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:01.145431    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-metrics-tls\""
Oct 31 17:31:01.155802 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:01.155758    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/9bbf6fec-0f52-4059-bd0e-e36748d42b4a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-75767875db-vnrd6\" (UID: \"9bbf6fec-0f52-4059-bd0e-e36748d42b4a\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-75767875db-vnrd6"
Oct 31 17:31:01.155802 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:01.155793    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/9bbf6fec-0f52-4059-bd0e-e36748d42b4a-manifests\") pod \"opendatahub-operator-controller-manager-75767875db-vnrd6\" (UID: \"9bbf6fec-0f52-4059-bd0e-e36748d42b4a\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-75767875db-vnrd6"
Oct 31 17:31:01.155906 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:01.155815    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/9bbf6fec-0f52-4059-bd0e-e36748d42b4a-metrics-cert\") pod \"opendatahub-operator-controller-manager-75767875db-vnrd6\" (UID: \"9bbf6fec-0f52-4059-bd0e-e36748d42b4a\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-75767875db-vnrd6"
Oct 31 17:31:01.155964 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:01.155947    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-49lmn\" (UniqueName: \"kubernetes.io/projected/9bbf6fec-0f52-4059-bd0e-e36748d42b4a-kube-api-access-49lmn\") pod \"opendatahub-operator-controller-manager-75767875db-vnrd6\" (UID: \"9bbf6fec-0f52-4059-bd0e-e36748d42b4a\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-75767875db-vnrd6"
Oct 31 17:31:01.155999 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:01.155982    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9bbf6fec-0f52-4059-bd0e-e36748d42b4a-tmp\") pod \"opendatahub-operator-controller-manager-75767875db-vnrd6\" (UID: \"9bbf6fec-0f52-4059-bd0e-e36748d42b4a\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-75767875db-vnrd6"
Oct 31 17:31:01.156032 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:01.156005    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9bbf6fec-0f52-4059-bd0e-e36748d42b4a-webhook-cert\") pod \"opendatahub-operator-controller-manager-75767875db-vnrd6\" (UID: \"9bbf6fec-0f52-4059-bd0e-e36748d42b4a\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-75767875db-vnrd6"
Oct 31 17:31:01.257203 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:01.257179    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9bbf6fec-0f52-4059-bd0e-e36748d42b4a-tmp\") pod \"opendatahub-operator-controller-manager-75767875db-vnrd6\" (UID: \"9bbf6fec-0f52-4059-bd0e-e36748d42b4a\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-75767875db-vnrd6"
Oct 31 17:31:01.257377 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:01.257210    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9bbf6fec-0f52-4059-bd0e-e36748d42b4a-webhook-cert\") pod \"opendatahub-operator-controller-manager-75767875db-vnrd6\" (UID: \"9bbf6fec-0f52-4059-bd0e-e36748d42b4a\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-75767875db-vnrd6"
Oct 31 17:31:01.257377 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:01.257232    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/9bbf6fec-0f52-4059-bd0e-e36748d42b4a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-75767875db-vnrd6\" (UID: \"9bbf6fec-0f52-4059-bd0e-e36748d42b4a\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-75767875db-vnrd6"
Oct 31 17:31:01.257377 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:01.257250    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/9bbf6fec-0f52-4059-bd0e-e36748d42b4a-manifests\") pod \"opendatahub-operator-controller-manager-75767875db-vnrd6\" (UID: \"9bbf6fec-0f52-4059-bd0e-e36748d42b4a\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-75767875db-vnrd6"
Oct 31 17:31:01.257377 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:01.257279    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/9bbf6fec-0f52-4059-bd0e-e36748d42b4a-metrics-cert\") pod \"opendatahub-operator-controller-manager-75767875db-vnrd6\" (UID: \"9bbf6fec-0f52-4059-bd0e-e36748d42b4a\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-75767875db-vnrd6"
Oct 31 17:31:01.257377 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:01.257348    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-49lmn\" (UniqueName: \"kubernetes.io/projected/9bbf6fec-0f52-4059-bd0e-e36748d42b4a-kube-api-access-49lmn\") pod \"opendatahub-operator-controller-manager-75767875db-vnrd6\" (UID: \"9bbf6fec-0f52-4059-bd0e-e36748d42b4a\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-75767875db-vnrd6"
Oct 31 17:31:01.257704 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:01.257633    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/9bbf6fec-0f52-4059-bd0e-e36748d42b4a-manifests\") pod \"opendatahub-operator-controller-manager-75767875db-vnrd6\" (UID: \"9bbf6fec-0f52-4059-bd0e-e36748d42b4a\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-75767875db-vnrd6"
Oct 31 17:31:01.259621 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:01.259596    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9bbf6fec-0f52-4059-bd0e-e36748d42b4a-tmp\") pod \"opendatahub-operator-controller-manager-75767875db-vnrd6\" (UID: \"9bbf6fec-0f52-4059-bd0e-e36748d42b4a\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-75767875db-vnrd6"
Oct 31 17:31:01.259960 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:01.259942    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/9bbf6fec-0f52-4059-bd0e-e36748d42b4a-metrics-cert\") pod \"opendatahub-operator-controller-manager-75767875db-vnrd6\" (UID: \"9bbf6fec-0f52-4059-bd0e-e36748d42b4a\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-75767875db-vnrd6"
Oct 31 17:31:01.260012 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:01.259991    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9bbf6fec-0f52-4059-bd0e-e36748d42b4a-webhook-cert\") pod \"opendatahub-operator-controller-manager-75767875db-vnrd6\" (UID: \"9bbf6fec-0f52-4059-bd0e-e36748d42b4a\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-75767875db-vnrd6"
Oct 31 17:31:01.260053 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:01.259999    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/9bbf6fec-0f52-4059-bd0e-e36748d42b4a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-75767875db-vnrd6\" (UID: \"9bbf6fec-0f52-4059-bd0e-e36748d42b4a\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-75767875db-vnrd6"
Oct 31 17:31:01.278563 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:01.278543    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-49lmn\" (UniqueName: \"kubernetes.io/projected/9bbf6fec-0f52-4059-bd0e-e36748d42b4a-kube-api-access-49lmn\") pod \"opendatahub-operator-controller-manager-75767875db-vnrd6\" (UID: \"9bbf6fec-0f52-4059-bd0e-e36748d42b4a\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-75767875db-vnrd6"
Oct 31 17:31:01.292553 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:01.292530    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:31:01.369224 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:01.369205    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-75767875db-vnrd6"
Oct 31 17:31:01.428372 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:01.428319    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:31:01.550555 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:01.550524    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:31:01.675354 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:01.675275    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:31:01.747210 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:31:01.747184    4276 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9bbf6fec_0f52_4059_bd0e_e36748d42b4a.slice/crio-cd2bd0cb0771b9a3ed5c41031ea91517cbab62ff77f3cc7c3844f95649b39f1a WatchSource:0}: Error finding container cd2bd0cb0771b9a3ed5c41031ea91517cbab62ff77f3cc7c3844f95649b39f1a: Status 404 returned error can't find the container with id cd2bd0cb0771b9a3ed5c41031ea91517cbab62ff77f3cc7c3844f95649b39f1a
Oct 31 17:31:01.758706 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:01.758682    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-75767875db-vnrd6"]
Oct 31 17:31:01.838896 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:01.838869    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:31:01.969356 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:01.969292    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:31:02.203552 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:02.203510    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-75767875db-vnrd6" event={"ID":"9bbf6fec-0f52-4059-bd0e-e36748d42b4a","Type":"ContainerStarted","Data":"cd2bd0cb0771b9a3ed5c41031ea91517cbab62ff77f3cc7c3844f95649b39f1a"}
Oct 31 17:31:03.031848 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:03.031815    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:31:04.642616 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:04.642585    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:31:05.003109 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:05.003079    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:31:05.216503 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:05.216420    4276 generic.go:356] "Generic (PLEG): container finished" podID="9bbf6fec-0f52-4059-bd0e-e36748d42b4a" containerID="c9b046aa4f66ab4abdb972aa7f2c1f65c8a95537624fd82c30f4117f10e92d0f" exitCode=0
Oct 31 17:31:05.216503 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:05.216487    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-75767875db-vnrd6" event={"ID":"9bbf6fec-0f52-4059-bd0e-e36748d42b4a","Type":"ContainerDied","Data":"c9b046aa4f66ab4abdb972aa7f2c1f65c8a95537624fd82c30f4117f10e92d0f"}
Oct 31 17:31:05.630305 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:05.630262    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:31:06.221595 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:06.221556    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-75767875db-vnrd6" event={"ID":"9bbf6fec-0f52-4059-bd0e-e36748d42b4a","Type":"ContainerStarted","Data":"8e23e2eba8fd3850021ef1a8ce9810465708471c06e5782ba7f62533fa3eb07e"}
Oct 31 17:31:06.221982 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:06.221644    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-75767875db-vnrd6"
Oct 31 17:31:06.309561 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:06.309506    4276 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-75767875db-vnrd6" podStartSLOduration=2.84809923 podStartE2EDuration="6.309487479s" totalImagesPullingTime="3.019528322s" totalInitContainerRuntime="441.859927ms" isStatefulPod=false podCreationTimestamp="2026-10-31 17:31:00 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 17:31:06.295229769 +0000 UTC m=+229.092974720" watchObservedRunningTime="2026-10-31 17:31:06.309487479 +0000 UTC m=+229.107232411"
Oct 31 17:31:06.590169 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:06.590140    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:31:07.038884 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:07.038792    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:31:08.795827 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:08.795795    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:31:09.088373 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:09.088346    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:31:10.897266 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:10.897229    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:31:11.275379 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:11.275282    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:31:12.234188 ip-10-0-129-127 kubenswrapper[4276]: E1031 17:31:12.234151    4276 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tg5qk" containerName="extractor"
Oct 31 17:31:13.398520 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:13.398480    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:31:14.821816 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:14.821760    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:31:17.227318 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:17.227290    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-75767875db-vnrd6"
Oct 31 17:31:18.053885 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:18.053855    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:31:18.197893 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:18.197861    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:31:18.605675 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:18.605643    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:31:18.740963 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:18.740931    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:31:19.652644 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:19.652611    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:31:19.907126 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:19.907058    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:31:20.212390 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:20.212322    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:31:20.385981 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:20.385946    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:31:20.506315 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:20.506237    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:31:23.611680 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:23.611649    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:31:23.727617 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:23.727591    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:31:24.257547 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:24.257513    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:31:24.374601 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:24.374567    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:31:24.613472 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:24.613432    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:31:24.736578 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:24.736547    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:31:25.572347 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:25.572316    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:31:25.689283 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:25.689251    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:31:27.381555 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:27.381519    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:31:27.506053 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:27.506027    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:31:27.721967 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:27.721885    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:31:28.053416 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:28.053386    4276 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-77485b4dcc-7nnp2"]
Oct 31 17:31:28.059892 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:28.059870    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-77485b4dcc-7nnp2"
Oct 31 17:31:28.093048 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:28.093021    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Oct 31 17:31:28.093251 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:28.093041    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-5lqk4\""
Oct 31 17:31:28.093251 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:28.093148    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Oct 31 17:31:28.094335 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:28.094310    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-77485b4dcc-7nnp2"]
Oct 31 17:31:28.097602 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:28.097582    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:31:28.159208 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:28.159172    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v98p6\" (UniqueName: \"kubernetes.io/projected/fc3fdf2a-ea8b-4a6a-a63b-aa5bd7f77486-kube-api-access-v98p6\") pod \"kube-auth-proxy-77485b4dcc-7nnp2\" (UID: \"fc3fdf2a-ea8b-4a6a-a63b-aa5bd7f77486\") " pod="openshift-ingress/kube-auth-proxy-77485b4dcc-7nnp2"
Oct 31 17:31:28.159397 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:28.159238    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fc3fdf2a-ea8b-4a6a-a63b-aa5bd7f77486-tls-certs\") pod \"kube-auth-proxy-77485b4dcc-7nnp2\" (UID: \"fc3fdf2a-ea8b-4a6a-a63b-aa5bd7f77486\") " pod="openshift-ingress/kube-auth-proxy-77485b4dcc-7nnp2"
Oct 31 17:31:28.159397 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:28.159265    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fc3fdf2a-ea8b-4a6a-a63b-aa5bd7f77486-tmp\") pod \"kube-auth-proxy-77485b4dcc-7nnp2\" (UID: \"fc3fdf2a-ea8b-4a6a-a63b-aa5bd7f77486\") " pod="openshift-ingress/kube-auth-proxy-77485b4dcc-7nnp2"
Oct 31 17:31:28.239273 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:28.239234    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:31:28.260425 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:28.260393    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fc3fdf2a-ea8b-4a6a-a63b-aa5bd7f77486-tmp\") pod \"kube-auth-proxy-77485b4dcc-7nnp2\" (UID: \"fc3fdf2a-ea8b-4a6a-a63b-aa5bd7f77486\") " pod="openshift-ingress/kube-auth-proxy-77485b4dcc-7nnp2"
Oct 31 17:31:28.260607 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:28.260521    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v98p6\" (UniqueName: \"kubernetes.io/projected/fc3fdf2a-ea8b-4a6a-a63b-aa5bd7f77486-kube-api-access-v98p6\") pod \"kube-auth-proxy-77485b4dcc-7nnp2\" (UID: \"fc3fdf2a-ea8b-4a6a-a63b-aa5bd7f77486\") " pod="openshift-ingress/kube-auth-proxy-77485b4dcc-7nnp2"
Oct 31 17:31:28.260607 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:28.260570    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fc3fdf2a-ea8b-4a6a-a63b-aa5bd7f77486-tls-certs\") pod \"kube-auth-proxy-77485b4dcc-7nnp2\" (UID: \"fc3fdf2a-ea8b-4a6a-a63b-aa5bd7f77486\") " pod="openshift-ingress/kube-auth-proxy-77485b4dcc-7nnp2"
Oct 31 17:31:28.263137 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:28.263115    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fc3fdf2a-ea8b-4a6a-a63b-aa5bd7f77486-tls-certs\") pod \"kube-auth-proxy-77485b4dcc-7nnp2\" (UID: \"fc3fdf2a-ea8b-4a6a-a63b-aa5bd7f77486\") " pod="openshift-ingress/kube-auth-proxy-77485b4dcc-7nnp2"
Oct 31 17:31:28.265632 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:28.265611    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fc3fdf2a-ea8b-4a6a-a63b-aa5bd7f77486-tmp\") pod \"kube-auth-proxy-77485b4dcc-7nnp2\" (UID: \"fc3fdf2a-ea8b-4a6a-a63b-aa5bd7f77486\") " pod="openshift-ingress/kube-auth-proxy-77485b4dcc-7nnp2"
Oct 31 17:31:28.281954 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:28.281924    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v98p6\" (UniqueName: \"kubernetes.io/projected/fc3fdf2a-ea8b-4a6a-a63b-aa5bd7f77486-kube-api-access-v98p6\") pod \"kube-auth-proxy-77485b4dcc-7nnp2\" (UID: \"fc3fdf2a-ea8b-4a6a-a63b-aa5bd7f77486\") " pod="openshift-ingress/kube-auth-proxy-77485b4dcc-7nnp2"
Oct 31 17:31:28.362966 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:28.362898    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:31:28.371778 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:28.371751    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-77485b4dcc-7nnp2"
Oct 31 17:31:28.497554 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:28.497514    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:31:28.652793 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:28.652719    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:31:28.665882 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:28.665855    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-77485b4dcc-7nnp2"]
Oct 31 17:31:28.669490 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:31:28.669465    4276 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc3fdf2a_ea8b_4a6a_a63b_aa5bd7f77486.slice/crio-723e43496c126afc62f9f45268dafc91ae039c7aa3519aefb47890217c25f3f4 WatchSource:0}: Error finding container 723e43496c126afc62f9f45268dafc91ae039c7aa3519aefb47890217c25f3f4: Status 404 returned error can't find the container with id 723e43496c126afc62f9f45268dafc91ae039c7aa3519aefb47890217c25f3f4
Oct 31 17:31:28.772311 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:28.772286    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:31:28.894670 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:28.894634    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:31:29.145708 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:29.145677    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:31:29.271010 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:29.267463    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:31:29.299065 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:29.299028    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-77485b4dcc-7nnp2" event={"ID":"fc3fdf2a-ea8b-4a6a-a63b-aa5bd7f77486","Type":"ContainerStarted","Data":"723e43496c126afc62f9f45268dafc91ae039c7aa3519aefb47890217c25f3f4"}
Oct 31 17:31:29.396395 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:29.396319    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:31:29.733193 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:29.733104    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:31:30.328128 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:30.328096    4276 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-597b95cb84-68f2w"]
Oct 31 17:31:30.336942 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:30.336916    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-597b95cb84-68f2w"
Oct 31 17:31:30.350679 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:30.350633    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-597b95cb84-68f2w"]
Oct 31 17:31:30.363709 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:30.363484    4276 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Oct 31 17:31:30.363709 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:30.363623    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-f5lr6\""
Oct 31 17:31:30.363709 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:30.363681    4276 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Oct 31 17:31:30.479892 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:30.479856    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-scr2g\" (UniqueName: \"kubernetes.io/projected/4e5ddae0-134c-4b5d-9e17-ba08a6d20620-kube-api-access-scr2g\") pod \"cert-manager-597b95cb84-68f2w\" (UID: \"4e5ddae0-134c-4b5d-9e17-ba08a6d20620\") " pod="cert-manager/cert-manager-597b95cb84-68f2w"
Oct 31 17:31:30.480077 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:30.479965    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4e5ddae0-134c-4b5d-9e17-ba08a6d20620-bound-sa-token\") pod \"cert-manager-597b95cb84-68f2w\" (UID: \"4e5ddae0-134c-4b5d-9e17-ba08a6d20620\") " pod="cert-manager/cert-manager-597b95cb84-68f2w"
Oct 31 17:31:30.581497 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:30.581353    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-scr2g\" (UniqueName: \"kubernetes.io/projected/4e5ddae0-134c-4b5d-9e17-ba08a6d20620-kube-api-access-scr2g\") pod \"cert-manager-597b95cb84-68f2w\" (UID: \"4e5ddae0-134c-4b5d-9e17-ba08a6d20620\") " pod="cert-manager/cert-manager-597b95cb84-68f2w"
Oct 31 17:31:30.581627 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:30.581542    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4e5ddae0-134c-4b5d-9e17-ba08a6d20620-bound-sa-token\") pod \"cert-manager-597b95cb84-68f2w\" (UID: \"4e5ddae0-134c-4b5d-9e17-ba08a6d20620\") " pod="cert-manager/cert-manager-597b95cb84-68f2w"
Oct 31 17:31:30.602548 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:30.602525    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4e5ddae0-134c-4b5d-9e17-ba08a6d20620-bound-sa-token\") pod \"cert-manager-597b95cb84-68f2w\" (UID: \"4e5ddae0-134c-4b5d-9e17-ba08a6d20620\") " pod="cert-manager/cert-manager-597b95cb84-68f2w"
Oct 31 17:31:30.602770 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:30.602752    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-scr2g\" (UniqueName: \"kubernetes.io/projected/4e5ddae0-134c-4b5d-9e17-ba08a6d20620-kube-api-access-scr2g\") pod \"cert-manager-597b95cb84-68f2w\" (UID: \"4e5ddae0-134c-4b5d-9e17-ba08a6d20620\") " pod="cert-manager/cert-manager-597b95cb84-68f2w"
Oct 31 17:31:30.660635 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:30.660603    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-597b95cb84-68f2w"
Oct 31 17:31:30.874102 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:30.874020    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:31:30.976757 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:30.976734    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-597b95cb84-68f2w"]
Oct 31 17:31:30.979140 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:31:30.979106    4276 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4e5ddae0_134c_4b5d_9e17_ba08a6d20620.slice/crio-69408be4f4d8c930b90849194803700db9bb0b203ad4a3e994de894252d27882 WatchSource:0}: Error finding container 69408be4f4d8c930b90849194803700db9bb0b203ad4a3e994de894252d27882: Status 404 returned error can't find the container with id 69408be4f4d8c930b90849194803700db9bb0b203ad4a3e994de894252d27882
Oct 31 17:31:31.309232 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:31.309184    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-597b95cb84-68f2w" event={"ID":"4e5ddae0-134c-4b5d-9e17-ba08a6d20620","Type":"ContainerStarted","Data":"69408be4f4d8c930b90849194803700db9bb0b203ad4a3e994de894252d27882"}
Oct 31 17:31:31.310785 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:31.310754    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-77485b4dcc-7nnp2" event={"ID":"fc3fdf2a-ea8b-4a6a-a63b-aa5bd7f77486","Type":"ContainerStarted","Data":"371f85a8b7f02c42bc6e5f1424566293cd2b78cbf432f94d04f47c760c3c44cf"}
Oct 31 17:31:31.366733 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:31.366687    4276 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-77485b4dcc-7nnp2" podStartSLOduration=0.996032535 podStartE2EDuration="3.36667545s" totalImagesPullingTime="2.370642915s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 17:31:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 17:31:31.366162563 +0000 UTC m=+254.163907492" watchObservedRunningTime="2026-10-31 17:31:31.36667545 +0000 UTC m=+254.164420378"
Oct 31 17:31:34.324671 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:34.324635    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-597b95cb84-68f2w" event={"ID":"4e5ddae0-134c-4b5d-9e17-ba08a6d20620","Type":"ContainerStarted","Data":"dd955b01f4fd9aa4ede865740d39556b682aa256df723aabdfc1086060060971"}
Oct 31 17:31:34.373224 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:34.373179    4276 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="cert-manager/cert-manager-597b95cb84-68f2w" podStartSLOduration=1.353507745 podStartE2EDuration="4.373162269s" totalImagesPullingTime="3.019654524s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 17:31:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 17:31:34.372554838 +0000 UTC m=+257.170299767" watchObservedRunningTime="2026-10-31 17:31:34.373162269 +0000 UTC m=+257.170907197"
Oct 31 17:31:34.836204 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:34.836171    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:31:36.672955 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:36.672926    4276 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-module-controller-manager-7566cf5fc8-mb26r"]
Oct 31 17:31:36.676106 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:36.676089    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-mb26r"
Oct 31 17:31:36.695783 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:36.695755    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-module-controller-manager-7566cf5fc8-mb26r"]
Oct 31 17:31:36.708260 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:36.708235    4276 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Oct 31 17:31:36.708519 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:36.708494    4276 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Oct 31 17:31:36.708519 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:36.708500    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-module-controller-manager-dockercfg-zm2zr\""
Oct 31 17:31:36.736266 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:36.736241    4276 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"odh-trusted-ca-bundle\""
Oct 31 17:31:36.831207 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:36.831175    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/de946e91-8b1c-42d2-b859-e7d93a6fb7cd-manifests\") pod \"kserve-module-controller-manager-7566cf5fc8-mb26r\" (UID: \"de946e91-8b1c-42d2-b859-e7d93a6fb7cd\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-mb26r"
Oct 31 17:31:36.831369 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:36.831238    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/de946e91-8b1c-42d2-b859-e7d93a6fb7cd-tmp\") pod \"kserve-module-controller-manager-7566cf5fc8-mb26r\" (UID: \"de946e91-8b1c-42d2-b859-e7d93a6fb7cd\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-mb26r"
Oct 31 17:31:36.831369 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:36.831317    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jxx6h\" (UniqueName: \"kubernetes.io/projected/de946e91-8b1c-42d2-b859-e7d93a6fb7cd-kube-api-access-jxx6h\") pod \"kserve-module-controller-manager-7566cf5fc8-mb26r\" (UID: \"de946e91-8b1c-42d2-b859-e7d93a6fb7cd\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-mb26r"
Oct 31 17:31:36.831369 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:36.831348    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/de946e91-8b1c-42d2-b859-e7d93a6fb7cd-trusted-ca\") pod \"kserve-module-controller-manager-7566cf5fc8-mb26r\" (UID: \"de946e91-8b1c-42d2-b859-e7d93a6fb7cd\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-mb26r"
Oct 31 17:31:36.932263 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:36.932199    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jxx6h\" (UniqueName: \"kubernetes.io/projected/de946e91-8b1c-42d2-b859-e7d93a6fb7cd-kube-api-access-jxx6h\") pod \"kserve-module-controller-manager-7566cf5fc8-mb26r\" (UID: \"de946e91-8b1c-42d2-b859-e7d93a6fb7cd\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-mb26r"
Oct 31 17:31:36.932388 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:36.932264    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/de946e91-8b1c-42d2-b859-e7d93a6fb7cd-trusted-ca\") pod \"kserve-module-controller-manager-7566cf5fc8-mb26r\" (UID: \"de946e91-8b1c-42d2-b859-e7d93a6fb7cd\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-mb26r"
Oct 31 17:31:36.932388 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:36.932286    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/de946e91-8b1c-42d2-b859-e7d93a6fb7cd-manifests\") pod \"kserve-module-controller-manager-7566cf5fc8-mb26r\" (UID: \"de946e91-8b1c-42d2-b859-e7d93a6fb7cd\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-mb26r"
Oct 31 17:31:36.932388 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:36.932333    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/de946e91-8b1c-42d2-b859-e7d93a6fb7cd-tmp\") pod \"kserve-module-controller-manager-7566cf5fc8-mb26r\" (UID: \"de946e91-8b1c-42d2-b859-e7d93a6fb7cd\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-mb26r"
Oct 31 17:31:36.932776 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:36.932755    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/de946e91-8b1c-42d2-b859-e7d93a6fb7cd-manifests\") pod \"kserve-module-controller-manager-7566cf5fc8-mb26r\" (UID: \"de946e91-8b1c-42d2-b859-e7d93a6fb7cd\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-mb26r"
Oct 31 17:31:36.932812 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:36.932797    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/de946e91-8b1c-42d2-b859-e7d93a6fb7cd-tmp\") pod \"kserve-module-controller-manager-7566cf5fc8-mb26r\" (UID: \"de946e91-8b1c-42d2-b859-e7d93a6fb7cd\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-mb26r"
Oct 31 17:31:36.933171 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:36.933151    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/de946e91-8b1c-42d2-b859-e7d93a6fb7cd-trusted-ca\") pod \"kserve-module-controller-manager-7566cf5fc8-mb26r\" (UID: \"de946e91-8b1c-42d2-b859-e7d93a6fb7cd\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-mb26r"
Oct 31 17:31:36.952035 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:36.952004    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jxx6h\" (UniqueName: \"kubernetes.io/projected/de946e91-8b1c-42d2-b859-e7d93a6fb7cd-kube-api-access-jxx6h\") pod \"kserve-module-controller-manager-7566cf5fc8-mb26r\" (UID: \"de946e91-8b1c-42d2-b859-e7d93a6fb7cd\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-mb26r"
Oct 31 17:31:36.987880 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:36.987854    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-mb26r"
Oct 31 17:31:37.077356 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:37.077318    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:31:37.295624 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:37.295601    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-module-controller-manager-7566cf5fc8-mb26r"]
Oct 31 17:31:37.297836 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:31:37.297806    4276 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde946e91_8b1c_42d2_b859_e7d93a6fb7cd.slice/crio-a19e80dc9816c6dc3bf787f6f0c583cef57e83a44954924d820a11a12bdd6347 WatchSource:0}: Error finding container a19e80dc9816c6dc3bf787f6f0c583cef57e83a44954924d820a11a12bdd6347: Status 404 returned error can't find the container with id a19e80dc9816c6dc3bf787f6f0c583cef57e83a44954924d820a11a12bdd6347
Oct 31 17:31:37.336170 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:37.336147    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-mb26r" event={"ID":"de946e91-8b1c-42d2-b859-e7d93a6fb7cd","Type":"ContainerStarted","Data":"a19e80dc9816c6dc3bf787f6f0c583cef57e83a44954924d820a11a12bdd6347"}
Oct 31 17:31:39.209166 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:39.209092    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:31:39.345979 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:39.345929    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-mb26r" event={"ID":"de946e91-8b1c-42d2-b859-e7d93a6fb7cd","Type":"ContainerStarted","Data":"e5777351264086248e32e41af325e958cf7e9d12168f115b214d2821bb6d68d3"}
Oct 31 17:31:39.346147 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:39.346050    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-mb26r"
Oct 31 17:31:39.401852 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:39.401818    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:31:39.414963 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:39.414922    4276 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-mb26r" podStartSLOduration=1.896992264 podStartE2EDuration="3.414909309s" totalImagesPullingTime="1.517917045s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 17:31:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 17:31:39.413420624 +0000 UTC m=+262.211165551" watchObservedRunningTime="2026-10-31 17:31:39.414909309 +0000 UTC m=+262.212654236"
Oct 31 17:31:42.236764 ip-10-0-129-127 kubenswrapper[4276]: E1031 17:31:42.236723    4276 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tg5qk" containerName="extractor"
Oct 31 17:31:43.014884 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:43.014845    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:31:43.151853 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:43.151821    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:31:43.269340 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:43.269261    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:31:44.498318 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:44.498268    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:31:44.612551 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:44.612524    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:31:45.259223 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:45.259182    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:31:47.525272 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:47.525237    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:31:47.636497 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:47.636460    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:31:49.001630 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:49.001593    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:31:49.112745 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:49.112713    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:31:49.220590 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:49.220560    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:31:49.434753 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:49.434720    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:31:49.540865 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:49.540824    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:31:52.677741 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:52.677703    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:31:52.793769 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:52.793738    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:31:55.278060 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:55.278027    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:31:55.383890 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:55.383848    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:31:55.673911 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:55.673878    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:31:55.779637 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:31:55.779599    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:32:00.351345 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:00.351314    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-mb26r"
Oct 31 17:32:00.968161 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:00.968124    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:32:01.073673 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:01.073642    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:32:01.765841 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:01.765808    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:32:04.134455 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:04.134410    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:32:04.245916 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:04.245879    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:32:04.359120 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:04.359091    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:32:04.477392 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:04.477325    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:32:07.869616 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:07.869578    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:32:07.984221 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:07.984185    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:32:09.234515 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:09.234484    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:32:09.437485 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:09.437427    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:32:12.233934 ip-10-0-129-127 kubenswrapper[4276]: E1031 17:32:12.233898    4276 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tg5qk" containerName="extractor"
Oct 31 17:32:17.369762 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:17.369734    4276 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zpf4d_5819fd64-9ac0-4436-b8fe-dcd3a27827ac/ovn-acl-logging/0.log"
Oct 31 17:32:17.370274 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:17.369788    4276 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zpf4d_5819fd64-9ac0-4436-b8fe-dcd3a27827ac/ovn-acl-logging/0.log"
Oct 31 17:32:29.372001 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:29.371970    4276 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-xxspc"]
Oct 31 17:32:29.383968 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:29.383948    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-xxspc"
Oct 31 17:32:29.402577 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:29.402553    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-xxspc"]
Oct 31 17:32:29.565187 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:29.565139    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/00caa007-32e1-4ff0-a5d5-71ab1b73eca9-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-xxspc\" (UID: \"00caa007-32e1-4ff0-a5d5-71ab1b73eca9\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-xxspc"
Oct 31 17:32:29.565365 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:29.565258    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/00caa007-32e1-4ff0-a5d5-71ab1b73eca9-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-xxspc\" (UID: \"00caa007-32e1-4ff0-a5d5-71ab1b73eca9\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-xxspc"
Oct 31 17:32:29.565365 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:29.565292    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/00caa007-32e1-4ff0-a5d5-71ab1b73eca9-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-xxspc\" (UID: \"00caa007-32e1-4ff0-a5d5-71ab1b73eca9\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-xxspc"
Oct 31 17:32:29.565365 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:29.565311    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/00caa007-32e1-4ff0-a5d5-71ab1b73eca9-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-xxspc\" (UID: \"00caa007-32e1-4ff0-a5d5-71ab1b73eca9\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-xxspc"
Oct 31 17:32:29.565365 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:29.565330    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/00caa007-32e1-4ff0-a5d5-71ab1b73eca9-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-xxspc\" (UID: \"00caa007-32e1-4ff0-a5d5-71ab1b73eca9\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-xxspc"
Oct 31 17:32:29.565606 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:29.565435    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/00caa007-32e1-4ff0-a5d5-71ab1b73eca9-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-xxspc\" (UID: \"00caa007-32e1-4ff0-a5d5-71ab1b73eca9\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-xxspc"
Oct 31 17:32:29.565606 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:29.565502    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7bc98\" (UniqueName: \"kubernetes.io/projected/00caa007-32e1-4ff0-a5d5-71ab1b73eca9-kube-api-access-7bc98\") pod \"istiod-openshift-gateway-7ddd9cfdc8-xxspc\" (UID: \"00caa007-32e1-4ff0-a5d5-71ab1b73eca9\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-xxspc"
Oct 31 17:32:29.666369 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:29.666297    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/00caa007-32e1-4ff0-a5d5-71ab1b73eca9-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-xxspc\" (UID: \"00caa007-32e1-4ff0-a5d5-71ab1b73eca9\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-xxspc"
Oct 31 17:32:29.666369 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:29.666333    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/00caa007-32e1-4ff0-a5d5-71ab1b73eca9-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-xxspc\" (UID: \"00caa007-32e1-4ff0-a5d5-71ab1b73eca9\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-xxspc"
Oct 31 17:32:29.666369 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:29.666357    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/00caa007-32e1-4ff0-a5d5-71ab1b73eca9-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-xxspc\" (UID: \"00caa007-32e1-4ff0-a5d5-71ab1b73eca9\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-xxspc"
Oct 31 17:32:29.666655 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:29.666394    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/00caa007-32e1-4ff0-a5d5-71ab1b73eca9-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-xxspc\" (UID: \"00caa007-32e1-4ff0-a5d5-71ab1b73eca9\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-xxspc"
Oct 31 17:32:29.666655 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:29.666422    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7bc98\" (UniqueName: \"kubernetes.io/projected/00caa007-32e1-4ff0-a5d5-71ab1b73eca9-kube-api-access-7bc98\") pod \"istiod-openshift-gateway-7ddd9cfdc8-xxspc\" (UID: \"00caa007-32e1-4ff0-a5d5-71ab1b73eca9\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-xxspc"
Oct 31 17:32:29.666655 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:29.666474    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/00caa007-32e1-4ff0-a5d5-71ab1b73eca9-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-xxspc\" (UID: \"00caa007-32e1-4ff0-a5d5-71ab1b73eca9\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-xxspc"
Oct 31 17:32:29.666655 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:29.666548    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/00caa007-32e1-4ff0-a5d5-71ab1b73eca9-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-xxspc\" (UID: \"00caa007-32e1-4ff0-a5d5-71ab1b73eca9\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-xxspc"
Oct 31 17:32:29.667139 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:29.667110    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/00caa007-32e1-4ff0-a5d5-71ab1b73eca9-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-xxspc\" (UID: \"00caa007-32e1-4ff0-a5d5-71ab1b73eca9\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-xxspc"
Oct 31 17:32:29.669214 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:29.669188    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/00caa007-32e1-4ff0-a5d5-71ab1b73eca9-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-xxspc\" (UID: \"00caa007-32e1-4ff0-a5d5-71ab1b73eca9\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-xxspc"
Oct 31 17:32:29.669309 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:29.669196    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/00caa007-32e1-4ff0-a5d5-71ab1b73eca9-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-xxspc\" (UID: \"00caa007-32e1-4ff0-a5d5-71ab1b73eca9\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-xxspc"
Oct 31 17:32:29.669361 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:29.669319    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/00caa007-32e1-4ff0-a5d5-71ab1b73eca9-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-xxspc\" (UID: \"00caa007-32e1-4ff0-a5d5-71ab1b73eca9\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-xxspc"
Oct 31 17:32:29.669398 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:29.669367    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/00caa007-32e1-4ff0-a5d5-71ab1b73eca9-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-xxspc\" (UID: \"00caa007-32e1-4ff0-a5d5-71ab1b73eca9\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-xxspc"
Oct 31 17:32:29.686853 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:29.686831    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/00caa007-32e1-4ff0-a5d5-71ab1b73eca9-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-xxspc\" (UID: \"00caa007-32e1-4ff0-a5d5-71ab1b73eca9\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-xxspc"
Oct 31 17:32:29.687007 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:29.686989    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7bc98\" (UniqueName: \"kubernetes.io/projected/00caa007-32e1-4ff0-a5d5-71ab1b73eca9-kube-api-access-7bc98\") pod \"istiod-openshift-gateway-7ddd9cfdc8-xxspc\" (UID: \"00caa007-32e1-4ff0-a5d5-71ab1b73eca9\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-xxspc"
Oct 31 17:32:29.695692 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:29.695675    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-xxspc"
Oct 31 17:32:30.002269 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:30.002196    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-xxspc"]
Oct 31 17:32:30.006649 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:32:30.006625    4276 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00caa007_32e1_4ff0_a5d5_71ab1b73eca9.slice/crio-f35a37948565970d578770e547d4183b6fdcd78cd0431de102c80cdb0811cd5a WatchSource:0}: Error finding container f35a37948565970d578770e547d4183b6fdcd78cd0431de102c80cdb0811cd5a: Status 404 returned error can't find the container with id f35a37948565970d578770e547d4183b6fdcd78cd0431de102c80cdb0811cd5a
Oct 31 17:32:30.008570 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:30.008542    4276 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 17:32:30.008645 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:30.008599    4276 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 17:32:30.532511 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:30.532476    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:32:30.550598 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:30.549683    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-xxspc" event={"ID":"00caa007-32e1-4ff0-a5d5-71ab1b73eca9","Type":"ContainerStarted","Data":"5fa1879574b282c012c1a5593a44970aaff45dac29486f07a895d00e879a92c2"}
Oct 31 17:32:30.550598 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:30.549743    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-xxspc" event={"ID":"00caa007-32e1-4ff0-a5d5-71ab1b73eca9","Type":"ContainerStarted","Data":"f35a37948565970d578770e547d4183b6fdcd78cd0431de102c80cdb0811cd5a"}
Oct 31 17:32:30.550598 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:30.550511    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-xxspc"
Oct 31 17:32:30.602680 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:30.602620    4276 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-xxspc" podStartSLOduration=1.602602315 podStartE2EDuration="1.602602315s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 17:32:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 17:32:30.598180423 +0000 UTC m=+313.395925352" watchObservedRunningTime="2026-10-31 17:32:30.602602315 +0000 UTC m=+313.400347244"
Oct 31 17:32:30.664653 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:30.664623    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:32:30.779657 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:30.779623    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:32:30.915023 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:30.914992    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:32:31.026975 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:31.026944    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:32:31.304125 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:31.304095    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:32:31.554970 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:31.554903    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-xxspc"
Oct 31 17:32:31.662220 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:31.662187    4276 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-9pf62"]
Oct 31 17:32:31.662409 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:31.662388    4276 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-9pf62" podUID="7327c156-422f-4691-a3f2-84ead2505894" containerName="discovery" containerID="cri-o://2b162f1d4f871220f1658d2ce23b506f8a37f019191cdbb20e87a22d2f245b62" gracePeriod=30
Oct 31 17:32:31.905015 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:31.904991    4276 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-9pf62"
Oct 31 17:32:31.984927 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:31.984900    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/7327c156-422f-4691-a3f2-84ead2505894-istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7327c156-422f-4691-a3f2-84ead2505894-istio-kubeconfig\") pod \"7327c156-422f-4691-a3f2-84ead2505894\" (UID: \"7327c156-422f-4691-a3f2-84ead2505894\") "
Oct 31 17:32:31.984927 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:31.984928    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/7327c156-422f-4691-a3f2-84ead2505894-cacerts\" (UniqueName: \"kubernetes.io/secret/7327c156-422f-4691-a3f2-84ead2505894-cacerts\") pod \"7327c156-422f-4691-a3f2-84ead2505894\" (UID: \"7327c156-422f-4691-a3f2-84ead2505894\") "
Oct 31 17:32:31.985121 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:31.984979    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/7327c156-422f-4691-a3f2-84ead2505894-istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/7327c156-422f-4691-a3f2-84ead2505894-istio-csr-dns-cert\") pod \"7327c156-422f-4691-a3f2-84ead2505894\" (UID: \"7327c156-422f-4691-a3f2-84ead2505894\") "
Oct 31 17:32:31.985121 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:31.984997    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7327c156-422f-4691-a3f2-84ead2505894-istio-token\" (UniqueName: \"kubernetes.io/projected/7327c156-422f-4691-a3f2-84ead2505894-istio-token\") pod \"7327c156-422f-4691-a3f2-84ead2505894\" (UID: \"7327c156-422f-4691-a3f2-84ead2505894\") "
Oct 31 17:32:31.985121 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:31.985016    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7327c156-422f-4691-a3f2-84ead2505894-local-certs\" (UniqueName: \"kubernetes.io/empty-dir/7327c156-422f-4691-a3f2-84ead2505894-local-certs\") pod \"7327c156-422f-4691-a3f2-84ead2505894\" (UID: \"7327c156-422f-4691-a3f2-84ead2505894\") "
Oct 31 17:32:31.985121 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:31.985037    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7327c156-422f-4691-a3f2-84ead2505894-istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/7327c156-422f-4691-a3f2-84ead2505894-istio-csr-ca-configmap\") pod \"7327c156-422f-4691-a3f2-84ead2505894\" (UID: \"7327c156-422f-4691-a3f2-84ead2505894\") "
Oct 31 17:32:31.985121 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:31.985073    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7327c156-422f-4691-a3f2-84ead2505894-kube-api-access-jmtcp\" (UniqueName: \"kubernetes.io/projected/7327c156-422f-4691-a3f2-84ead2505894-kube-api-access-jmtcp\") pod \"7327c156-422f-4691-a3f2-84ead2505894\" (UID: \"7327c156-422f-4691-a3f2-84ead2505894\") "
Oct 31 17:32:31.985536 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:31.985491    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7327c156-422f-4691-a3f2-84ead2505894-istio-csr-ca-configmap" pod "7327c156-422f-4691-a3f2-84ead2505894" (UID: "7327c156-422f-4691-a3f2-84ead2505894"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 17:32:31.987294 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:31.987266    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7327c156-422f-4691-a3f2-84ead2505894-istio-kubeconfig" pod "7327c156-422f-4691-a3f2-84ead2505894" (UID: "7327c156-422f-4691-a3f2-84ead2505894"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 17:32:31.987531 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:31.987510    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7327c156-422f-4691-a3f2-84ead2505894-istio-token" pod "7327c156-422f-4691-a3f2-84ead2505894" (UID: "7327c156-422f-4691-a3f2-84ead2505894"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 17:32:31.987591 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:31.987509    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7327c156-422f-4691-a3f2-84ead2505894-istio-csr-dns-cert" pod "7327c156-422f-4691-a3f2-84ead2505894" (UID: "7327c156-422f-4691-a3f2-84ead2505894"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 17:32:31.987591 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:31.987532    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7327c156-422f-4691-a3f2-84ead2505894-local-certs" pod "7327c156-422f-4691-a3f2-84ead2505894" (UID: "7327c156-422f-4691-a3f2-84ead2505894"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:32:31.987861 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:31.987841    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7327c156-422f-4691-a3f2-84ead2505894-kube-api-access-jmtcp" pod "7327c156-422f-4691-a3f2-84ead2505894" (UID: "7327c156-422f-4691-a3f2-84ead2505894"). InnerVolumeSpecName "kube-api-access-jmtcp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 17:32:31.987963 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:31.987895    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7327c156-422f-4691-a3f2-84ead2505894-cacerts" pod "7327c156-422f-4691-a3f2-84ead2505894" (UID: "7327c156-422f-4691-a3f2-84ead2505894"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 17:32:32.085824 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:32.085774    4276 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7327c156-422f-4691-a3f2-84ead2505894-istio-kubeconfig\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:32:32.085824 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:32.085798    4276 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/7327c156-422f-4691-a3f2-84ead2505894-cacerts\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:32:32.085824 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:32.085808    4276 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/7327c156-422f-4691-a3f2-84ead2505894-istio-csr-dns-cert\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:32:32.085824 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:32.085817    4276 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7327c156-422f-4691-a3f2-84ead2505894-istio-token\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:32:32.085824 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:32.085825    4276 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/7327c156-422f-4691-a3f2-84ead2505894-local-certs\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:32:32.086053 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:32.085833    4276 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/7327c156-422f-4691-a3f2-84ead2505894-istio-csr-ca-configmap\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:32:32.086053 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:32.085842    4276 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jmtcp\" (UniqueName: \"kubernetes.io/projected/7327c156-422f-4691-a3f2-84ead2505894-kube-api-access-jmtcp\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:32:32.558613 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:32.558582    4276 generic.go:356] "Generic (PLEG): container finished" podID="7327c156-422f-4691-a3f2-84ead2505894" containerID="2b162f1d4f871220f1658d2ce23b506f8a37f019191cdbb20e87a22d2f245b62" exitCode=0
Oct 31 17:32:32.559079 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:32.558625    4276 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-9pf62"
Oct 31 17:32:32.559079 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:32.558666    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-9pf62" event={"ID":"7327c156-422f-4691-a3f2-84ead2505894","Type":"ContainerDied","Data":"2b162f1d4f871220f1658d2ce23b506f8a37f019191cdbb20e87a22d2f245b62"}
Oct 31 17:32:32.559079 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:32.558721    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-9pf62" event={"ID":"7327c156-422f-4691-a3f2-84ead2505894","Type":"ContainerDied","Data":"3285b22d44023ebd6959c13e04fca5a2916fb21e85cedf0706e8853609ba51bd"}
Oct 31 17:32:32.559079 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:32.558749    4276 scope.go:122] "RemoveContainer" containerID="2b162f1d4f871220f1658d2ce23b506f8a37f019191cdbb20e87a22d2f245b62"
Oct 31 17:32:32.569982 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:32.569962    4276 scope.go:122] "RemoveContainer" containerID="2b162f1d4f871220f1658d2ce23b506f8a37f019191cdbb20e87a22d2f245b62"
Oct 31 17:32:32.570339 ip-10-0-129-127 kubenswrapper[4276]: E1031 17:32:32.570320    4276 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2b162f1d4f871220f1658d2ce23b506f8a37f019191cdbb20e87a22d2f245b62\": container with ID starting with 2b162f1d4f871220f1658d2ce23b506f8a37f019191cdbb20e87a22d2f245b62 not found: ID does not exist" containerID="2b162f1d4f871220f1658d2ce23b506f8a37f019191cdbb20e87a22d2f245b62"
Oct 31 17:32:32.570411 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:32.570352    4276 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2b162f1d4f871220f1658d2ce23b506f8a37f019191cdbb20e87a22d2f245b62"} err="failed to get container status \"2b162f1d4f871220f1658d2ce23b506f8a37f019191cdbb20e87a22d2f245b62\": rpc error: code = NotFound desc = could not find container \"2b162f1d4f871220f1658d2ce23b506f8a37f019191cdbb20e87a22d2f245b62\": container with ID starting with 2b162f1d4f871220f1658d2ce23b506f8a37f019191cdbb20e87a22d2f245b62 not found: ID does not exist"
Oct 31 17:32:32.611963 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:32.611944    4276 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-9pf62"]
Oct 31 17:32:32.625756 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:32.625737    4276 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-9pf62"]
Oct 31 17:32:33.111948 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:33.111919    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:32:33.230836 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:33.230804    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:32:33.337853 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:33.337820    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:32:33.365304 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:33.365239    4276 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7327c156-422f-4691-a3f2-84ead2505894" path="/var/lib/kubelet/pods/7327c156-422f-4691-a3f2-84ead2505894/volumes"
Oct 31 17:32:33.449982 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:33.449950    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:32:33.560074 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:33.560040    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:32:38.324238 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:38.324204    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:32:38.438807 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:38.438779    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:32:39.811531 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:39.811498    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:32:39.925238 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:39.925210    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:32:40.040754 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:40.040721    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:32:40.059315 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:40.059290    4276 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-6d669c7fcd-qkt2f"]
Oct 31 17:32:40.059653 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:40.059637    4276 cpu_manager.go:414] "containerMap: removing container" podUID="7327c156-422f-4691-a3f2-84ead2505894" containerName="discovery"
Oct 31 17:32:40.059740 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:40.059655    4276 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7327c156-422f-4691-a3f2-84ead2505894" containerName="discovery"
Oct 31 17:32:40.059740 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:40.059729    4276 memory_manager.go:365] "RemoveStaleState removing state" podUID="7327c156-422f-4691-a3f2-84ead2505894" containerName="discovery"
Oct 31 17:32:40.063841 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:40.063789    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-qkt2f"
Oct 31 17:32:40.080997 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:40.080977    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6d669c7fcd-qkt2f"]
Oct 31 17:32:40.093629 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:40.093609    4276 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 31 17:32:40.093751 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:40.093609    4276 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 31 17:32:40.093820 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:40.093765    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Oct 31 17:32:40.097123 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:40.097103    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-4s8xz\""
Oct 31 17:32:40.097219 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:40.097132    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Oct 31 17:32:40.147620 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:40.147595    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f34c5540-3adf-48e7-a459-9482aab37685-openshift-service-ca\") pod \"maas-api-6d669c7fcd-qkt2f\" (UID: \"f34c5540-3adf-48e7-a459-9482aab37685\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-qkt2f"
Oct 31 17:32:40.147714 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:40.147628    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sbtdb\" (UniqueName: \"kubernetes.io/projected/f34c5540-3adf-48e7-a459-9482aab37685-kube-api-access-sbtdb\") pod \"maas-api-6d669c7fcd-qkt2f\" (UID: \"f34c5540-3adf-48e7-a459-9482aab37685\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-qkt2f"
Oct 31 17:32:40.147714 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:40.147648    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f34c5540-3adf-48e7-a459-9482aab37685-maas-api-tls\") pod \"maas-api-6d669c7fcd-qkt2f\" (UID: \"f34c5540-3adf-48e7-a459-9482aab37685\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-qkt2f"
Oct 31 17:32:40.147830 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:40.147708    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f34c5540-3adf-48e7-a459-9482aab37685-metrics-certs\") pod \"maas-api-6d669c7fcd-qkt2f\" (UID: \"f34c5540-3adf-48e7-a459-9482aab37685\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-qkt2f"
Oct 31 17:32:40.248620 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:40.248585    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f34c5540-3adf-48e7-a459-9482aab37685-openshift-service-ca\") pod \"maas-api-6d669c7fcd-qkt2f\" (UID: \"f34c5540-3adf-48e7-a459-9482aab37685\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-qkt2f"
Oct 31 17:32:40.248770 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:40.248636    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sbtdb\" (UniqueName: \"kubernetes.io/projected/f34c5540-3adf-48e7-a459-9482aab37685-kube-api-access-sbtdb\") pod \"maas-api-6d669c7fcd-qkt2f\" (UID: \"f34c5540-3adf-48e7-a459-9482aab37685\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-qkt2f"
Oct 31 17:32:40.248770 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:40.248663    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f34c5540-3adf-48e7-a459-9482aab37685-maas-api-tls\") pod \"maas-api-6d669c7fcd-qkt2f\" (UID: \"f34c5540-3adf-48e7-a459-9482aab37685\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-qkt2f"
Oct 31 17:32:40.248770 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:40.248700    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f34c5540-3adf-48e7-a459-9482aab37685-metrics-certs\") pod \"maas-api-6d669c7fcd-qkt2f\" (UID: \"f34c5540-3adf-48e7-a459-9482aab37685\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-qkt2f"
Oct 31 17:32:40.249365 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:40.249337    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f34c5540-3adf-48e7-a459-9482aab37685-openshift-service-ca\") pod \"maas-api-6d669c7fcd-qkt2f\" (UID: \"f34c5540-3adf-48e7-a459-9482aab37685\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-qkt2f"
Oct 31 17:32:40.251154 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:40.251132    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f34c5540-3adf-48e7-a459-9482aab37685-maas-api-tls\") pod \"maas-api-6d669c7fcd-qkt2f\" (UID: \"f34c5540-3adf-48e7-a459-9482aab37685\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-qkt2f"
Oct 31 17:32:40.251154 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:40.251146    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f34c5540-3adf-48e7-a459-9482aab37685-metrics-certs\") pod \"maas-api-6d669c7fcd-qkt2f\" (UID: \"f34c5540-3adf-48e7-a459-9482aab37685\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-qkt2f"
Oct 31 17:32:40.268491 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:40.268466    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sbtdb\" (UniqueName: \"kubernetes.io/projected/f34c5540-3adf-48e7-a459-9482aab37685-kube-api-access-sbtdb\") pod \"maas-api-6d669c7fcd-qkt2f\" (UID: \"f34c5540-3adf-48e7-a459-9482aab37685\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-qkt2f"
Oct 31 17:32:40.375347 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:40.375276    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-qkt2f"
Oct 31 17:32:40.673128 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:40.673106    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6d669c7fcd-qkt2f"]
Oct 31 17:32:40.675143 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:32:40.675114    4276 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf34c5540_3adf_48e7_a459_9482aab37685.slice/crio-d10a4e56d4a876b93c6cbeba28a3760d21a57417e4afcc1936a6de9f4060fc7b WatchSource:0}: Error finding container d10a4e56d4a876b93c6cbeba28a3760d21a57417e4afcc1936a6de9f4060fc7b: Status 404 returned error can't find the container with id d10a4e56d4a876b93c6cbeba28a3760d21a57417e4afcc1936a6de9f4060fc7b
Oct 31 17:32:40.676347 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:40.676328    4276 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 31 17:32:41.595617 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:41.595567    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-qkt2f" event={"ID":"f34c5540-3adf-48e7-a459-9482aab37685","Type":"ContainerStarted","Data":"d10a4e56d4a876b93c6cbeba28a3760d21a57417e4afcc1936a6de9f4060fc7b"}
Oct 31 17:32:42.237345 ip-10-0-129-127 kubenswrapper[4276]: E1031 17:32:42.237307    4276 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tg5qk" containerName="extractor"
Oct 31 17:32:43.605157 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:43.605120    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-qkt2f" event={"ID":"f34c5540-3adf-48e7-a459-9482aab37685","Type":"ContainerStarted","Data":"55278e211ca93ed9e104e6e73beb9e2b560de0d4020e7c8fc20784a81c2ad867"}
Oct 31 17:32:43.605581 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:43.605250    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-qkt2f"
Oct 31 17:32:43.647664 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:43.647620    4276 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-qkt2f" podStartSLOduration=1.422479404 podStartE2EDuration="3.647606645s" totalImagesPullingTime="2.225127241s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 17:32:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 17:32:43.646085234 +0000 UTC m=+326.443830162" watchObservedRunningTime="2026-10-31 17:32:43.647606645 +0000 UTC m=+326.445351572"
Oct 31 17:32:49.614852 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:49.614824    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-qkt2f"
Oct 31 17:32:50.065053 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:50.065020    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:32:50.446686 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:50.446618    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:32:50.518843 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:50.518816    4276 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-7bb55cd9d-x784h"]
Oct 31 17:32:50.524736 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:50.524717    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7bb55cd9d-x784h"
Oct 31 17:32:50.549084 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:50.549062    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-7bb55cd9d-x784h"]
Oct 31 17:32:50.629272 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:50.629247    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/87635e23-0e63-46d0-b92f-5f37b5720cc9-metrics-certs\") pod \"maas-api-7bb55cd9d-x784h\" (UID: \"87635e23-0e63-46d0-b92f-5f37b5720cc9\") " pod="odh-ai-gateway-infra/maas-api-7bb55cd9d-x784h"
Oct 31 17:32:50.629662 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:50.629280    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/87635e23-0e63-46d0-b92f-5f37b5720cc9-maas-api-tls\") pod \"maas-api-7bb55cd9d-x784h\" (UID: \"87635e23-0e63-46d0-b92f-5f37b5720cc9\") " pod="odh-ai-gateway-infra/maas-api-7bb55cd9d-x784h"
Oct 31 17:32:50.629662 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:50.629307    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/87635e23-0e63-46d0-b92f-5f37b5720cc9-openshift-service-ca\") pod \"maas-api-7bb55cd9d-x784h\" (UID: \"87635e23-0e63-46d0-b92f-5f37b5720cc9\") " pod="odh-ai-gateway-infra/maas-api-7bb55cd9d-x784h"
Oct 31 17:32:50.629662 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:50.629429    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cn2cp\" (UniqueName: \"kubernetes.io/projected/87635e23-0e63-46d0-b92f-5f37b5720cc9-kube-api-access-cn2cp\") pod \"maas-api-7bb55cd9d-x784h\" (UID: \"87635e23-0e63-46d0-b92f-5f37b5720cc9\") " pod="odh-ai-gateway-infra/maas-api-7bb55cd9d-x784h"
Oct 31 17:32:50.730497 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:50.730422    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/87635e23-0e63-46d0-b92f-5f37b5720cc9-metrics-certs\") pod \"maas-api-7bb55cd9d-x784h\" (UID: \"87635e23-0e63-46d0-b92f-5f37b5720cc9\") " pod="odh-ai-gateway-infra/maas-api-7bb55cd9d-x784h"
Oct 31 17:32:50.730497 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:50.730483    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/87635e23-0e63-46d0-b92f-5f37b5720cc9-maas-api-tls\") pod \"maas-api-7bb55cd9d-x784h\" (UID: \"87635e23-0e63-46d0-b92f-5f37b5720cc9\") " pod="odh-ai-gateway-infra/maas-api-7bb55cd9d-x784h"
Oct 31 17:32:50.730686 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:50.730515    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/87635e23-0e63-46d0-b92f-5f37b5720cc9-openshift-service-ca\") pod \"maas-api-7bb55cd9d-x784h\" (UID: \"87635e23-0e63-46d0-b92f-5f37b5720cc9\") " pod="odh-ai-gateway-infra/maas-api-7bb55cd9d-x784h"
Oct 31 17:32:50.730686 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:50.730540    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cn2cp\" (UniqueName: \"kubernetes.io/projected/87635e23-0e63-46d0-b92f-5f37b5720cc9-kube-api-access-cn2cp\") pod \"maas-api-7bb55cd9d-x784h\" (UID: \"87635e23-0e63-46d0-b92f-5f37b5720cc9\") " pod="odh-ai-gateway-infra/maas-api-7bb55cd9d-x784h"
Oct 31 17:32:50.731182 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:50.731160    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/87635e23-0e63-46d0-b92f-5f37b5720cc9-openshift-service-ca\") pod \"maas-api-7bb55cd9d-x784h\" (UID: \"87635e23-0e63-46d0-b92f-5f37b5720cc9\") " pod="odh-ai-gateway-infra/maas-api-7bb55cd9d-x784h"
Oct 31 17:32:50.732901 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:50.732878    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/87635e23-0e63-46d0-b92f-5f37b5720cc9-maas-api-tls\") pod \"maas-api-7bb55cd9d-x784h\" (UID: \"87635e23-0e63-46d0-b92f-5f37b5720cc9\") " pod="odh-ai-gateway-infra/maas-api-7bb55cd9d-x784h"
Oct 31 17:32:50.732989 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:50.732930    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/87635e23-0e63-46d0-b92f-5f37b5720cc9-metrics-certs\") pod \"maas-api-7bb55cd9d-x784h\" (UID: \"87635e23-0e63-46d0-b92f-5f37b5720cc9\") " pod="odh-ai-gateway-infra/maas-api-7bb55cd9d-x784h"
Oct 31 17:32:50.750584 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:50.750556    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cn2cp\" (UniqueName: \"kubernetes.io/projected/87635e23-0e63-46d0-b92f-5f37b5720cc9-kube-api-access-cn2cp\") pod \"maas-api-7bb55cd9d-x784h\" (UID: \"87635e23-0e63-46d0-b92f-5f37b5720cc9\") " pod="odh-ai-gateway-infra/maas-api-7bb55cd9d-x784h"
Oct 31 17:32:50.838138 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:50.838109    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7bb55cd9d-x784h"
Oct 31 17:32:51.136438 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:51.136405    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-7bb55cd9d-x784h"]
Oct 31 17:32:51.139505 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:32:51.139478    4276 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87635e23_0e63_46d0_b92f_5f37b5720cc9.slice/crio-285998e90b39c10a09d9a1a56bfc34a78219b293f44ad0f8637d74ffc9164c6a WatchSource:0}: Error finding container 285998e90b39c10a09d9a1a56bfc34a78219b293f44ad0f8637d74ffc9164c6a: Status 404 returned error can't find the container with id 285998e90b39c10a09d9a1a56bfc34a78219b293f44ad0f8637d74ffc9164c6a
Oct 31 17:32:51.637965 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:51.637935    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7bb55cd9d-x784h" event={"ID":"87635e23-0e63-46d0-b92f-5f37b5720cc9","Type":"ContainerStarted","Data":"9a3a2a77b4db4c8d11a71a8c5e99571116ee76858fe8ac41610325495a281d8d"}
Oct 31 17:32:51.638356 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:51.637986    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7bb55cd9d-x784h" event={"ID":"87635e23-0e63-46d0-b92f-5f37b5720cc9","Type":"ContainerStarted","Data":"285998e90b39c10a09d9a1a56bfc34a78219b293f44ad0f8637d74ffc9164c6a"}
Oct 31 17:32:51.638356 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:51.638085    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-7bb55cd9d-x784h"
Oct 31 17:32:51.679179 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:51.679130    4276 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-7bb55cd9d-x784h" podStartSLOduration=1.350303815 podStartE2EDuration="1.679117731s" totalImagesPullingTime="328.813916ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 17:32:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 17:32:51.67838193 +0000 UTC m=+334.476126872" watchObservedRunningTime="2026-10-31 17:32:51.679117731 +0000 UTC m=+334.476862658"
Oct 31 17:32:57.647724 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:57.647689    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-7bb55cd9d-x784h"
Oct 31 17:32:57.713915 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:57.713886    4276 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-6d669c7fcd-qkt2f"]
Oct 31 17:32:57.714143 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:57.714100    4276 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-qkt2f" podUID="f34c5540-3adf-48e7-a459-9482aab37685" containerName="maas-api" containerID="cri-o://55278e211ca93ed9e104e6e73beb9e2b560de0d4020e7c8fc20784a81c2ad867" gracePeriod=30
Oct 31 17:32:57.960496 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:57.960468    4276 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-qkt2f"
Oct 31 17:32:58.090949 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:58.090917    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f34c5540-3adf-48e7-a459-9482aab37685-kube-api-access-sbtdb\" (UniqueName: \"kubernetes.io/projected/f34c5540-3adf-48e7-a459-9482aab37685-kube-api-access-sbtdb\") pod \"f34c5540-3adf-48e7-a459-9482aab37685\" (UID: \"f34c5540-3adf-48e7-a459-9482aab37685\") "
Oct 31 17:32:58.091114 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:58.091008    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f34c5540-3adf-48e7-a459-9482aab37685-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f34c5540-3adf-48e7-a459-9482aab37685-maas-api-tls\") pod \"f34c5540-3adf-48e7-a459-9482aab37685\" (UID: \"f34c5540-3adf-48e7-a459-9482aab37685\") "
Oct 31 17:32:58.091114 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:58.091046    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f34c5540-3adf-48e7-a459-9482aab37685-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f34c5540-3adf-48e7-a459-9482aab37685-openshift-service-ca\") pod \"f34c5540-3adf-48e7-a459-9482aab37685\" (UID: \"f34c5540-3adf-48e7-a459-9482aab37685\") "
Oct 31 17:32:58.091114 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:58.091078    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f34c5540-3adf-48e7-a459-9482aab37685-metrics-certs\" (UniqueName: \"kubernetes.io/secret/f34c5540-3adf-48e7-a459-9482aab37685-metrics-certs\") pod \"f34c5540-3adf-48e7-a459-9482aab37685\" (UID: \"f34c5540-3adf-48e7-a459-9482aab37685\") "
Oct 31 17:32:58.091458 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:58.091408    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f34c5540-3adf-48e7-a459-9482aab37685-openshift-service-ca" pod "f34c5540-3adf-48e7-a459-9482aab37685" (UID: "f34c5540-3adf-48e7-a459-9482aab37685"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 17:32:58.093149 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:58.093124    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f34c5540-3adf-48e7-a459-9482aab37685-maas-api-tls" pod "f34c5540-3adf-48e7-a459-9482aab37685" (UID: "f34c5540-3adf-48e7-a459-9482aab37685"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 17:32:58.093222 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:58.093143    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f34c5540-3adf-48e7-a459-9482aab37685-metrics-certs" pod "f34c5540-3adf-48e7-a459-9482aab37685" (UID: "f34c5540-3adf-48e7-a459-9482aab37685"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 17:32:58.093222 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:58.093210    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f34c5540-3adf-48e7-a459-9482aab37685-kube-api-access-sbtdb" pod "f34c5540-3adf-48e7-a459-9482aab37685" (UID: "f34c5540-3adf-48e7-a459-9482aab37685"). InnerVolumeSpecName "kube-api-access-sbtdb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 17:32:58.192165 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:58.192095    4276 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f34c5540-3adf-48e7-a459-9482aab37685-maas-api-tls\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:32:58.192165 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:58.192119    4276 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f34c5540-3adf-48e7-a459-9482aab37685-openshift-service-ca\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:32:58.192165 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:58.192128    4276 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f34c5540-3adf-48e7-a459-9482aab37685-metrics-certs\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:32:58.192165 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:58.192137    4276 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sbtdb\" (UniqueName: \"kubernetes.io/projected/f34c5540-3adf-48e7-a459-9482aab37685-kube-api-access-sbtdb\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:32:58.664945 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:58.664909    4276 generic.go:356] "Generic (PLEG): container finished" podID="f34c5540-3adf-48e7-a459-9482aab37685" containerID="55278e211ca93ed9e104e6e73beb9e2b560de0d4020e7c8fc20784a81c2ad867" exitCode=0
Oct 31 17:32:58.665427 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:58.664955    4276 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-qkt2f"
Oct 31 17:32:58.665427 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:58.664994    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-qkt2f" event={"ID":"f34c5540-3adf-48e7-a459-9482aab37685","Type":"ContainerDied","Data":"55278e211ca93ed9e104e6e73beb9e2b560de0d4020e7c8fc20784a81c2ad867"}
Oct 31 17:32:58.665427 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:58.665047    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-qkt2f" event={"ID":"f34c5540-3adf-48e7-a459-9482aab37685","Type":"ContainerDied","Data":"d10a4e56d4a876b93c6cbeba28a3760d21a57417e4afcc1936a6de9f4060fc7b"}
Oct 31 17:32:58.665427 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:58.665069    4276 scope.go:122] "RemoveContainer" containerID="55278e211ca93ed9e104e6e73beb9e2b560de0d4020e7c8fc20784a81c2ad867"
Oct 31 17:32:58.673344 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:58.673328    4276 scope.go:122] "RemoveContainer" containerID="55278e211ca93ed9e104e6e73beb9e2b560de0d4020e7c8fc20784a81c2ad867"
Oct 31 17:32:58.673621 ip-10-0-129-127 kubenswrapper[4276]: E1031 17:32:58.673604    4276 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"55278e211ca93ed9e104e6e73beb9e2b560de0d4020e7c8fc20784a81c2ad867\": container with ID starting with 55278e211ca93ed9e104e6e73beb9e2b560de0d4020e7c8fc20784a81c2ad867 not found: ID does not exist" containerID="55278e211ca93ed9e104e6e73beb9e2b560de0d4020e7c8fc20784a81c2ad867"
Oct 31 17:32:58.673679 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:58.673631    4276 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"55278e211ca93ed9e104e6e73beb9e2b560de0d4020e7c8fc20784a81c2ad867"} err="failed to get container status \"55278e211ca93ed9e104e6e73beb9e2b560de0d4020e7c8fc20784a81c2ad867\": rpc error: code = NotFound desc = could not find container \"55278e211ca93ed9e104e6e73beb9e2b560de0d4020e7c8fc20784a81c2ad867\": container with ID starting with 55278e211ca93ed9e104e6e73beb9e2b560de0d4020e7c8fc20784a81c2ad867 not found: ID does not exist"
Oct 31 17:32:58.718281 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:58.718256    4276 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-6d669c7fcd-qkt2f"]
Oct 31 17:32:58.731956 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:58.731936    4276 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-6d669c7fcd-qkt2f"]
Oct 31 17:32:59.369950 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:32:59.369883    4276 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f34c5540-3adf-48e7-a459-9482aab37685" path="/var/lib/kubelet/pods/f34c5540-3adf-48e7-a459-9482aab37685/volumes"
Oct 31 17:33:01.861722 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:33:01.861690    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:33:12.233576 ip-10-0-129-127 kubenswrapper[4276]: E1031 17:33:12.233539    4276 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tg5qk" containerName="extractor"
Oct 31 17:33:42.233515 ip-10-0-129-127 kubenswrapper[4276]: E1031 17:33:42.233473    4276 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tg5qk" containerName="extractor"
Oct 31 17:33:59.500561 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:33:59.500515    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:34:01.999269 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:01.999170    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:34:12.233616 ip-10-0-129-127 kubenswrapper[4276]: E1031 17:34:12.233579    4276 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tg5qk" containerName="extractor"
Oct 31 17:34:21.028174 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:21.028142    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:34:21.149130 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:21.149093    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:34:25.184032 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:25.183995    4276 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-cbd4f5586-bzprg"]
Oct 31 17:34:25.184492 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:25.184293    4276 cpu_manager.go:414] "containerMap: removing container" podUID="f34c5540-3adf-48e7-a459-9482aab37685" containerName="maas-api"
Oct 31 17:34:25.184492 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:25.184308    4276 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f34c5540-3adf-48e7-a459-9482aab37685" containerName="maas-api"
Oct 31 17:34:25.184492 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:25.184392    4276 memory_manager.go:365] "RemoveStaleState removing state" podUID="f34c5540-3adf-48e7-a459-9482aab37685" containerName="maas-api"
Oct 31 17:34:25.188817 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:25.188789    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-bzprg"
Oct 31 17:34:25.205587 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:25.205562    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-cbd4f5586-bzprg"]
Oct 31 17:34:25.218292 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:25.218268    4276 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Oct 31 17:34:25.218433 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:25.218415    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-vhxrl\""
Oct 31 17:34:25.218433 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:25.218424    4276 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Oct 31 17:34:25.218555 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:25.218472    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Oct 31 17:34:25.258792 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:25.258764    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:34:25.265895 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:25.265877    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8f8e3a21-f02f-4bcd-94f5-587416670156-home\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-bzprg\" (UID: \"8f8e3a21-f02f-4bcd-94f5-587416670156\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-bzprg"
Oct 31 17:34:25.265950 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:25.265909    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8f8e3a21-f02f-4bcd-94f5-587416670156-model-cache\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-bzprg\" (UID: \"8f8e3a21-f02f-4bcd-94f5-587416670156\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-bzprg"
Oct 31 17:34:25.265950 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:25.265929    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bxdx4\" (UniqueName: \"kubernetes.io/projected/8f8e3a21-f02f-4bcd-94f5-587416670156-kube-api-access-bxdx4\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-bzprg\" (UID: \"8f8e3a21-f02f-4bcd-94f5-587416670156\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-bzprg"
Oct 31 17:34:25.266043 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:25.266028    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8f8e3a21-f02f-4bcd-94f5-587416670156-dshm\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-bzprg\" (UID: \"8f8e3a21-f02f-4bcd-94f5-587416670156\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-bzprg"
Oct 31 17:34:25.266094 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:25.266082    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8f8e3a21-f02f-4bcd-94f5-587416670156-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-bzprg\" (UID: \"8f8e3a21-f02f-4bcd-94f5-587416670156\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-bzprg"
Oct 31 17:34:25.266170 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:25.266157    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8f8e3a21-f02f-4bcd-94f5-587416670156-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-bzprg\" (UID: \"8f8e3a21-f02f-4bcd-94f5-587416670156\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-bzprg"
Oct 31 17:34:25.366663 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:25.366632    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8f8e3a21-f02f-4bcd-94f5-587416670156-home\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-bzprg\" (UID: \"8f8e3a21-f02f-4bcd-94f5-587416670156\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-bzprg"
Oct 31 17:34:25.366663 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:25.366669    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8f8e3a21-f02f-4bcd-94f5-587416670156-model-cache\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-bzprg\" (UID: \"8f8e3a21-f02f-4bcd-94f5-587416670156\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-bzprg"
Oct 31 17:34:25.366907 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:25.366685    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bxdx4\" (UniqueName: \"kubernetes.io/projected/8f8e3a21-f02f-4bcd-94f5-587416670156-kube-api-access-bxdx4\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-bzprg\" (UID: \"8f8e3a21-f02f-4bcd-94f5-587416670156\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-bzprg"
Oct 31 17:34:25.366907 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:25.366705    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8f8e3a21-f02f-4bcd-94f5-587416670156-dshm\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-bzprg\" (UID: \"8f8e3a21-f02f-4bcd-94f5-587416670156\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-bzprg"
Oct 31 17:34:25.366907 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:25.366723    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8f8e3a21-f02f-4bcd-94f5-587416670156-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-bzprg\" (UID: \"8f8e3a21-f02f-4bcd-94f5-587416670156\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-bzprg"
Oct 31 17:34:25.366907 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:25.366770    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8f8e3a21-f02f-4bcd-94f5-587416670156-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-bzprg\" (UID: \"8f8e3a21-f02f-4bcd-94f5-587416670156\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-bzprg"
Oct 31 17:34:25.367130 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:25.367106    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8f8e3a21-f02f-4bcd-94f5-587416670156-model-cache\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-bzprg\" (UID: \"8f8e3a21-f02f-4bcd-94f5-587416670156\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-bzprg"
Oct 31 17:34:25.367184 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:25.367128    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8f8e3a21-f02f-4bcd-94f5-587416670156-home\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-bzprg\" (UID: \"8f8e3a21-f02f-4bcd-94f5-587416670156\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-bzprg"
Oct 31 17:34:25.367184 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:25.367170    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8f8e3a21-f02f-4bcd-94f5-587416670156-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-bzprg\" (UID: \"8f8e3a21-f02f-4bcd-94f5-587416670156\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-bzprg"
Oct 31 17:34:25.369010 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:25.368983    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8f8e3a21-f02f-4bcd-94f5-587416670156-dshm\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-bzprg\" (UID: \"8f8e3a21-f02f-4bcd-94f5-587416670156\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-bzprg"
Oct 31 17:34:25.369244 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:25.369227    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8f8e3a21-f02f-4bcd-94f5-587416670156-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-bzprg\" (UID: \"8f8e3a21-f02f-4bcd-94f5-587416670156\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-bzprg"
Oct 31 17:34:25.386971 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:25.386943    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bxdx4\" (UniqueName: \"kubernetes.io/projected/8f8e3a21-f02f-4bcd-94f5-587416670156-kube-api-access-bxdx4\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-bzprg\" (UID: \"8f8e3a21-f02f-4bcd-94f5-587416670156\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-bzprg"
Oct 31 17:34:25.502303 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:25.502229    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-bzprg"
Oct 31 17:34:25.801675 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:25.801656    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-cbd4f5586-bzprg"]
Oct 31 17:34:25.803474 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:34:25.803421    4276 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f8e3a21_f02f_4bcd_94f5_587416670156.slice/crio-a0d0c9262e074dc0bd7284837907abe89304650395350cca7ba3d9988953f903 WatchSource:0}: Error finding container a0d0c9262e074dc0bd7284837907abe89304650395350cca7ba3d9988953f903: Status 404 returned error can't find the container with id a0d0c9262e074dc0bd7284837907abe89304650395350cca7ba3d9988953f903
Oct 31 17:34:25.995678 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:25.995644    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-bzprg" event={"ID":"8f8e3a21-f02f-4bcd-94f5-587416670156","Type":"ContainerStarted","Data":"a0d0c9262e074dc0bd7284837907abe89304650395350cca7ba3d9988953f903"}
Oct 31 17:34:27.001347 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:27.001311    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-bzprg" event={"ID":"8f8e3a21-f02f-4bcd-94f5-587416670156","Type":"ContainerStarted","Data":"3af9373896851b6420236eb2ed219d608e44ec743f944ff736e4b0836400b7a2"}
Oct 31 17:34:27.043984 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:27.043935    4276 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-bzprg" podStartSLOduration=0.953927825 podStartE2EDuration="2.043922782s" totalImagesPullingTime="1.089994957s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 17:34:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 17:34:27.042982257 +0000 UTC m=+429.840727186" watchObservedRunningTime="2026-10-31 17:34:27.043922782 +0000 UTC m=+429.841667731"
Oct 31 17:34:27.503329 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:27.503290    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-bzprg"
Oct 31 17:34:28.769938 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:28.769903    4276 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-7b998b79d9-mpnd7"]
Oct 31 17:34:28.773775 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:28.773750    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-mpnd7"
Oct 31 17:34:28.791083 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:28.791059    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-7b998b79d9-mpnd7"]
Oct 31 17:34:28.803561 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:28.803541    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Oct 31 17:34:28.852616 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:28.852592    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:34:28.898226 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:28.898202    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1800b0cf-d90b-455f-a6d3-ece0eea69cec-tls-certs\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-mpnd7\" (UID: \"1800b0cf-d90b-455f-a6d3-ece0eea69cec\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-mpnd7"
Oct 31 17:34:28.898317 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:28.898236    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1800b0cf-d90b-455f-a6d3-ece0eea69cec-tmp-dir\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-mpnd7\" (UID: \"1800b0cf-d90b-455f-a6d3-ece0eea69cec\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-mpnd7"
Oct 31 17:34:28.898317 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:28.898272    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1800b0cf-d90b-455f-a6d3-ece0eea69cec-home\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-mpnd7\" (UID: \"1800b0cf-d90b-455f-a6d3-ece0eea69cec\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-mpnd7"
Oct 31 17:34:28.898420 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:28.898401    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1800b0cf-d90b-455f-a6d3-ece0eea69cec-model-cache\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-mpnd7\" (UID: \"1800b0cf-d90b-455f-a6d3-ece0eea69cec\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-mpnd7"
Oct 31 17:34:28.898491 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:28.898477    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1800b0cf-d90b-455f-a6d3-ece0eea69cec-dshm\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-mpnd7\" (UID: \"1800b0cf-d90b-455f-a6d3-ece0eea69cec\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-mpnd7"
Oct 31 17:34:28.898530 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:28.898517    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w44wj\" (UniqueName: \"kubernetes.io/projected/1800b0cf-d90b-455f-a6d3-ece0eea69cec-kube-api-access-w44wj\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-mpnd7\" (UID: \"1800b0cf-d90b-455f-a6d3-ece0eea69cec\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-mpnd7"
Oct 31 17:34:28.999567 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:28.999540    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1800b0cf-d90b-455f-a6d3-ece0eea69cec-home\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-mpnd7\" (UID: \"1800b0cf-d90b-455f-a6d3-ece0eea69cec\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-mpnd7"
Oct 31 17:34:28.999739 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:28.999600    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1800b0cf-d90b-455f-a6d3-ece0eea69cec-model-cache\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-mpnd7\" (UID: \"1800b0cf-d90b-455f-a6d3-ece0eea69cec\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-mpnd7"
Oct 31 17:34:28.999739 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:28.999657    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1800b0cf-d90b-455f-a6d3-ece0eea69cec-dshm\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-mpnd7\" (UID: \"1800b0cf-d90b-455f-a6d3-ece0eea69cec\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-mpnd7"
Oct 31 17:34:28.999739 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:28.999684    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w44wj\" (UniqueName: \"kubernetes.io/projected/1800b0cf-d90b-455f-a6d3-ece0eea69cec-kube-api-access-w44wj\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-mpnd7\" (UID: \"1800b0cf-d90b-455f-a6d3-ece0eea69cec\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-mpnd7"
Oct 31 17:34:28.999739 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:28.999710    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1800b0cf-d90b-455f-a6d3-ece0eea69cec-tls-certs\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-mpnd7\" (UID: \"1800b0cf-d90b-455f-a6d3-ece0eea69cec\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-mpnd7"
Oct 31 17:34:28.999954 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:28.999919    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1800b0cf-d90b-455f-a6d3-ece0eea69cec-tmp-dir\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-mpnd7\" (UID: \"1800b0cf-d90b-455f-a6d3-ece0eea69cec\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-mpnd7"
Oct 31 17:34:29.000091 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:29.000068    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1800b0cf-d90b-455f-a6d3-ece0eea69cec-model-cache\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-mpnd7\" (UID: \"1800b0cf-d90b-455f-a6d3-ece0eea69cec\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-mpnd7"
Oct 31 17:34:29.000198 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:29.000091    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1800b0cf-d90b-455f-a6d3-ece0eea69cec-home\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-mpnd7\" (UID: \"1800b0cf-d90b-455f-a6d3-ece0eea69cec\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-mpnd7"
Oct 31 17:34:29.000333 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:29.000303    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1800b0cf-d90b-455f-a6d3-ece0eea69cec-tmp-dir\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-mpnd7\" (UID: \"1800b0cf-d90b-455f-a6d3-ece0eea69cec\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-mpnd7"
Oct 31 17:34:29.001960 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:29.001936    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1800b0cf-d90b-455f-a6d3-ece0eea69cec-dshm\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-mpnd7\" (UID: \"1800b0cf-d90b-455f-a6d3-ece0eea69cec\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-mpnd7"
Oct 31 17:34:29.002158 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:29.002140    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1800b0cf-d90b-455f-a6d3-ece0eea69cec-tls-certs\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-mpnd7\" (UID: \"1800b0cf-d90b-455f-a6d3-ece0eea69cec\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-mpnd7"
Oct 31 17:34:29.021185 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:29.021136    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w44wj\" (UniqueName: \"kubernetes.io/projected/1800b0cf-d90b-455f-a6d3-ece0eea69cec-kube-api-access-w44wj\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-mpnd7\" (UID: \"1800b0cf-d90b-455f-a6d3-ece0eea69cec\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-mpnd7"
Oct 31 17:34:29.087068 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:29.087046    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-mpnd7"
Oct 31 17:34:29.381461 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:29.381418    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-7b998b79d9-mpnd7"]
Oct 31 17:34:29.383179 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:34:29.383155    4276 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1800b0cf_d90b_455f_a6d3_ece0eea69cec.slice/crio-b7b24d2098fba15980f06171de816b02a0b6938af58ac2afbb9183affe91d92b WatchSource:0}: Error finding container b7b24d2098fba15980f06171de816b02a0b6938af58ac2afbb9183affe91d92b: Status 404 returned error can't find the container with id b7b24d2098fba15980f06171de816b02a0b6938af58ac2afbb9183affe91d92b
Oct 31 17:34:30.014742 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:30.014696    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-mpnd7" event={"ID":"1800b0cf-d90b-455f-a6d3-ece0eea69cec","Type":"ContainerStarted","Data":"882a8d5458417c70c4ffb9c7e21a8d98871b9379f33cb7269b7cee54fc33a747"}
Oct 31 17:34:30.015143 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:30.014748    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-mpnd7" event={"ID":"1800b0cf-d90b-455f-a6d3-ece0eea69cec","Type":"ContainerStarted","Data":"b7b24d2098fba15980f06171de816b02a0b6938af58ac2afbb9183affe91d92b"}
Oct 31 17:34:30.059815 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:30.059770    4276 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-mpnd7" podStartSLOduration=1.898849222 podStartE2EDuration="2.059756784s" totalImagesPullingTime="160.907562ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 17:34:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 17:34:30.0588843 +0000 UTC m=+432.856629227" watchObservedRunningTime="2026-10-31 17:34:30.059756784 +0000 UTC m=+432.857501712"
Oct 31 17:34:30.088103 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:30.088072    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-mpnd7"
Oct 31 17:34:30.213552 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:30.213518    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:34:32.066487 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:32.066435    4276 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-69b588897d-ctkqw"]
Oct 31 17:34:32.069724 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:32.069694    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-ctkqw"
Oct 31 17:34:32.091118 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:32.091097    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-69b588897d-ctkqw"]
Oct 31 17:34:32.100590 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:32.100574    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Oct 31 17:34:32.141031 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:32.140967    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:34:32.227270 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:32.227238    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cfd9048c-8ac0-4f5a-abec-71f640158e61-home\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-ctkqw\" (UID: \"cfd9048c-8ac0-4f5a-abec-71f640158e61\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-ctkqw"
Oct 31 17:34:32.227414 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:32.227296    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cfd9048c-8ac0-4f5a-abec-71f640158e61-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-ctkqw\" (UID: \"cfd9048c-8ac0-4f5a-abec-71f640158e61\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-ctkqw"
Oct 31 17:34:32.227414 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:32.227361    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cfd9048c-8ac0-4f5a-abec-71f640158e61-model-cache\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-ctkqw\" (UID: \"cfd9048c-8ac0-4f5a-abec-71f640158e61\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-ctkqw"
Oct 31 17:34:32.227526 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:32.227413    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cfd9048c-8ac0-4f5a-abec-71f640158e61-dshm\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-ctkqw\" (UID: \"cfd9048c-8ac0-4f5a-abec-71f640158e61\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-ctkqw"
Oct 31 17:34:32.227526 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:32.227433    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cfd9048c-8ac0-4f5a-abec-71f640158e61-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-ctkqw\" (UID: \"cfd9048c-8ac0-4f5a-abec-71f640158e61\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-ctkqw"
Oct 31 17:34:32.227526 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:32.227509    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9pvhh\" (UniqueName: \"kubernetes.io/projected/cfd9048c-8ac0-4f5a-abec-71f640158e61-kube-api-access-9pvhh\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-ctkqw\" (UID: \"cfd9048c-8ac0-4f5a-abec-71f640158e61\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-ctkqw"
Oct 31 17:34:32.328111 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:32.328030    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cfd9048c-8ac0-4f5a-abec-71f640158e61-model-cache\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-ctkqw\" (UID: \"cfd9048c-8ac0-4f5a-abec-71f640158e61\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-ctkqw"
Oct 31 17:34:32.328111 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:32.328079    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cfd9048c-8ac0-4f5a-abec-71f640158e61-dshm\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-ctkqw\" (UID: \"cfd9048c-8ac0-4f5a-abec-71f640158e61\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-ctkqw"
Oct 31 17:34:32.328111 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:32.328094    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cfd9048c-8ac0-4f5a-abec-71f640158e61-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-ctkqw\" (UID: \"cfd9048c-8ac0-4f5a-abec-71f640158e61\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-ctkqw"
Oct 31 17:34:32.328367 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:32.328212    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9pvhh\" (UniqueName: \"kubernetes.io/projected/cfd9048c-8ac0-4f5a-abec-71f640158e61-kube-api-access-9pvhh\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-ctkqw\" (UID: \"cfd9048c-8ac0-4f5a-abec-71f640158e61\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-ctkqw"
Oct 31 17:34:32.328367 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:32.328255    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cfd9048c-8ac0-4f5a-abec-71f640158e61-home\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-ctkqw\" (UID: \"cfd9048c-8ac0-4f5a-abec-71f640158e61\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-ctkqw"
Oct 31 17:34:32.328509 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:32.328485    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cfd9048c-8ac0-4f5a-abec-71f640158e61-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-ctkqw\" (UID: \"cfd9048c-8ac0-4f5a-abec-71f640158e61\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-ctkqw"
Oct 31 17:34:32.328569 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:32.328515    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cfd9048c-8ac0-4f5a-abec-71f640158e61-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-ctkqw\" (UID: \"cfd9048c-8ac0-4f5a-abec-71f640158e61\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-ctkqw"
Oct 31 17:34:32.328625 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:32.328582    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cfd9048c-8ac0-4f5a-abec-71f640158e61-model-cache\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-ctkqw\" (UID: \"cfd9048c-8ac0-4f5a-abec-71f640158e61\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-ctkqw"
Oct 31 17:34:32.328664 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:32.328636    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cfd9048c-8ac0-4f5a-abec-71f640158e61-home\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-ctkqw\" (UID: \"cfd9048c-8ac0-4f5a-abec-71f640158e61\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-ctkqw"
Oct 31 17:34:32.330239 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:32.330214    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cfd9048c-8ac0-4f5a-abec-71f640158e61-dshm\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-ctkqw\" (UID: \"cfd9048c-8ac0-4f5a-abec-71f640158e61\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-ctkqw"
Oct 31 17:34:32.330732 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:32.330713    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cfd9048c-8ac0-4f5a-abec-71f640158e61-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-ctkqw\" (UID: \"cfd9048c-8ac0-4f5a-abec-71f640158e61\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-ctkqw"
Oct 31 17:34:32.348461 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:32.348427    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9pvhh\" (UniqueName: \"kubernetes.io/projected/cfd9048c-8ac0-4f5a-abec-71f640158e61-kube-api-access-9pvhh\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-ctkqw\" (UID: \"cfd9048c-8ac0-4f5a-abec-71f640158e61\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-ctkqw"
Oct 31 17:34:32.383317 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:32.383283    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-ctkqw"
Oct 31 17:34:32.680739 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:32.680718    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-69b588897d-ctkqw"]
Oct 31 17:34:32.682800 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:34:32.682775    4276 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcfd9048c_8ac0_4f5a_abec_71f640158e61.slice/crio-5d8f2d0fbb42c5f3f43b77c0b34e9ab74a1762169182d0f148211ddfe62b5e91 WatchSource:0}: Error finding container 5d8f2d0fbb42c5f3f43b77c0b34e9ab74a1762169182d0f148211ddfe62b5e91: Status 404 returned error can't find the container with id 5d8f2d0fbb42c5f3f43b77c0b34e9ab74a1762169182d0f148211ddfe62b5e91
Oct 31 17:34:33.028515 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:33.028425    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-ctkqw" event={"ID":"cfd9048c-8ac0-4f5a-abec-71f640158e61","Type":"ContainerStarted","Data":"d16570fd784221294e65adcc12bbe3d99c8a4b68a9c450f3e76825a6a697aa6b"}
Oct 31 17:34:33.028515 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:33.028500    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-ctkqw" event={"ID":"cfd9048c-8ac0-4f5a-abec-71f640158e61","Type":"ContainerStarted","Data":"5d8f2d0fbb42c5f3f43b77c0b34e9ab74a1762169182d0f148211ddfe62b5e91"}
Oct 31 17:34:33.070189 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:33.070143    4276 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-ctkqw" podStartSLOduration=0.885438727 podStartE2EDuration="1.070129149s" totalImagesPullingTime="184.690422ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 17:34:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 17:34:33.06895903 +0000 UTC m=+435.866703957" watchObservedRunningTime="2026-10-31 17:34:33.070129149 +0000 UTC m=+435.867874076"
Oct 31 17:34:33.383534 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:33.383495    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-ctkqw"
Oct 31 17:34:34.531206 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:34.531170    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:34:35.503004 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:35.502973    4276 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-bzprg"
Oct 31 17:34:35.515258 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:35.515231    4276 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-bzprg"
Oct 31 17:34:36.052626 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:36.052598    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-bzprg"
Oct 31 17:34:37.870812 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:37.870782    4276 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/echo-simulator-kserve-64b6f8f96c-68q2x"]
Oct 31 17:34:37.875168 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:37.875146    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/echo-simulator-kserve-64b6f8f96c-68q2x"
Oct 31 17:34:37.891061 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:37.891039    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"echo-simulator-kserve-self-signed-certs\""
Oct 31 17:34:37.906131 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:37.906110    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/echo-simulator-kserve-64b6f8f96c-68q2x"]
Oct 31 17:34:37.951390 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:37.951353    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:34:37.978741 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:37.978721    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bd5cf2da-6510-434d-81f9-579baecddaf3-tmp-dir\") pod \"echo-simulator-kserve-64b6f8f96c-68q2x\" (UID: \"bd5cf2da-6510-434d-81f9-579baecddaf3\") " pod="llm/echo-simulator-kserve-64b6f8f96c-68q2x"
Oct 31 17:34:37.978856 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:37.978747    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bd5cf2da-6510-434d-81f9-579baecddaf3-home\") pod \"echo-simulator-kserve-64b6f8f96c-68q2x\" (UID: \"bd5cf2da-6510-434d-81f9-579baecddaf3\") " pod="llm/echo-simulator-kserve-64b6f8f96c-68q2x"
Oct 31 17:34:37.978856 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:37.978781    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bd5cf2da-6510-434d-81f9-579baecddaf3-model-cache\") pod \"echo-simulator-kserve-64b6f8f96c-68q2x\" (UID: \"bd5cf2da-6510-434d-81f9-579baecddaf3\") " pod="llm/echo-simulator-kserve-64b6f8f96c-68q2x"
Oct 31 17:34:37.978856 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:37.978845    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5hrs2\" (UniqueName: \"kubernetes.io/projected/bd5cf2da-6510-434d-81f9-579baecddaf3-kube-api-access-5hrs2\") pod \"echo-simulator-kserve-64b6f8f96c-68q2x\" (UID: \"bd5cf2da-6510-434d-81f9-579baecddaf3\") " pod="llm/echo-simulator-kserve-64b6f8f96c-68q2x"
Oct 31 17:34:37.978985 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:37.978882    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bd5cf2da-6510-434d-81f9-579baecddaf3-tls-certs\") pod \"echo-simulator-kserve-64b6f8f96c-68q2x\" (UID: \"bd5cf2da-6510-434d-81f9-579baecddaf3\") " pod="llm/echo-simulator-kserve-64b6f8f96c-68q2x"
Oct 31 17:34:37.978985 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:37.978968    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bd5cf2da-6510-434d-81f9-579baecddaf3-dshm\") pod \"echo-simulator-kserve-64b6f8f96c-68q2x\" (UID: \"bd5cf2da-6510-434d-81f9-579baecddaf3\") " pod="llm/echo-simulator-kserve-64b6f8f96c-68q2x"
Oct 31 17:34:38.079561 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:38.079524    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bd5cf2da-6510-434d-81f9-579baecddaf3-tmp-dir\") pod \"echo-simulator-kserve-64b6f8f96c-68q2x\" (UID: \"bd5cf2da-6510-434d-81f9-579baecddaf3\") " pod="llm/echo-simulator-kserve-64b6f8f96c-68q2x"
Oct 31 17:34:38.079561 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:38.079559    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bd5cf2da-6510-434d-81f9-579baecddaf3-home\") pod \"echo-simulator-kserve-64b6f8f96c-68q2x\" (UID: \"bd5cf2da-6510-434d-81f9-579baecddaf3\") " pod="llm/echo-simulator-kserve-64b6f8f96c-68q2x"
Oct 31 17:34:38.079784 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:38.079591    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bd5cf2da-6510-434d-81f9-579baecddaf3-model-cache\") pod \"echo-simulator-kserve-64b6f8f96c-68q2x\" (UID: \"bd5cf2da-6510-434d-81f9-579baecddaf3\") " pod="llm/echo-simulator-kserve-64b6f8f96c-68q2x"
Oct 31 17:34:38.079784 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:38.079612    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5hrs2\" (UniqueName: \"kubernetes.io/projected/bd5cf2da-6510-434d-81f9-579baecddaf3-kube-api-access-5hrs2\") pod \"echo-simulator-kserve-64b6f8f96c-68q2x\" (UID: \"bd5cf2da-6510-434d-81f9-579baecddaf3\") " pod="llm/echo-simulator-kserve-64b6f8f96c-68q2x"
Oct 31 17:34:38.079784 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:38.079641    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bd5cf2da-6510-434d-81f9-579baecddaf3-tls-certs\") pod \"echo-simulator-kserve-64b6f8f96c-68q2x\" (UID: \"bd5cf2da-6510-434d-81f9-579baecddaf3\") " pod="llm/echo-simulator-kserve-64b6f8f96c-68q2x"
Oct 31 17:34:38.079784 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:38.079687    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bd5cf2da-6510-434d-81f9-579baecddaf3-dshm\") pod \"echo-simulator-kserve-64b6f8f96c-68q2x\" (UID: \"bd5cf2da-6510-434d-81f9-579baecddaf3\") " pod="llm/echo-simulator-kserve-64b6f8f96c-68q2x"
Oct 31 17:34:38.080025 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:38.079999    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bd5cf2da-6510-434d-81f9-579baecddaf3-home\") pod \"echo-simulator-kserve-64b6f8f96c-68q2x\" (UID: \"bd5cf2da-6510-434d-81f9-579baecddaf3\") " pod="llm/echo-simulator-kserve-64b6f8f96c-68q2x"
Oct 31 17:34:38.080086 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:38.080031    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bd5cf2da-6510-434d-81f9-579baecddaf3-model-cache\") pod \"echo-simulator-kserve-64b6f8f96c-68q2x\" (UID: \"bd5cf2da-6510-434d-81f9-579baecddaf3\") " pod="llm/echo-simulator-kserve-64b6f8f96c-68q2x"
Oct 31 17:34:38.080123 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:38.080088    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bd5cf2da-6510-434d-81f9-579baecddaf3-tmp-dir\") pod \"echo-simulator-kserve-64b6f8f96c-68q2x\" (UID: \"bd5cf2da-6510-434d-81f9-579baecddaf3\") " pod="llm/echo-simulator-kserve-64b6f8f96c-68q2x"
Oct 31 17:34:38.081946 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:38.081929    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bd5cf2da-6510-434d-81f9-579baecddaf3-dshm\") pod \"echo-simulator-kserve-64b6f8f96c-68q2x\" (UID: \"bd5cf2da-6510-434d-81f9-579baecddaf3\") " pod="llm/echo-simulator-kserve-64b6f8f96c-68q2x"
Oct 31 17:34:38.082109 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:38.082091    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bd5cf2da-6510-434d-81f9-579baecddaf3-tls-certs\") pod \"echo-simulator-kserve-64b6f8f96c-68q2x\" (UID: \"bd5cf2da-6510-434d-81f9-579baecddaf3\") " pod="llm/echo-simulator-kserve-64b6f8f96c-68q2x"
Oct 31 17:34:38.100619 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:38.100592    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5hrs2\" (UniqueName: \"kubernetes.io/projected/bd5cf2da-6510-434d-81f9-579baecddaf3-kube-api-access-5hrs2\") pod \"echo-simulator-kserve-64b6f8f96c-68q2x\" (UID: \"bd5cf2da-6510-434d-81f9-579baecddaf3\") " pod="llm/echo-simulator-kserve-64b6f8f96c-68q2x"
Oct 31 17:34:38.190330 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:38.190244    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/echo-simulator-kserve-64b6f8f96c-68q2x"
Oct 31 17:34:38.488333 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:38.488306    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/echo-simulator-kserve-64b6f8f96c-68q2x"]
Oct 31 17:34:38.489708 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:34:38.489678    4276 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd5cf2da_6510_434d_81f9_579baecddaf3.slice/crio-15bc1ea3776184e5e7ad193ada499315a71284e0b62c1d0676d9ebf1cf145030 WatchSource:0}: Error finding container 15bc1ea3776184e5e7ad193ada499315a71284e0b62c1d0676d9ebf1cf145030: Status 404 returned error can't find the container with id 15bc1ea3776184e5e7ad193ada499315a71284e0b62c1d0676d9ebf1cf145030
Oct 31 17:34:39.057913 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:39.057880    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/echo-simulator-kserve-64b6f8f96c-68q2x" event={"ID":"bd5cf2da-6510-434d-81f9-579baecddaf3","Type":"ContainerStarted","Data":"15bc1ea3776184e5e7ad193ada499315a71284e0b62c1d0676d9ebf1cf145030"}
Oct 31 17:34:39.087740 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:39.087708    4276 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-mpnd7"
Oct 31 17:34:39.103936 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:39.103910    4276 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-mpnd7"
Oct 31 17:34:40.064025 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:40.063989    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/echo-simulator-kserve-64b6f8f96c-68q2x" event={"ID":"bd5cf2da-6510-434d-81f9-579baecddaf3","Type":"ContainerStarted","Data":"f78a760d33e1ed23f1d84fcad53cdf932695da416139e9bb06d84515872fa7db"}
Oct 31 17:34:40.075737 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:40.075709    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-mpnd7"
Oct 31 17:34:40.104983 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:40.104945    4276 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/echo-simulator-kserve-64b6f8f96c-68q2x" podStartSLOduration=1.631300776 podStartE2EDuration="3.104931998s" totalImagesPullingTime="1.473631222s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 17:34:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 17:34:40.104529272 +0000 UTC m=+442.902274200" watchObservedRunningTime="2026-10-31 17:34:40.104931998 +0000 UTC m=+442.902676954"
Oct 31 17:34:40.190838 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:40.190811    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/echo-simulator-kserve-64b6f8f96c-68q2x"
Oct 31 17:34:40.658711 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:40.658681    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:34:41.837321 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:41.837260    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:34:42.233819 ip-10-0-129-127 kubenswrapper[4276]: E1031 17:34:42.233742    4276 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tg5qk" containerName="extractor"
Oct 31 17:34:42.383989 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:42.383956    4276 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-ctkqw"
Oct 31 17:34:42.396485 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:42.396464    4276 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-ctkqw"
Oct 31 17:34:43.089413 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:43.089386    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-ctkqw"
Oct 31 17:34:48.190763 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:48.190730    4276 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/echo-simulator-kserve-64b6f8f96c-68q2x"
Oct 31 17:34:48.203296 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:48.203269    4276 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/echo-simulator-kserve-64b6f8f96c-68q2x"
Oct 31 17:34:49.113583 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:49.113553    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/echo-simulator-kserve-64b6f8f96c-68q2x"
Oct 31 17:34:57.204073 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:57.204033    4276 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-pfv5k"]
Oct 31 17:34:57.210049 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:57.210020    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-pfv5k"
Oct 31 17:34:57.227066 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:57.227042    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-9kmpm\""
Oct 31 17:34:57.228262 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:57.228239    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-pfv5k"]
Oct 31 17:34:57.233690 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:57.233670    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a0744b4f-5676-4385-b50c-5796ca2365a0-workload-socket\") pod \"maas-default-gateway-openshift-default-7d9897b54c-pfv5k\" (UID: \"a0744b4f-5676-4385-b50c-5796ca2365a0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-pfv5k"
Oct 31 17:34:57.233772 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:57.233696    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a0744b4f-5676-4385-b50c-5796ca2365a0-istio-token\") pod \"maas-default-gateway-openshift-default-7d9897b54c-pfv5k\" (UID: \"a0744b4f-5676-4385-b50c-5796ca2365a0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-pfv5k"
Oct 31 17:34:57.233809 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:57.233759    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a0744b4f-5676-4385-b50c-5796ca2365a0-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7d9897b54c-pfv5k\" (UID: \"a0744b4f-5676-4385-b50c-5796ca2365a0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-pfv5k"
Oct 31 17:34:57.233844 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:57.233832    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a0744b4f-5676-4385-b50c-5796ca2365a0-istio-data\") pod \"maas-default-gateway-openshift-default-7d9897b54c-pfv5k\" (UID: \"a0744b4f-5676-4385-b50c-5796ca2365a0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-pfv5k"
Oct 31 17:34:57.233884 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:57.233870    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a0744b4f-5676-4385-b50c-5796ca2365a0-workload-certs\") pod \"maas-default-gateway-openshift-default-7d9897b54c-pfv5k\" (UID: \"a0744b4f-5676-4385-b50c-5796ca2365a0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-pfv5k"
Oct 31 17:34:57.233932 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:57.233896    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a0744b4f-5676-4385-b50c-5796ca2365a0-credential-socket\") pod \"maas-default-gateway-openshift-default-7d9897b54c-pfv5k\" (UID: \"a0744b4f-5676-4385-b50c-5796ca2365a0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-pfv5k"
Oct 31 17:34:57.233990 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:57.233951    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a0744b4f-5676-4385-b50c-5796ca2365a0-istio-envoy\") pod \"maas-default-gateway-openshift-default-7d9897b54c-pfv5k\" (UID: \"a0744b4f-5676-4385-b50c-5796ca2365a0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-pfv5k"
Oct 31 17:34:57.234045 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:57.233998    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ngslw\" (UniqueName: \"kubernetes.io/projected/a0744b4f-5676-4385-b50c-5796ca2365a0-kube-api-access-ngslw\") pod \"maas-default-gateway-openshift-default-7d9897b54c-pfv5k\" (UID: \"a0744b4f-5676-4385-b50c-5796ca2365a0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-pfv5k"
Oct 31 17:34:57.234141 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:57.234126    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a0744b4f-5676-4385-b50c-5796ca2365a0-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7d9897b54c-pfv5k\" (UID: \"a0744b4f-5676-4385-b50c-5796ca2365a0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-pfv5k"
Oct 31 17:34:57.334534 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:57.334505    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a0744b4f-5676-4385-b50c-5796ca2365a0-workload-socket\") pod \"maas-default-gateway-openshift-default-7d9897b54c-pfv5k\" (UID: \"a0744b4f-5676-4385-b50c-5796ca2365a0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-pfv5k"
Oct 31 17:34:57.334534 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:57.334537    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a0744b4f-5676-4385-b50c-5796ca2365a0-istio-token\") pod \"maas-default-gateway-openshift-default-7d9897b54c-pfv5k\" (UID: \"a0744b4f-5676-4385-b50c-5796ca2365a0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-pfv5k"
Oct 31 17:34:57.334718 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:57.334554    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a0744b4f-5676-4385-b50c-5796ca2365a0-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7d9897b54c-pfv5k\" (UID: \"a0744b4f-5676-4385-b50c-5796ca2365a0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-pfv5k"
Oct 31 17:34:57.334718 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:57.334582    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a0744b4f-5676-4385-b50c-5796ca2365a0-istio-data\") pod \"maas-default-gateway-openshift-default-7d9897b54c-pfv5k\" (UID: \"a0744b4f-5676-4385-b50c-5796ca2365a0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-pfv5k"
Oct 31 17:34:57.334718 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:57.334612    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a0744b4f-5676-4385-b50c-5796ca2365a0-workload-certs\") pod \"maas-default-gateway-openshift-default-7d9897b54c-pfv5k\" (UID: \"a0744b4f-5676-4385-b50c-5796ca2365a0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-pfv5k"
Oct 31 17:34:57.334718 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:57.334632    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a0744b4f-5676-4385-b50c-5796ca2365a0-credential-socket\") pod \"maas-default-gateway-openshift-default-7d9897b54c-pfv5k\" (UID: \"a0744b4f-5676-4385-b50c-5796ca2365a0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-pfv5k"
Oct 31 17:34:57.334718 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:57.334649    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a0744b4f-5676-4385-b50c-5796ca2365a0-istio-envoy\") pod \"maas-default-gateway-openshift-default-7d9897b54c-pfv5k\" (UID: \"a0744b4f-5676-4385-b50c-5796ca2365a0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-pfv5k"
Oct 31 17:34:57.334718 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:57.334666    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ngslw\" (UniqueName: \"kubernetes.io/projected/a0744b4f-5676-4385-b50c-5796ca2365a0-kube-api-access-ngslw\") pod \"maas-default-gateway-openshift-default-7d9897b54c-pfv5k\" (UID: \"a0744b4f-5676-4385-b50c-5796ca2365a0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-pfv5k"
Oct 31 17:34:57.334718 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:57.334697    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a0744b4f-5676-4385-b50c-5796ca2365a0-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7d9897b54c-pfv5k\" (UID: \"a0744b4f-5676-4385-b50c-5796ca2365a0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-pfv5k"
Oct 31 17:34:57.335095 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:57.335040    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a0744b4f-5676-4385-b50c-5796ca2365a0-workload-certs\") pod \"maas-default-gateway-openshift-default-7d9897b54c-pfv5k\" (UID: \"a0744b4f-5676-4385-b50c-5796ca2365a0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-pfv5k"
Oct 31 17:34:57.335226 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:57.335131    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a0744b4f-5676-4385-b50c-5796ca2365a0-istio-data\") pod \"maas-default-gateway-openshift-default-7d9897b54c-pfv5k\" (UID: \"a0744b4f-5676-4385-b50c-5796ca2365a0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-pfv5k"
Oct 31 17:34:57.335226 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:57.335137    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a0744b4f-5676-4385-b50c-5796ca2365a0-credential-socket\") pod \"maas-default-gateway-openshift-default-7d9897b54c-pfv5k\" (UID: \"a0744b4f-5676-4385-b50c-5796ca2365a0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-pfv5k"
Oct 31 17:34:57.335323 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:57.335221    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a0744b4f-5676-4385-b50c-5796ca2365a0-workload-socket\") pod \"maas-default-gateway-openshift-default-7d9897b54c-pfv5k\" (UID: \"a0744b4f-5676-4385-b50c-5796ca2365a0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-pfv5k"
Oct 31 17:34:57.335361 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:57.335339    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a0744b4f-5676-4385-b50c-5796ca2365a0-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7d9897b54c-pfv5k\" (UID: \"a0744b4f-5676-4385-b50c-5796ca2365a0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-pfv5k"
Oct 31 17:34:57.337004 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:57.336981    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a0744b4f-5676-4385-b50c-5796ca2365a0-istio-envoy\") pod \"maas-default-gateway-openshift-default-7d9897b54c-pfv5k\" (UID: \"a0744b4f-5676-4385-b50c-5796ca2365a0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-pfv5k"
Oct 31 17:34:57.337335 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:57.337305    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a0744b4f-5676-4385-b50c-5796ca2365a0-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7d9897b54c-pfv5k\" (UID: \"a0744b4f-5676-4385-b50c-5796ca2365a0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-pfv5k"
Oct 31 17:34:57.354735 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:57.354716    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a0744b4f-5676-4385-b50c-5796ca2365a0-istio-token\") pod \"maas-default-gateway-openshift-default-7d9897b54c-pfv5k\" (UID: \"a0744b4f-5676-4385-b50c-5796ca2365a0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-pfv5k"
Oct 31 17:34:57.354891 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:57.354874    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ngslw\" (UniqueName: \"kubernetes.io/projected/a0744b4f-5676-4385-b50c-5796ca2365a0-kube-api-access-ngslw\") pod \"maas-default-gateway-openshift-default-7d9897b54c-pfv5k\" (UID: \"a0744b4f-5676-4385-b50c-5796ca2365a0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-pfv5k"
Oct 31 17:34:57.522982 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:57.522910    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-pfv5k"
Oct 31 17:34:57.845337 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:57.845313    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-pfv5k"]
Oct 31 17:34:57.847178 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:34:57.847149    4276 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0744b4f_5676_4385_b50c_5796ca2365a0.slice/crio-64bd990abceac66704a67882600c5512505d76afa74d1a7db7851ca230ee5e2a WatchSource:0}: Error finding container 64bd990abceac66704a67882600c5512505d76afa74d1a7db7851ca230ee5e2a: Status 404 returned error can't find the container with id 64bd990abceac66704a67882600c5512505d76afa74d1a7db7851ca230ee5e2a
Oct 31 17:34:58.140827 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:58.140741    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-pfv5k" event={"ID":"a0744b4f-5676-4385-b50c-5796ca2365a0","Type":"ContainerStarted","Data":"64bd990abceac66704a67882600c5512505d76afa74d1a7db7851ca230ee5e2a"}
Oct 31 17:34:59.119781 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:59.119749    4276 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/ai-gateway-controller-697b76b867-5dmn8"]
Oct 31 17:34:59.123282 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:59.123260    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-697b76b867-5dmn8"
Oct 31 17:34:59.140539 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:59.140439    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/ai-gateway-controller-697b76b867-5dmn8"]
Oct 31 17:34:59.141904 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:59.141873    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:34:59.150906 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:59.150815    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q58zp\" (UniqueName: \"kubernetes.io/projected/cf4dbf25-e450-4168-bbbc-388f806a1497-kube-api-access-q58zp\") pod \"ai-gateway-controller-697b76b867-5dmn8\" (UID: \"cf4dbf25-e450-4168-bbbc-388f806a1497\") " pod="opendatahub/ai-gateway-controller-697b76b867-5dmn8"
Oct 31 17:34:59.151174 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:59.151018    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cf4dbf25-e450-4168-bbbc-388f806a1497-metrics-certs\") pod \"ai-gateway-controller-697b76b867-5dmn8\" (UID: \"cf4dbf25-e450-4168-bbbc-388f806a1497\") " pod="opendatahub/ai-gateway-controller-697b76b867-5dmn8"
Oct 31 17:34:59.152731 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:59.152674    4276 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"ai-gateway-controller-parameters\""
Oct 31 17:34:59.152731 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:59.152688    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"ai-gateway-controller-metrics-tls\""
Oct 31 17:34:59.152883 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:59.152760    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"ai-gateway-controller-dockercfg-lwz2q\""
Oct 31 17:34:59.252612 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:59.252579    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q58zp\" (UniqueName: \"kubernetes.io/projected/cf4dbf25-e450-4168-bbbc-388f806a1497-kube-api-access-q58zp\") pod \"ai-gateway-controller-697b76b867-5dmn8\" (UID: \"cf4dbf25-e450-4168-bbbc-388f806a1497\") " pod="opendatahub/ai-gateway-controller-697b76b867-5dmn8"
Oct 31 17:34:59.252785 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:59.252671    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cf4dbf25-e450-4168-bbbc-388f806a1497-metrics-certs\") pod \"ai-gateway-controller-697b76b867-5dmn8\" (UID: \"cf4dbf25-e450-4168-bbbc-388f806a1497\") " pod="opendatahub/ai-gateway-controller-697b76b867-5dmn8"
Oct 31 17:34:59.255854 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:59.255831    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cf4dbf25-e450-4168-bbbc-388f806a1497-metrics-certs\") pod \"ai-gateway-controller-697b76b867-5dmn8\" (UID: \"cf4dbf25-e450-4168-bbbc-388f806a1497\") " pod="opendatahub/ai-gateway-controller-697b76b867-5dmn8"
Oct 31 17:34:59.263112 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:59.263086    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:34:59.274307 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:59.274288    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q58zp\" (UniqueName: \"kubernetes.io/projected/cf4dbf25-e450-4168-bbbc-388f806a1497-kube-api-access-q58zp\") pod \"ai-gateway-controller-697b76b867-5dmn8\" (UID: \"cf4dbf25-e450-4168-bbbc-388f806a1497\") " pod="opendatahub/ai-gateway-controller-697b76b867-5dmn8"
Oct 31 17:34:59.439155 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:59.439071    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-697b76b867-5dmn8"
Oct 31 17:34:59.746260 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:34:59.746236    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/ai-gateway-controller-697b76b867-5dmn8"]
Oct 31 17:34:59.748058 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:34:59.748019    4276 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf4dbf25_e450_4168_bbbc_388f806a1497.slice/crio-386b677c6c5bb36a67116bb6ada3036796bef4d8a774a62b3d5a29bac26c5687 WatchSource:0}: Error finding container 386b677c6c5bb36a67116bb6ada3036796bef4d8a774a62b3d5a29bac26c5687: Status 404 returned error can't find the container with id 386b677c6c5bb36a67116bb6ada3036796bef4d8a774a62b3d5a29bac26c5687
Oct 31 17:35:00.152264 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:00.152232    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-697b76b867-5dmn8" event={"ID":"cf4dbf25-e450-4168-bbbc-388f806a1497","Type":"ContainerStarted","Data":"386b677c6c5bb36a67116bb6ada3036796bef4d8a774a62b3d5a29bac26c5687"}
Oct 31 17:35:00.419969 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:00.419937    4276 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 17:35:00.420097 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:00.420011    4276 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 17:35:00.420097 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:00.420043    4276 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 17:35:01.179509 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:01.179475    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-pfv5k" event={"ID":"a0744b4f-5676-4385-b50c-5796ca2365a0","Type":"ContainerStarted","Data":"803185f204b3ed4af983404e91b40aabc0074bac2bbea13d7117dc6de846d535"}
Oct 31 17:35:01.238252 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:01.238205    4276 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-pfv5k" podStartSLOduration=1.6684655849999999 podStartE2EDuration="4.238191634s" totalImagesPullingTime="2.569726049s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 17:34:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 17:35:01.237425285 +0000 UTC m=+464.035170214" watchObservedRunningTime="2026-10-31 17:35:01.238191634 +0000 UTC m=+464.035936621"
Oct 31 17:35:01.523499 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:01.523396    4276 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-pfv5k"
Oct 31 17:35:01.525115 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:01.525088    4276 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7d9897b54c-pfv5k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused" start-of-body=
Oct 31 17:35:01.525270 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:01.525137    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-pfv5k" podUID="a0744b4f-5676-4385-b50c-5796ca2365a0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused"
Oct 31 17:35:02.090957 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:02.090928    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:35:02.185505 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:02.185468    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-697b76b867-5dmn8" event={"ID":"cf4dbf25-e450-4168-bbbc-388f806a1497","Type":"ContainerStarted","Data":"494e30088c0386c372dd37819362f05bf512a6765536de5165ffad5e6bbcb6f7"}
Oct 31 17:35:02.185936 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:02.185569    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/ai-gateway-controller-697b76b867-5dmn8"
Oct 31 17:35:02.225503 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:02.225437    4276 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/ai-gateway-controller-697b76b867-5dmn8" podStartSLOduration=1.75432422 podStartE2EDuration="3.225425539s" totalImagesPullingTime="1.471101319s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 17:34:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 17:35:02.22534627 +0000 UTC m=+465.023091198" watchObservedRunningTime="2026-10-31 17:35:02.225425539 +0000 UTC m=+465.023170464"
Oct 31 17:35:02.517528 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:02.517431    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:35:02.523555 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:02.523523    4276 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7d9897b54c-pfv5k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused" start-of-body=
Oct 31 17:35:02.523703 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:02.523588    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-pfv5k" podUID="a0744b4f-5676-4385-b50c-5796ca2365a0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused"
Oct 31 17:35:02.623860 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:02.623827    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:35:02.677361 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:02.677330    4276 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7b98bb678c-w9z9b"]
Oct 31 17:35:02.684984 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:02.684958    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7b98bb678c-w9z9b"
Oct 31 17:35:02.698137 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:02.698112    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7b98bb678c-w9z9b"]
Oct 31 17:35:02.700790 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:02.700771    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-4vqh2\""
Oct 31 17:35:02.700899 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:02.700830    4276 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Oct 31 17:35:02.778261 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:02.778176    4276 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-55fbbbf9dc-lbpsv"]
Oct 31 17:35:02.781645 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:02.781625    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-55fbbbf9dc-lbpsv"
Oct 31 17:35:02.791361 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:02.791324    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8d0cf02b-9581-4727-aac8-4047b97998c1-plugins-config-volume\") pod \"payload-pre-processing-7b98bb678c-w9z9b\" (UID: \"8d0cf02b-9581-4727-aac8-4047b97998c1\") " pod="openshift-ingress/payload-pre-processing-7b98bb678c-w9z9b"
Oct 31 17:35:02.791464 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:02.791384    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zqdg9\" (UniqueName: \"kubernetes.io/projected/d8705881-7dcb-40a0-b599-e14596d7697c-kube-api-access-zqdg9\") pod \"payload-processing-55fbbbf9dc-lbpsv\" (UID: \"d8705881-7dcb-40a0-b599-e14596d7697c\") " pod="openshift-ingress/payload-processing-55fbbbf9dc-lbpsv"
Oct 31 17:35:02.791505 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:02.791462    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2s6nd\" (UniqueName: \"kubernetes.io/projected/8d0cf02b-9581-4727-aac8-4047b97998c1-kube-api-access-2s6nd\") pod \"payload-pre-processing-7b98bb678c-w9z9b\" (UID: \"8d0cf02b-9581-4727-aac8-4047b97998c1\") " pod="openshift-ingress/payload-pre-processing-7b98bb678c-w9z9b"
Oct 31 17:35:02.791505 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:02.791498    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d8705881-7dcb-40a0-b599-e14596d7697c-plugins-config-volume\") pod \"payload-processing-55fbbbf9dc-lbpsv\" (UID: \"d8705881-7dcb-40a0-b599-e14596d7697c\") " pod="openshift-ingress/payload-processing-55fbbbf9dc-lbpsv"
Oct 31 17:35:02.797682 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:02.797662    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-55fbbbf9dc-lbpsv"]
Oct 31 17:35:02.892280 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:02.892242    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8d0cf02b-9581-4727-aac8-4047b97998c1-plugins-config-volume\") pod \"payload-pre-processing-7b98bb678c-w9z9b\" (UID: \"8d0cf02b-9581-4727-aac8-4047b97998c1\") " pod="openshift-ingress/payload-pre-processing-7b98bb678c-w9z9b"
Oct 31 17:35:02.892479 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:02.892290    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zqdg9\" (UniqueName: \"kubernetes.io/projected/d8705881-7dcb-40a0-b599-e14596d7697c-kube-api-access-zqdg9\") pod \"payload-processing-55fbbbf9dc-lbpsv\" (UID: \"d8705881-7dcb-40a0-b599-e14596d7697c\") " pod="openshift-ingress/payload-processing-55fbbbf9dc-lbpsv"
Oct 31 17:35:02.892479 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:02.892308    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2s6nd\" (UniqueName: \"kubernetes.io/projected/8d0cf02b-9581-4727-aac8-4047b97998c1-kube-api-access-2s6nd\") pod \"payload-pre-processing-7b98bb678c-w9z9b\" (UID: \"8d0cf02b-9581-4727-aac8-4047b97998c1\") " pod="openshift-ingress/payload-pre-processing-7b98bb678c-w9z9b"
Oct 31 17:35:02.892479 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:02.892331    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d8705881-7dcb-40a0-b599-e14596d7697c-plugins-config-volume\") pod \"payload-processing-55fbbbf9dc-lbpsv\" (UID: \"d8705881-7dcb-40a0-b599-e14596d7697c\") " pod="openshift-ingress/payload-processing-55fbbbf9dc-lbpsv"
Oct 31 17:35:02.893222 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:02.893197    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8d0cf02b-9581-4727-aac8-4047b97998c1-plugins-config-volume\") pod \"payload-pre-processing-7b98bb678c-w9z9b\" (UID: \"8d0cf02b-9581-4727-aac8-4047b97998c1\") " pod="openshift-ingress/payload-pre-processing-7b98bb678c-w9z9b"
Oct 31 17:35:02.893297 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:02.893197    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d8705881-7dcb-40a0-b599-e14596d7697c-plugins-config-volume\") pod \"payload-processing-55fbbbf9dc-lbpsv\" (UID: \"d8705881-7dcb-40a0-b599-e14596d7697c\") " pod="openshift-ingress/payload-processing-55fbbbf9dc-lbpsv"
Oct 31 17:35:02.913783 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:02.913755    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zqdg9\" (UniqueName: \"kubernetes.io/projected/d8705881-7dcb-40a0-b599-e14596d7697c-kube-api-access-zqdg9\") pod \"payload-processing-55fbbbf9dc-lbpsv\" (UID: \"d8705881-7dcb-40a0-b599-e14596d7697c\") " pod="openshift-ingress/payload-processing-55fbbbf9dc-lbpsv"
Oct 31 17:35:02.914167 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:02.914140    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2s6nd\" (UniqueName: \"kubernetes.io/projected/8d0cf02b-9581-4727-aac8-4047b97998c1-kube-api-access-2s6nd\") pod \"payload-pre-processing-7b98bb678c-w9z9b\" (UID: \"8d0cf02b-9581-4727-aac8-4047b97998c1\") " pod="openshift-ingress/payload-pre-processing-7b98bb678c-w9z9b"
Oct 31 17:35:02.998407 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:02.998374    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7b98bb678c-w9z9b"
Oct 31 17:35:03.094663 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:03.094636    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-55fbbbf9dc-lbpsv"
Oct 31 17:35:03.311171 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:03.311151    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7b98bb678c-w9z9b"]
Oct 31 17:35:03.313140 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:35:03.313116    4276 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d0cf02b_9581_4727_aac8_4047b97998c1.slice/crio-80476d720e388ccdf456070e476717b54b5df5cf4f1d050c979caa601292e5cc WatchSource:0}: Error finding container 80476d720e388ccdf456070e476717b54b5df5cf4f1d050c979caa601292e5cc: Status 404 returned error can't find the container with id 80476d720e388ccdf456070e476717b54b5df5cf4f1d050c979caa601292e5cc
Oct 31 17:35:03.395132 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:03.395070    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-55fbbbf9dc-lbpsv"]
Oct 31 17:35:03.397667 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:35:03.397635    4276 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd8705881_7dcb_40a0_b599_e14596d7697c.slice/crio-5fec71658fe965008147ae91bb8ac203543fb217a098bb8f11ae420b1dd5c165 WatchSource:0}: Error finding container 5fec71658fe965008147ae91bb8ac203543fb217a098bb8f11ae420b1dd5c165: Status 404 returned error can't find the container with id 5fec71658fe965008147ae91bb8ac203543fb217a098bb8f11ae420b1dd5c165
Oct 31 17:35:03.524466 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:03.524413    4276 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7d9897b54c-pfv5k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused" start-of-body=
Oct 31 17:35:03.524643 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:03.524499    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-pfv5k" podUID="a0744b4f-5676-4385-b50c-5796ca2365a0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused"
Oct 31 17:35:04.197618 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:04.197571    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-55fbbbf9dc-lbpsv" event={"ID":"d8705881-7dcb-40a0-b599-e14596d7697c","Type":"ContainerStarted","Data":"5fec71658fe965008147ae91bb8ac203543fb217a098bb8f11ae420b1dd5c165"}
Oct 31 17:35:04.199487 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:04.199421    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7b98bb678c-w9z9b" event={"ID":"8d0cf02b-9581-4727-aac8-4047b97998c1","Type":"ContainerStarted","Data":"80476d720e388ccdf456070e476717b54b5df5cf4f1d050c979caa601292e5cc"}
Oct 31 17:35:04.523969 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:04.523899    4276 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7d9897b54c-pfv5k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused" start-of-body=
Oct 31 17:35:04.523969 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:04.523946    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-pfv5k" podUID="a0744b4f-5676-4385-b50c-5796ca2365a0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused"
Oct 31 17:35:05.523878 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:05.523844    4276 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7d9897b54c-pfv5k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused" start-of-body=
Oct 31 17:35:05.524085 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:05.523893    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-pfv5k" podUID="a0744b4f-5676-4385-b50c-5796ca2365a0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused"
Oct 31 17:35:06.523844 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:06.523808    4276 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7d9897b54c-pfv5k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused" start-of-body=
Oct 31 17:35:06.524013 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:06.523871    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-pfv5k" podUID="a0744b4f-5676-4385-b50c-5796ca2365a0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused"
Oct 31 17:35:07.220271 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:07.220237    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-55fbbbf9dc-lbpsv" event={"ID":"d8705881-7dcb-40a0-b599-e14596d7697c","Type":"ContainerStarted","Data":"c5649190065e3faa69d46bf5d534cfba4fe725ae2f672f9b8749e954ac17e822"}
Oct 31 17:35:07.220731 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:07.220335    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-55fbbbf9dc-lbpsv"
Oct 31 17:35:07.221821 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:07.221798    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7b98bb678c-w9z9b" event={"ID":"8d0cf02b-9581-4727-aac8-4047b97998c1","Type":"ContainerStarted","Data":"ccbc87d04c7b5535e2bd3f32e16060cff24c47f4c1809633807bfa4ddb1c52bb"}
Oct 31 17:35:07.221943 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:07.221910    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7b98bb678c-w9z9b"
Oct 31 17:35:07.261122 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:07.261082    4276 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-55fbbbf9dc-lbpsv" podStartSLOduration=2.282863897 podStartE2EDuration="5.261070974s" totalImagesPullingTime="2.978207077s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 17:35:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 17:35:07.259829789 +0000 UTC m=+470.057574717" watchObservedRunningTime="2026-10-31 17:35:07.261070974 +0000 UTC m=+470.058815901"
Oct 31 17:35:07.300394 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:07.300334    4276 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7b98bb678c-w9z9b" podStartSLOduration=2.2081602399999998 podStartE2EDuration="5.300317905s" totalImagesPullingTime="3.092157665s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 17:35:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 17:35:07.298868449 +0000 UTC m=+470.096613390" watchObservedRunningTime="2026-10-31 17:35:07.300317905 +0000 UTC m=+470.098062836"
Oct 31 17:35:07.523341 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:07.523262    4276 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7d9897b54c-pfv5k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused" start-of-body=
Oct 31 17:35:07.523341 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:07.523315    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-pfv5k" podUID="a0744b4f-5676-4385-b50c-5796ca2365a0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused"
Oct 31 17:35:09.525086 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:09.525043    4276 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7d9897b54c-pfv5k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.30:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 17:35:09.525608 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:09.525111    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-pfv5k" podUID="a0744b4f-5676-4385-b50c-5796ca2365a0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.30:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 17:35:10.566580 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:10.566541    4276 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7d9897b54c-pfv5k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.30:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 17:35:10.567081 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:10.566589    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-pfv5k" podUID="a0744b4f-5676-4385-b50c-5796ca2365a0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.30:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 17:35:10.911339 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:10.911253    4276 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-pfv5k"
Oct 31 17:35:10.911722 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:10.911698    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-pfv5k"
Oct 31 17:35:10.912658 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:10.912638    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-pfv5k"
Oct 31 17:35:12.224180 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:12.224146    4276 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-b8gmm"]
Oct 31 17:35:12.230256 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:12.230234    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-b8gmm"
Oct 31 17:35:12.236228 ip-10-0-129-127 kubenswrapper[4276]: E1031 17:35:12.236197    4276 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tg5qk" containerName="extractor"
Oct 31 17:35:12.263525 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:12.263499    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-b8gmm"]
Oct 31 17:35:12.279319 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:12.279294    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b42k6\" (UniqueName: \"kubernetes.io/projected/5530ee8f-be36-43d3-a268-051794343d8b-kube-api-access-b42k6\") pod \"maas-default-gateway-openshift-default-7d9897b54c-b8gmm\" (UID: \"5530ee8f-be36-43d3-a268-051794343d8b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-b8gmm"
Oct 31 17:35:12.279489 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:12.279354    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5530ee8f-be36-43d3-a268-051794343d8b-istio-token\") pod \"maas-default-gateway-openshift-default-7d9897b54c-b8gmm\" (UID: \"5530ee8f-be36-43d3-a268-051794343d8b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-b8gmm"
Oct 31 17:35:12.279489 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:12.279469    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5530ee8f-be36-43d3-a268-051794343d8b-istio-envoy\") pod \"maas-default-gateway-openshift-default-7d9897b54c-b8gmm\" (UID: \"5530ee8f-be36-43d3-a268-051794343d8b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-b8gmm"
Oct 31 17:35:12.284873 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:12.279962    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5530ee8f-be36-43d3-a268-051794343d8b-workload-certs\") pod \"maas-default-gateway-openshift-default-7d9897b54c-b8gmm\" (UID: \"5530ee8f-be36-43d3-a268-051794343d8b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-b8gmm"
Oct 31 17:35:12.284873 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:12.280102    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5530ee8f-be36-43d3-a268-051794343d8b-credential-socket\") pod \"maas-default-gateway-openshift-default-7d9897b54c-b8gmm\" (UID: \"5530ee8f-be36-43d3-a268-051794343d8b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-b8gmm"
Oct 31 17:35:12.284873 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:12.280147    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5530ee8f-be36-43d3-a268-051794343d8b-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7d9897b54c-b8gmm\" (UID: \"5530ee8f-be36-43d3-a268-051794343d8b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-b8gmm"
Oct 31 17:35:12.284873 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:12.280184    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5530ee8f-be36-43d3-a268-051794343d8b-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7d9897b54c-b8gmm\" (UID: \"5530ee8f-be36-43d3-a268-051794343d8b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-b8gmm"
Oct 31 17:35:12.284873 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:12.280245    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5530ee8f-be36-43d3-a268-051794343d8b-workload-socket\") pod \"maas-default-gateway-openshift-default-7d9897b54c-b8gmm\" (UID: \"5530ee8f-be36-43d3-a268-051794343d8b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-b8gmm"
Oct 31 17:35:12.284873 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:12.280292    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5530ee8f-be36-43d3-a268-051794343d8b-istio-data\") pod \"maas-default-gateway-openshift-default-7d9897b54c-b8gmm\" (UID: \"5530ee8f-be36-43d3-a268-051794343d8b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-b8gmm"
Oct 31 17:35:12.380959 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:12.380927    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5530ee8f-be36-43d3-a268-051794343d8b-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7d9897b54c-b8gmm\" (UID: \"5530ee8f-be36-43d3-a268-051794343d8b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-b8gmm"
Oct 31 17:35:12.380959 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:12.380962    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5530ee8f-be36-43d3-a268-051794343d8b-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7d9897b54c-b8gmm\" (UID: \"5530ee8f-be36-43d3-a268-051794343d8b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-b8gmm"
Oct 31 17:35:12.381145 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:12.380984    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5530ee8f-be36-43d3-a268-051794343d8b-workload-socket\") pod \"maas-default-gateway-openshift-default-7d9897b54c-b8gmm\" (UID: \"5530ee8f-be36-43d3-a268-051794343d8b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-b8gmm"
Oct 31 17:35:12.381145 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:12.381016    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5530ee8f-be36-43d3-a268-051794343d8b-istio-data\") pod \"maas-default-gateway-openshift-default-7d9897b54c-b8gmm\" (UID: \"5530ee8f-be36-43d3-a268-051794343d8b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-b8gmm"
Oct 31 17:35:12.381145 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:12.381040    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b42k6\" (UniqueName: \"kubernetes.io/projected/5530ee8f-be36-43d3-a268-051794343d8b-kube-api-access-b42k6\") pod \"maas-default-gateway-openshift-default-7d9897b54c-b8gmm\" (UID: \"5530ee8f-be36-43d3-a268-051794343d8b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-b8gmm"
Oct 31 17:35:12.381145 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:12.381069    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5530ee8f-be36-43d3-a268-051794343d8b-istio-token\") pod \"maas-default-gateway-openshift-default-7d9897b54c-b8gmm\" (UID: \"5530ee8f-be36-43d3-a268-051794343d8b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-b8gmm"
Oct 31 17:35:12.381145 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:12.381114    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5530ee8f-be36-43d3-a268-051794343d8b-istio-envoy\") pod \"maas-default-gateway-openshift-default-7d9897b54c-b8gmm\" (UID: \"5530ee8f-be36-43d3-a268-051794343d8b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-b8gmm"
Oct 31 17:35:12.381430 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:12.381402    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5530ee8f-be36-43d3-a268-051794343d8b-workload-certs\") pod \"maas-default-gateway-openshift-default-7d9897b54c-b8gmm\" (UID: \"5530ee8f-be36-43d3-a268-051794343d8b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-b8gmm"
Oct 31 17:35:12.381532 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:12.381474    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5530ee8f-be36-43d3-a268-051794343d8b-workload-socket\") pod \"maas-default-gateway-openshift-default-7d9897b54c-b8gmm\" (UID: \"5530ee8f-be36-43d3-a268-051794343d8b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-b8gmm"
Oct 31 17:35:12.381532 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:12.381493    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5530ee8f-be36-43d3-a268-051794343d8b-credential-socket\") pod \"maas-default-gateway-openshift-default-7d9897b54c-b8gmm\" (UID: \"5530ee8f-be36-43d3-a268-051794343d8b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-b8gmm"
Oct 31 17:35:12.381621 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:12.381546    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5530ee8f-be36-43d3-a268-051794343d8b-istio-data\") pod \"maas-default-gateway-openshift-default-7d9897b54c-b8gmm\" (UID: \"5530ee8f-be36-43d3-a268-051794343d8b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-b8gmm"
Oct 31 17:35:12.381870 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:12.381851    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5530ee8f-be36-43d3-a268-051794343d8b-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7d9897b54c-b8gmm\" (UID: \"5530ee8f-be36-43d3-a268-051794343d8b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-b8gmm"
Oct 31 17:35:12.382019 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:12.381998    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5530ee8f-be36-43d3-a268-051794343d8b-credential-socket\") pod \"maas-default-gateway-openshift-default-7d9897b54c-b8gmm\" (UID: \"5530ee8f-be36-43d3-a268-051794343d8b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-b8gmm"
Oct 31 17:35:12.382054 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:12.382029    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5530ee8f-be36-43d3-a268-051794343d8b-workload-certs\") pod \"maas-default-gateway-openshift-default-7d9897b54c-b8gmm\" (UID: \"5530ee8f-be36-43d3-a268-051794343d8b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-b8gmm"
Oct 31 17:35:12.383301 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:12.383279    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5530ee8f-be36-43d3-a268-051794343d8b-istio-envoy\") pod \"maas-default-gateway-openshift-default-7d9897b54c-b8gmm\" (UID: \"5530ee8f-be36-43d3-a268-051794343d8b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-b8gmm"
Oct 31 17:35:12.383456 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:12.383422    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5530ee8f-be36-43d3-a268-051794343d8b-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7d9897b54c-b8gmm\" (UID: \"5530ee8f-be36-43d3-a268-051794343d8b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-b8gmm"
Oct 31 17:35:12.401103 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:12.401077    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5530ee8f-be36-43d3-a268-051794343d8b-istio-token\") pod \"maas-default-gateway-openshift-default-7d9897b54c-b8gmm\" (UID: \"5530ee8f-be36-43d3-a268-051794343d8b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-b8gmm"
Oct 31 17:35:12.401244 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:12.401226    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b42k6\" (UniqueName: \"kubernetes.io/projected/5530ee8f-be36-43d3-a268-051794343d8b-kube-api-access-b42k6\") pod \"maas-default-gateway-openshift-default-7d9897b54c-b8gmm\" (UID: \"5530ee8f-be36-43d3-a268-051794343d8b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-b8gmm"
Oct 31 17:35:12.544193 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:12.544114    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-b8gmm"
Oct 31 17:35:12.863804 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:12.863783    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-b8gmm"]
Oct 31 17:35:12.865970 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:35:12.865941    4276 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5530ee8f_be36_43d3_a268_051794343d8b.slice/crio-cd5162cd6d1798f922fc78800ea51c0dd3cc87c4b24d7ca6da83fc0b84ef52c8 WatchSource:0}: Error finding container cd5162cd6d1798f922fc78800ea51c0dd3cc87c4b24d7ca6da83fc0b84ef52c8: Status 404 returned error can't find the container with id cd5162cd6d1798f922fc78800ea51c0dd3cc87c4b24d7ca6da83fc0b84ef52c8
Oct 31 17:35:12.869001 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:12.868967    4276 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 17:35:12.869091 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:12.869039    4276 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 17:35:12.869091 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:12.869071    4276 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 17:35:13.191487 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:13.191385    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/ai-gateway-controller-697b76b867-5dmn8"
Oct 31 17:35:13.250587 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:13.250547    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-b8gmm" event={"ID":"5530ee8f-be36-43d3-a268-051794343d8b","Type":"ContainerStarted","Data":"87418244d9619814f817b8d65acf7fc570555a66465e27b408fcc967bd9ad65e"}
Oct 31 17:35:13.251056 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:13.250608    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-b8gmm" event={"ID":"5530ee8f-be36-43d3-a268-051794343d8b","Type":"ContainerStarted","Data":"cd5162cd6d1798f922fc78800ea51c0dd3cc87c4b24d7ca6da83fc0b84ef52c8"}
Oct 31 17:35:13.301613 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:13.301548    4276 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-b8gmm" podStartSLOduration=1.301514369 podStartE2EDuration="1.301514369s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 17:35:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 17:35:13.301105564 +0000 UTC m=+476.098850491" watchObservedRunningTime="2026-10-31 17:35:13.301514369 +0000 UTC m=+476.099259300"
Oct 31 17:35:13.459028 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:13.458931    4276 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/ai-gateway-controller-848fd4df45-tbvzh"]
Oct 31 17:35:13.469300 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:13.469266    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-848fd4df45-tbvzh"
Oct 31 17:35:13.477561 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:13.477528    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/ai-gateway-controller-848fd4df45-tbvzh"]
Oct 31 17:35:13.544306 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:13.544274    4276 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-b8gmm"
Oct 31 17:35:13.545518 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:13.545495    4276 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7d9897b54c-b8gmm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused" start-of-body=
Oct 31 17:35:13.545618 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:13.545547    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-b8gmm" podUID="5530ee8f-be36-43d3-a268-051794343d8b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused"
Oct 31 17:35:13.591558 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:13.591510    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9474cc98-e6c9-4c56-ab07-664b1c552440-metrics-certs\") pod \"ai-gateway-controller-848fd4df45-tbvzh\" (UID: \"9474cc98-e6c9-4c56-ab07-664b1c552440\") " pod="opendatahub/ai-gateway-controller-848fd4df45-tbvzh"
Oct 31 17:35:13.591725 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:13.591574    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jbmwp\" (UniqueName: \"kubernetes.io/projected/9474cc98-e6c9-4c56-ab07-664b1c552440-kube-api-access-jbmwp\") pod \"ai-gateway-controller-848fd4df45-tbvzh\" (UID: \"9474cc98-e6c9-4c56-ab07-664b1c552440\") " pod="opendatahub/ai-gateway-controller-848fd4df45-tbvzh"
Oct 31 17:35:13.692960 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:13.692916    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9474cc98-e6c9-4c56-ab07-664b1c552440-metrics-certs\") pod \"ai-gateway-controller-848fd4df45-tbvzh\" (UID: \"9474cc98-e6c9-4c56-ab07-664b1c552440\") " pod="opendatahub/ai-gateway-controller-848fd4df45-tbvzh"
Oct 31 17:35:13.693161 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:13.692986    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jbmwp\" (UniqueName: \"kubernetes.io/projected/9474cc98-e6c9-4c56-ab07-664b1c552440-kube-api-access-jbmwp\") pod \"ai-gateway-controller-848fd4df45-tbvzh\" (UID: \"9474cc98-e6c9-4c56-ab07-664b1c552440\") " pod="opendatahub/ai-gateway-controller-848fd4df45-tbvzh"
Oct 31 17:35:13.695427 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:13.695398    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9474cc98-e6c9-4c56-ab07-664b1c552440-metrics-certs\") pod \"ai-gateway-controller-848fd4df45-tbvzh\" (UID: \"9474cc98-e6c9-4c56-ab07-664b1c552440\") " pod="opendatahub/ai-gateway-controller-848fd4df45-tbvzh"
Oct 31 17:35:13.713922 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:13.713864    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jbmwp\" (UniqueName: \"kubernetes.io/projected/9474cc98-e6c9-4c56-ab07-664b1c552440-kube-api-access-jbmwp\") pod \"ai-gateway-controller-848fd4df45-tbvzh\" (UID: \"9474cc98-e6c9-4c56-ab07-664b1c552440\") " pod="opendatahub/ai-gateway-controller-848fd4df45-tbvzh"
Oct 31 17:35:13.795321 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:13.795285    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-848fd4df45-tbvzh"
Oct 31 17:35:14.090017 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:14.089994    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/ai-gateway-controller-848fd4df45-tbvzh"]
Oct 31 17:35:14.092410 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:35:14.092377    4276 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9474cc98_e6c9_4c56_ab07_664b1c552440.slice/crio-19ead960fbc6a4ee7ca50cb66dda12ef0e087c4f7df9308434007f62d235573a WatchSource:0}: Error finding container 19ead960fbc6a4ee7ca50cb66dda12ef0e087c4f7df9308434007f62d235573a: Status 404 returned error can't find the container with id 19ead960fbc6a4ee7ca50cb66dda12ef0e087c4f7df9308434007f62d235573a
Oct 31 17:35:14.275800 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:14.275731    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-848fd4df45-tbvzh" event={"ID":"9474cc98-e6c9-4c56-ab07-664b1c552440","Type":"ContainerStarted","Data":"19ead960fbc6a4ee7ca50cb66dda12ef0e087c4f7df9308434007f62d235573a"}
Oct 31 17:35:14.545234 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:14.545208    4276 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7d9897b54c-b8gmm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused" start-of-body=
Oct 31 17:35:14.545312 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:14.545265    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-b8gmm" podUID="5530ee8f-be36-43d3-a268-051794343d8b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused"
Oct 31 17:35:15.282844 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:15.282804    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-848fd4df45-tbvzh" event={"ID":"9474cc98-e6c9-4c56-ab07-664b1c552440","Type":"ContainerStarted","Data":"bf924af07289be6030c68d2b528698a0a42cc463bfd7fe1dcfaee8a6b280a579"}
Oct 31 17:35:15.283274 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:15.283124    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/ai-gateway-controller-848fd4df45-tbvzh"
Oct 31 17:35:15.337743 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:15.337690    4276 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/ai-gateway-controller-848fd4df45-tbvzh" podStartSLOduration=1.945071021 podStartE2EDuration="2.337676274s" totalImagesPullingTime="392.605253ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 17:35:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 17:35:15.323999926 +0000 UTC m=+478.121744853" watchObservedRunningTime="2026-10-31 17:35:15.337676274 +0000 UTC m=+478.135421202"
Oct 31 17:35:15.545332 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:15.545243    4276 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7d9897b54c-b8gmm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused" start-of-body=
Oct 31 17:35:15.545332 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:15.545300    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-b8gmm" podUID="5530ee8f-be36-43d3-a268-051794343d8b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused"
Oct 31 17:35:16.545061 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:16.545025    4276 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7d9897b54c-b8gmm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused" start-of-body=
Oct 31 17:35:16.545477 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:16.545076    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-b8gmm" podUID="5530ee8f-be36-43d3-a268-051794343d8b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused"
Oct 31 17:35:17.544500 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:17.544471    4276 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7d9897b54c-b8gmm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused" start-of-body=
Oct 31 17:35:17.544655 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:17.544525    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-b8gmm" podUID="5530ee8f-be36-43d3-a268-051794343d8b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused"
Oct 31 17:35:18.228558 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:18.228529    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7b98bb678c-w9z9b"
Oct 31 17:35:18.228958 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:18.228593    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-55fbbbf9dc-lbpsv"
Oct 31 17:35:18.545176 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:18.545099    4276 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7d9897b54c-b8gmm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused" start-of-body=
Oct 31 17:35:18.545176 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:18.545150    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-b8gmm" podUID="5530ee8f-be36-43d3-a268-051794343d8b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused"
Oct 31 17:35:19.545116 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:19.545072    4276 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7d9897b54c-b8gmm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused" start-of-body=
Oct 31 17:35:19.545592 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:19.545138    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-b8gmm" podUID="5530ee8f-be36-43d3-a268-051794343d8b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused"
Oct 31 17:35:21.042134 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:21.042100    4276 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-b8gmm"
Oct 31 17:35:21.042686 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:21.042663    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-b8gmm"
Oct 31 17:35:21.043438 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:21.043417    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-b8gmm"
Oct 31 17:35:26.288836 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:26.288806    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/ai-gateway-controller-848fd4df45-tbvzh"
Oct 31 17:35:26.355559 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:26.355523    4276 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/ai-gateway-controller-697b76b867-5dmn8"]
Oct 31 17:35:26.355776 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:26.355751    4276 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/ai-gateway-controller-697b76b867-5dmn8" podUID="cf4dbf25-e450-4168-bbbc-388f806a1497" containerName="manager" containerID="cri-o://494e30088c0386c372dd37819362f05bf512a6765536de5165ffad5e6bbcb6f7" gracePeriod=30
Oct 31 17:35:26.598114 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:26.598086    4276 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-697b76b867-5dmn8"
Oct 31 17:35:26.713650 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:26.713619    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cf4dbf25-e450-4168-bbbc-388f806a1497-kube-api-access-q58zp\" (UniqueName: \"kubernetes.io/projected/cf4dbf25-e450-4168-bbbc-388f806a1497-kube-api-access-q58zp\") pod \"cf4dbf25-e450-4168-bbbc-388f806a1497\" (UID: \"cf4dbf25-e450-4168-bbbc-388f806a1497\") "
Oct 31 17:35:26.713821 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:26.713713    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/cf4dbf25-e450-4168-bbbc-388f806a1497-metrics-certs\" (UniqueName: \"kubernetes.io/secret/cf4dbf25-e450-4168-bbbc-388f806a1497-metrics-certs\") pod \"cf4dbf25-e450-4168-bbbc-388f806a1497\" (UID: \"cf4dbf25-e450-4168-bbbc-388f806a1497\") "
Oct 31 17:35:26.715792 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:26.715764    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cf4dbf25-e450-4168-bbbc-388f806a1497-kube-api-access-q58zp" pod "cf4dbf25-e450-4168-bbbc-388f806a1497" (UID: "cf4dbf25-e450-4168-bbbc-388f806a1497"). InnerVolumeSpecName "kube-api-access-q58zp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 17:35:26.715896 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:26.715854    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cf4dbf25-e450-4168-bbbc-388f806a1497-metrics-certs" pod "cf4dbf25-e450-4168-bbbc-388f806a1497" (UID: "cf4dbf25-e450-4168-bbbc-388f806a1497"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 17:35:26.814956 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:26.814922    4276 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q58zp\" (UniqueName: \"kubernetes.io/projected/cf4dbf25-e450-4168-bbbc-388f806a1497-kube-api-access-q58zp\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:35:26.814956 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:26.814952    4276 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cf4dbf25-e450-4168-bbbc-388f806a1497-metrics-certs\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:35:27.335278 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:27.335243    4276 generic.go:356] "Generic (PLEG): container finished" podID="cf4dbf25-e450-4168-bbbc-388f806a1497" containerID="494e30088c0386c372dd37819362f05bf512a6765536de5165ffad5e6bbcb6f7" exitCode=0
Oct 31 17:35:27.335685 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:27.335285    4276 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-697b76b867-5dmn8"
Oct 31 17:35:27.335685 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:27.335332    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-697b76b867-5dmn8" event={"ID":"cf4dbf25-e450-4168-bbbc-388f806a1497","Type":"ContainerDied","Data":"494e30088c0386c372dd37819362f05bf512a6765536de5165ffad5e6bbcb6f7"}
Oct 31 17:35:27.335685 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:27.335384    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-697b76b867-5dmn8" event={"ID":"cf4dbf25-e450-4168-bbbc-388f806a1497","Type":"ContainerDied","Data":"386b677c6c5bb36a67116bb6ada3036796bef4d8a774a62b3d5a29bac26c5687"}
Oct 31 17:35:27.335685 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:27.335410    4276 scope.go:122] "RemoveContainer" containerID="494e30088c0386c372dd37819362f05bf512a6765536de5165ffad5e6bbcb6f7"
Oct 31 17:35:27.343860 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:27.343833    4276 scope.go:122] "RemoveContainer" containerID="494e30088c0386c372dd37819362f05bf512a6765536de5165ffad5e6bbcb6f7"
Oct 31 17:35:27.344081 ip-10-0-129-127 kubenswrapper[4276]: E1031 17:35:27.344062    4276 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"494e30088c0386c372dd37819362f05bf512a6765536de5165ffad5e6bbcb6f7\": container with ID starting with 494e30088c0386c372dd37819362f05bf512a6765536de5165ffad5e6bbcb6f7 not found: ID does not exist" containerID="494e30088c0386c372dd37819362f05bf512a6765536de5165ffad5e6bbcb6f7"
Oct 31 17:35:27.344126 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:27.344090    4276 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"494e30088c0386c372dd37819362f05bf512a6765536de5165ffad5e6bbcb6f7"} err="failed to get container status \"494e30088c0386c372dd37819362f05bf512a6765536de5165ffad5e6bbcb6f7\": rpc error: code = NotFound desc = could not find container \"494e30088c0386c372dd37819362f05bf512a6765536de5165ffad5e6bbcb6f7\": container with ID starting with 494e30088c0386c372dd37819362f05bf512a6765536de5165ffad5e6bbcb6f7 not found: ID does not exist"
Oct 31 17:35:27.388688 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:27.388662    4276 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/ai-gateway-controller-697b76b867-5dmn8"]
Oct 31 17:35:27.402416 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:27.402388    4276 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/ai-gateway-controller-697b76b867-5dmn8"]
Oct 31 17:35:27.929351 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:27.929323    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:35:29.365850 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:29.365816    4276 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="cf4dbf25-e450-4168-bbbc-388f806a1497" path="/var/lib/kubelet/pods/cf4dbf25-e450-4168-bbbc-388f806a1497/volumes"
Oct 31 17:35:38.383140 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:38.383106    4276 generic.go:356] "Generic (PLEG): container finished" podID="de946e91-8b1c-42d2-b859-e7d93a6fb7cd" containerID="e5777351264086248e32e41af325e958cf7e9d12168f115b214d2821bb6d68d3" exitCode=1
Oct 31 17:35:38.383557 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:38.383164    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-mb26r" event={"ID":"de946e91-8b1c-42d2-b859-e7d93a6fb7cd","Type":"ContainerDied","Data":"e5777351264086248e32e41af325e958cf7e9d12168f115b214d2821bb6d68d3"}
Oct 31 17:35:38.383557 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:38.383495    4276 scope.go:122] "RemoveContainer" containerID="e5777351264086248e32e41af325e958cf7e9d12168f115b214d2821bb6d68d3"
Oct 31 17:35:38.409840 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:38.409817    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:35:39.388757 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:39.388722    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-mb26r" event={"ID":"de946e91-8b1c-42d2-b859-e7d93a6fb7cd","Type":"ContainerStarted","Data":"416597c8f9eeda8af394d8541577aba215588912aaeec3d83f8046451bdb74ba"}
Oct 31 17:35:39.389148 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:39.388915    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-mb26r"
Oct 31 17:35:39.974718 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:39.974680    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:35:40.100801 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:40.100770    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:35:42.234186 ip-10-0-129-127 kubenswrapper[4276]: E1031 17:35:42.234152    4276 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tg5qk" containerName="extractor"
Oct 31 17:35:48.743527 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:48.743498    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:35:54.812938 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:54.812909    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:35:54.915294 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:35:54.915269    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:36:00.394380 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:00.394349    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-mb26r"
Oct 31 17:36:02.207100 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:02.207058    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:36:08.600437 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:08.600402    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:36:08.720425 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:08.720386    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:36:12.233350 ip-10-0-129-127 kubenswrapper[4276]: E1031 17:36:12.233309    4276 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tg5qk" containerName="extractor"
Oct 31 17:36:30.274752 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:30.274698    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:36:30.394040 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:30.393989    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:36:30.681019 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:30.680982    4276 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-c5dc6"]
Oct 31 17:36:30.681602 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:30.681582    4276 cpu_manager.go:414] "containerMap: removing container" podUID="cf4dbf25-e450-4168-bbbc-388f806a1497" containerName="manager"
Oct 31 17:36:30.681694 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:30.681605    4276 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cf4dbf25-e450-4168-bbbc-388f806a1497" containerName="manager"
Oct 31 17:36:30.681747 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:30.681700    4276 memory_manager.go:365] "RemoveStaleState removing state" podUID="cf4dbf25-e450-4168-bbbc-388f806a1497" containerName="manager"
Oct 31 17:36:30.686480 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:30.686456    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-c5dc6"
Oct 31 17:36:30.718727 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:30.718694    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-c5dc6"]
Oct 31 17:36:30.719305 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:30.719287    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w3-cf0a86-openshift-default-dockercfg-m2mbq\""
Oct 31 17:36:30.753896 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:30.753841    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:36:30.762994 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:30.762957    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/22af22dd-1130-409b-9a17-05229dc0de11-workload-socket\") pod \"e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-c5dc6\" (UID: \"22af22dd-1130-409b-9a17-05229dc0de11\") " pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-c5dc6"
Oct 31 17:36:30.763129 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:30.763006    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-57gbg\" (UniqueName: \"kubernetes.io/projected/22af22dd-1130-409b-9a17-05229dc0de11-kube-api-access-57gbg\") pod \"e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-c5dc6\" (UID: \"22af22dd-1130-409b-9a17-05229dc0de11\") " pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-c5dc6"
Oct 31 17:36:30.763189 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:30.763140    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/22af22dd-1130-409b-9a17-05229dc0de11-istio-data\") pod \"e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-c5dc6\" (UID: \"22af22dd-1130-409b-9a17-05229dc0de11\") " pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-c5dc6"
Oct 31 17:36:30.763424 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:30.763216    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/22af22dd-1130-409b-9a17-05229dc0de11-workload-certs\") pod \"e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-c5dc6\" (UID: \"22af22dd-1130-409b-9a17-05229dc0de11\") " pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-c5dc6"
Oct 31 17:36:30.763424 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:30.763310    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/22af22dd-1130-409b-9a17-05229dc0de11-istio-podinfo\") pod \"e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-c5dc6\" (UID: \"22af22dd-1130-409b-9a17-05229dc0de11\") " pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-c5dc6"
Oct 31 17:36:30.763424 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:30.763348    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/22af22dd-1130-409b-9a17-05229dc0de11-istio-envoy\") pod \"e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-c5dc6\" (UID: \"22af22dd-1130-409b-9a17-05229dc0de11\") " pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-c5dc6"
Oct 31 17:36:30.763424 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:30.763376    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/22af22dd-1130-409b-9a17-05229dc0de11-istio-token\") pod \"e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-c5dc6\" (UID: \"22af22dd-1130-409b-9a17-05229dc0de11\") " pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-c5dc6"
Oct 31 17:36:30.763424 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:30.763403    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/22af22dd-1130-409b-9a17-05229dc0de11-credential-socket\") pod \"e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-c5dc6\" (UID: \"22af22dd-1130-409b-9a17-05229dc0de11\") " pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-c5dc6"
Oct 31 17:36:30.763786 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:30.763428    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/22af22dd-1130-409b-9a17-05229dc0de11-istiod-ca-cert\") pod \"e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-c5dc6\" (UID: \"22af22dd-1130-409b-9a17-05229dc0de11\") " pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-c5dc6"
Oct 31 17:36:30.864521 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:30.864488    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/22af22dd-1130-409b-9a17-05229dc0de11-workload-socket\") pod \"e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-c5dc6\" (UID: \"22af22dd-1130-409b-9a17-05229dc0de11\") " pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-c5dc6"
Oct 31 17:36:30.864710 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:30.864530    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-57gbg\" (UniqueName: \"kubernetes.io/projected/22af22dd-1130-409b-9a17-05229dc0de11-kube-api-access-57gbg\") pod \"e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-c5dc6\" (UID: \"22af22dd-1130-409b-9a17-05229dc0de11\") " pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-c5dc6"
Oct 31 17:36:30.864710 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:30.864558    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/22af22dd-1130-409b-9a17-05229dc0de11-istio-data\") pod \"e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-c5dc6\" (UID: \"22af22dd-1130-409b-9a17-05229dc0de11\") " pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-c5dc6"
Oct 31 17:36:30.864710 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:30.864594    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/22af22dd-1130-409b-9a17-05229dc0de11-workload-certs\") pod \"e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-c5dc6\" (UID: \"22af22dd-1130-409b-9a17-05229dc0de11\") " pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-c5dc6"
Oct 31 17:36:30.864710 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:30.864667    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/22af22dd-1130-409b-9a17-05229dc0de11-istio-podinfo\") pod \"e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-c5dc6\" (UID: \"22af22dd-1130-409b-9a17-05229dc0de11\") " pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-c5dc6"
Oct 31 17:36:30.864710 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:30.864699    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/22af22dd-1130-409b-9a17-05229dc0de11-istio-envoy\") pod \"e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-c5dc6\" (UID: \"22af22dd-1130-409b-9a17-05229dc0de11\") " pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-c5dc6"
Oct 31 17:36:30.865003 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:30.864727    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/22af22dd-1130-409b-9a17-05229dc0de11-istio-token\") pod \"e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-c5dc6\" (UID: \"22af22dd-1130-409b-9a17-05229dc0de11\") " pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-c5dc6"
Oct 31 17:36:30.865003 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:30.864753    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/22af22dd-1130-409b-9a17-05229dc0de11-credential-socket\") pod \"e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-c5dc6\" (UID: \"22af22dd-1130-409b-9a17-05229dc0de11\") " pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-c5dc6"
Oct 31 17:36:30.865003 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:30.864775    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/22af22dd-1130-409b-9a17-05229dc0de11-istiod-ca-cert\") pod \"e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-c5dc6\" (UID: \"22af22dd-1130-409b-9a17-05229dc0de11\") " pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-c5dc6"
Oct 31 17:36:30.865003 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:30.864936    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/22af22dd-1130-409b-9a17-05229dc0de11-workload-socket\") pod \"e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-c5dc6\" (UID: \"22af22dd-1130-409b-9a17-05229dc0de11\") " pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-c5dc6"
Oct 31 17:36:30.865219 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:30.865180    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/22af22dd-1130-409b-9a17-05229dc0de11-istio-data\") pod \"e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-c5dc6\" (UID: \"22af22dd-1130-409b-9a17-05229dc0de11\") " pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-c5dc6"
Oct 31 17:36:30.865421 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:30.865400    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/22af22dd-1130-409b-9a17-05229dc0de11-workload-certs\") pod \"e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-c5dc6\" (UID: \"22af22dd-1130-409b-9a17-05229dc0de11\") " pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-c5dc6"
Oct 31 17:36:30.865540 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:30.865521    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/22af22dd-1130-409b-9a17-05229dc0de11-istiod-ca-cert\") pod \"e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-c5dc6\" (UID: \"22af22dd-1130-409b-9a17-05229dc0de11\") " pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-c5dc6"
Oct 31 17:36:30.865688 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:30.865667    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/22af22dd-1130-409b-9a17-05229dc0de11-credential-socket\") pod \"e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-c5dc6\" (UID: \"22af22dd-1130-409b-9a17-05229dc0de11\") " pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-c5dc6"
Oct 31 17:36:30.867928 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:30.867895    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/22af22dd-1130-409b-9a17-05229dc0de11-istio-envoy\") pod \"e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-c5dc6\" (UID: \"22af22dd-1130-409b-9a17-05229dc0de11\") " pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-c5dc6"
Oct 31 17:36:30.868108 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:30.868089    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/22af22dd-1130-409b-9a17-05229dc0de11-istio-podinfo\") pod \"e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-c5dc6\" (UID: \"22af22dd-1130-409b-9a17-05229dc0de11\") " pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-c5dc6"
Oct 31 17:36:30.886493 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:30.886436    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/22af22dd-1130-409b-9a17-05229dc0de11-istio-token\") pod \"e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-c5dc6\" (UID: \"22af22dd-1130-409b-9a17-05229dc0de11\") " pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-c5dc6"
Oct 31 17:36:30.886675 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:30.886652    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-57gbg\" (UniqueName: \"kubernetes.io/projected/22af22dd-1130-409b-9a17-05229dc0de11-kube-api-access-57gbg\") pod \"e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-c5dc6\" (UID: \"22af22dd-1130-409b-9a17-05229dc0de11\") " pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-c5dc6"
Oct 31 17:36:30.902421 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:30.902397    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:36:30.952729 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:30.952666    4276 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-ghwks"]
Oct 31 17:36:30.957692 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:30.957672    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-ghwks"
Oct 31 17:36:30.976282 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:30.976259    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w4-1c0bfd-openshift-default-dockercfg-27xm6\""
Oct 31 17:36:30.980708 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:30.980661    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-ghwks"]
Oct 31 17:36:31.002632 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:31.002605    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-c5dc6"
Oct 31 17:36:31.027508 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:31.027470    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:36:31.067363 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:31.066853    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/33671198-f448-4f99-9bc9-6a1ed6980c5b-istio-data\") pod \"e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-ghwks\" (UID: \"33671198-f448-4f99-9bc9-6a1ed6980c5b\") " pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-ghwks"
Oct 31 17:36:31.067363 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:31.066896    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/33671198-f448-4f99-9bc9-6a1ed6980c5b-istio-envoy\") pod \"e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-ghwks\" (UID: \"33671198-f448-4f99-9bc9-6a1ed6980c5b\") " pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-ghwks"
Oct 31 17:36:31.067363 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:31.066927    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/33671198-f448-4f99-9bc9-6a1ed6980c5b-istio-podinfo\") pod \"e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-ghwks\" (UID: \"33671198-f448-4f99-9bc9-6a1ed6980c5b\") " pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-ghwks"
Oct 31 17:36:31.067363 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:31.066958    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/33671198-f448-4f99-9bc9-6a1ed6980c5b-istiod-ca-cert\") pod \"e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-ghwks\" (UID: \"33671198-f448-4f99-9bc9-6a1ed6980c5b\") " pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-ghwks"
Oct 31 17:36:31.067363 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:31.067002    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/33671198-f448-4f99-9bc9-6a1ed6980c5b-workload-certs\") pod \"e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-ghwks\" (UID: \"33671198-f448-4f99-9bc9-6a1ed6980c5b\") " pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-ghwks"
Oct 31 17:36:31.067363 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:31.067029    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7tmrh\" (UniqueName: \"kubernetes.io/projected/33671198-f448-4f99-9bc9-6a1ed6980c5b-kube-api-access-7tmrh\") pod \"e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-ghwks\" (UID: \"33671198-f448-4f99-9bc9-6a1ed6980c5b\") " pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-ghwks"
Oct 31 17:36:31.067363 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:31.067109    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/33671198-f448-4f99-9bc9-6a1ed6980c5b-istio-token\") pod \"e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-ghwks\" (UID: \"33671198-f448-4f99-9bc9-6a1ed6980c5b\") " pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-ghwks"
Oct 31 17:36:31.067363 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:31.067146    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/33671198-f448-4f99-9bc9-6a1ed6980c5b-credential-socket\") pod \"e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-ghwks\" (UID: \"33671198-f448-4f99-9bc9-6a1ed6980c5b\") " pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-ghwks"
Oct 31 17:36:31.067363 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:31.067188    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/33671198-f448-4f99-9bc9-6a1ed6980c5b-workload-socket\") pod \"e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-ghwks\" (UID: \"33671198-f448-4f99-9bc9-6a1ed6980c5b\") " pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-ghwks"
Oct 31 17:36:31.158089 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:31.158054    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:36:31.168331 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:31.168302    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/33671198-f448-4f99-9bc9-6a1ed6980c5b-istio-data\") pod \"e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-ghwks\" (UID: \"33671198-f448-4f99-9bc9-6a1ed6980c5b\") " pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-ghwks"
Oct 31 17:36:31.169050 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:31.168346    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/33671198-f448-4f99-9bc9-6a1ed6980c5b-istio-envoy\") pod \"e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-ghwks\" (UID: \"33671198-f448-4f99-9bc9-6a1ed6980c5b\") " pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-ghwks"
Oct 31 17:36:31.169050 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:31.168539    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/33671198-f448-4f99-9bc9-6a1ed6980c5b-istio-podinfo\") pod \"e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-ghwks\" (UID: \"33671198-f448-4f99-9bc9-6a1ed6980c5b\") " pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-ghwks"
Oct 31 17:36:31.169050 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:31.168576    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/33671198-f448-4f99-9bc9-6a1ed6980c5b-istiod-ca-cert\") pod \"e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-ghwks\" (UID: \"33671198-f448-4f99-9bc9-6a1ed6980c5b\") " pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-ghwks"
Oct 31 17:36:31.169050 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:31.168637    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/33671198-f448-4f99-9bc9-6a1ed6980c5b-workload-certs\") pod \"e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-ghwks\" (UID: \"33671198-f448-4f99-9bc9-6a1ed6980c5b\") " pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-ghwks"
Oct 31 17:36:31.169050 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:31.168671    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7tmrh\" (UniqueName: \"kubernetes.io/projected/33671198-f448-4f99-9bc9-6a1ed6980c5b-kube-api-access-7tmrh\") pod \"e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-ghwks\" (UID: \"33671198-f448-4f99-9bc9-6a1ed6980c5b\") " pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-ghwks"
Oct 31 17:36:31.169050 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:31.168759    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/33671198-f448-4f99-9bc9-6a1ed6980c5b-istio-token\") pod \"e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-ghwks\" (UID: \"33671198-f448-4f99-9bc9-6a1ed6980c5b\") " pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-ghwks"
Oct 31 17:36:31.169050 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:31.168797    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/33671198-f448-4f99-9bc9-6a1ed6980c5b-credential-socket\") pod \"e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-ghwks\" (UID: \"33671198-f448-4f99-9bc9-6a1ed6980c5b\") " pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-ghwks"
Oct 31 17:36:31.169050 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:31.168840    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/33671198-f448-4f99-9bc9-6a1ed6980c5b-workload-socket\") pod \"e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-ghwks\" (UID: \"33671198-f448-4f99-9bc9-6a1ed6980c5b\") " pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-ghwks"
Oct 31 17:36:31.169602 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:31.169191    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/33671198-f448-4f99-9bc9-6a1ed6980c5b-workload-socket\") pod \"e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-ghwks\" (UID: \"33671198-f448-4f99-9bc9-6a1ed6980c5b\") " pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-ghwks"
Oct 31 17:36:31.170186 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:31.170165    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/33671198-f448-4f99-9bc9-6a1ed6980c5b-istio-data\") pod \"e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-ghwks\" (UID: \"33671198-f448-4f99-9bc9-6a1ed6980c5b\") " pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-ghwks"
Oct 31 17:36:31.170186 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:31.170179    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/33671198-f448-4f99-9bc9-6a1ed6980c5b-workload-certs\") pod \"e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-ghwks\" (UID: \"33671198-f448-4f99-9bc9-6a1ed6980c5b\") " pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-ghwks"
Oct 31 17:36:31.170368 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:31.170349    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/33671198-f448-4f99-9bc9-6a1ed6980c5b-credential-socket\") pod \"e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-ghwks\" (UID: \"33671198-f448-4f99-9bc9-6a1ed6980c5b\") " pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-ghwks"
Oct 31 17:36:31.170424 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:31.170349    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/33671198-f448-4f99-9bc9-6a1ed6980c5b-istiod-ca-cert\") pod \"e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-ghwks\" (UID: \"33671198-f448-4f99-9bc9-6a1ed6980c5b\") " pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-ghwks"
Oct 31 17:36:31.171243 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:31.171221    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/33671198-f448-4f99-9bc9-6a1ed6980c5b-istio-envoy\") pod \"e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-ghwks\" (UID: \"33671198-f448-4f99-9bc9-6a1ed6980c5b\") " pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-ghwks"
Oct 31 17:36:31.172504 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:31.172478    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/33671198-f448-4f99-9bc9-6a1ed6980c5b-istio-podinfo\") pod \"e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-ghwks\" (UID: \"33671198-f448-4f99-9bc9-6a1ed6980c5b\") " pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-ghwks"
Oct 31 17:36:31.192852 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:31.192831    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/33671198-f448-4f99-9bc9-6a1ed6980c5b-istio-token\") pod \"e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-ghwks\" (UID: \"33671198-f448-4f99-9bc9-6a1ed6980c5b\") " pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-ghwks"
Oct 31 17:36:31.192967 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:31.192881    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7tmrh\" (UniqueName: \"kubernetes.io/projected/33671198-f448-4f99-9bc9-6a1ed6980c5b-kube-api-access-7tmrh\") pod \"e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-ghwks\" (UID: \"33671198-f448-4f99-9bc9-6a1ed6980c5b\") " pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-ghwks"
Oct 31 17:36:31.217186 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:31.217116    4276 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-jxpd8"]
Oct 31 17:36:31.222787 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:31.222768    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-jxpd8"
Oct 31 17:36:31.241615 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:31.241586    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w2-5c446c-openshift-default-dockercfg-kfjdh\""
Oct 31 17:36:31.242385 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:31.242250    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-jxpd8"]
Oct 31 17:36:31.273575 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:31.273542    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:36:31.275488 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:31.275463    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-ghwks"
Oct 31 17:36:31.320085 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:31.320055    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-c5dc6"]
Oct 31 17:36:31.327491 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:36:31.326542    4276 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod22af22dd_1130_409b_9a17_05229dc0de11.slice/crio-43fcf2d064994ee73dff7f5da6a887f6283bbe413a689094db8a54614c485a06 WatchSource:0}: Error finding container 43fcf2d064994ee73dff7f5da6a887f6283bbe413a689094db8a54614c485a06: Status 404 returned error can't find the container with id 43fcf2d064994ee73dff7f5da6a887f6283bbe413a689094db8a54614c485a06
Oct 31 17:36:31.329060 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:31.329018    4276 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 17:36:31.329184 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:31.329102    4276 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 17:36:31.329184 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:31.329146    4276 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 17:36:31.372319 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:31.372293    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9d24572d-ec8f-4108-a83e-704c5ae976ff-istiod-ca-cert\") pod \"e2e-worker-w2-5c446c-openshift-default-797c6dc855-jxpd8\" (UID: \"9d24572d-ec8f-4108-a83e-704c5ae976ff\") " pod="openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-jxpd8"
Oct 31 17:36:31.372545 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:31.372525    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9d24572d-ec8f-4108-a83e-704c5ae976ff-workload-certs\") pod \"e2e-worker-w2-5c446c-openshift-default-797c6dc855-jxpd8\" (UID: \"9d24572d-ec8f-4108-a83e-704c5ae976ff\") " pod="openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-jxpd8"
Oct 31 17:36:31.372672 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:31.372657    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d24572d-ec8f-4108-a83e-704c5ae976ff-workload-socket\") pod \"e2e-worker-w2-5c446c-openshift-default-797c6dc855-jxpd8\" (UID: \"9d24572d-ec8f-4108-a83e-704c5ae976ff\") " pod="openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-jxpd8"
Oct 31 17:36:31.372771 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:31.372758    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d24572d-ec8f-4108-a83e-704c5ae976ff-credential-socket\") pod \"e2e-worker-w2-5c446c-openshift-default-797c6dc855-jxpd8\" (UID: \"9d24572d-ec8f-4108-a83e-704c5ae976ff\") " pod="openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-jxpd8"
Oct 31 17:36:31.372885 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:31.372874    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9d24572d-ec8f-4108-a83e-704c5ae976ff-istio-envoy\") pod \"e2e-worker-w2-5c446c-openshift-default-797c6dc855-jxpd8\" (UID: \"9d24572d-ec8f-4108-a83e-704c5ae976ff\") " pod="openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-jxpd8"
Oct 31 17:36:31.372996 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:31.372984    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9d24572d-ec8f-4108-a83e-704c5ae976ff-istio-data\") pod \"e2e-worker-w2-5c446c-openshift-default-797c6dc855-jxpd8\" (UID: \"9d24572d-ec8f-4108-a83e-704c5ae976ff\") " pod="openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-jxpd8"
Oct 31 17:36:31.373084 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:31.373072    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sfs6z\" (UniqueName: \"kubernetes.io/projected/9d24572d-ec8f-4108-a83e-704c5ae976ff-kube-api-access-sfs6z\") pod \"e2e-worker-w2-5c446c-openshift-default-797c6dc855-jxpd8\" (UID: \"9d24572d-ec8f-4108-a83e-704c5ae976ff\") " pod="openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-jxpd8"
Oct 31 17:36:31.373194 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:31.373179    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9d24572d-ec8f-4108-a83e-704c5ae976ff-istio-token\") pod \"e2e-worker-w2-5c446c-openshift-default-797c6dc855-jxpd8\" (UID: \"9d24572d-ec8f-4108-a83e-704c5ae976ff\") " pod="openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-jxpd8"
Oct 31 17:36:31.373306 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:31.373294    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9d24572d-ec8f-4108-a83e-704c5ae976ff-istio-podinfo\") pod \"e2e-worker-w2-5c446c-openshift-default-797c6dc855-jxpd8\" (UID: \"9d24572d-ec8f-4108-a83e-704c5ae976ff\") " pod="openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-jxpd8"
Oct 31 17:36:31.413854 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:31.413763    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:36:31.475163 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:31.474779    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9d24572d-ec8f-4108-a83e-704c5ae976ff-istiod-ca-cert\") pod \"e2e-worker-w2-5c446c-openshift-default-797c6dc855-jxpd8\" (UID: \"9d24572d-ec8f-4108-a83e-704c5ae976ff\") " pod="openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-jxpd8"
Oct 31 17:36:31.475163 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:31.474828    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9d24572d-ec8f-4108-a83e-704c5ae976ff-workload-certs\") pod \"e2e-worker-w2-5c446c-openshift-default-797c6dc855-jxpd8\" (UID: \"9d24572d-ec8f-4108-a83e-704c5ae976ff\") " pod="openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-jxpd8"
Oct 31 17:36:31.475163 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:31.474867    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d24572d-ec8f-4108-a83e-704c5ae976ff-workload-socket\") pod \"e2e-worker-w2-5c446c-openshift-default-797c6dc855-jxpd8\" (UID: \"9d24572d-ec8f-4108-a83e-704c5ae976ff\") " pod="openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-jxpd8"
Oct 31 17:36:31.475163 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:31.474898    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d24572d-ec8f-4108-a83e-704c5ae976ff-credential-socket\") pod \"e2e-worker-w2-5c446c-openshift-default-797c6dc855-jxpd8\" (UID: \"9d24572d-ec8f-4108-a83e-704c5ae976ff\") " pod="openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-jxpd8"
Oct 31 17:36:31.475163 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:31.474963    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9d24572d-ec8f-4108-a83e-704c5ae976ff-istio-envoy\") pod \"e2e-worker-w2-5c446c-openshift-default-797c6dc855-jxpd8\" (UID: \"9d24572d-ec8f-4108-a83e-704c5ae976ff\") " pod="openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-jxpd8"
Oct 31 17:36:31.475163 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:31.475027    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9d24572d-ec8f-4108-a83e-704c5ae976ff-istio-data\") pod \"e2e-worker-w2-5c446c-openshift-default-797c6dc855-jxpd8\" (UID: \"9d24572d-ec8f-4108-a83e-704c5ae976ff\") " pod="openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-jxpd8"
Oct 31 17:36:31.475163 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:31.475051    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sfs6z\" (UniqueName: \"kubernetes.io/projected/9d24572d-ec8f-4108-a83e-704c5ae976ff-kube-api-access-sfs6z\") pod \"e2e-worker-w2-5c446c-openshift-default-797c6dc855-jxpd8\" (UID: \"9d24572d-ec8f-4108-a83e-704c5ae976ff\") " pod="openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-jxpd8"
Oct 31 17:36:31.476291 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:31.475750    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9d24572d-ec8f-4108-a83e-704c5ae976ff-istio-token\") pod \"e2e-worker-w2-5c446c-openshift-default-797c6dc855-jxpd8\" (UID: \"9d24572d-ec8f-4108-a83e-704c5ae976ff\") " pod="openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-jxpd8"
Oct 31 17:36:31.476291 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:31.475820    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9d24572d-ec8f-4108-a83e-704c5ae976ff-istio-podinfo\") pod \"e2e-worker-w2-5c446c-openshift-default-797c6dc855-jxpd8\" (UID: \"9d24572d-ec8f-4108-a83e-704c5ae976ff\") " pod="openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-jxpd8"
Oct 31 17:36:31.476291 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:31.476028    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d24572d-ec8f-4108-a83e-704c5ae976ff-credential-socket\") pod \"e2e-worker-w2-5c446c-openshift-default-797c6dc855-jxpd8\" (UID: \"9d24572d-ec8f-4108-a83e-704c5ae976ff\") " pod="openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-jxpd8"
Oct 31 17:36:31.476534 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:31.476498    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9d24572d-ec8f-4108-a83e-704c5ae976ff-workload-certs\") pod \"e2e-worker-w2-5c446c-openshift-default-797c6dc855-jxpd8\" (UID: \"9d24572d-ec8f-4108-a83e-704c5ae976ff\") " pod="openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-jxpd8"
Oct 31 17:36:31.476720 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:31.476702    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d24572d-ec8f-4108-a83e-704c5ae976ff-workload-socket\") pod \"e2e-worker-w2-5c446c-openshift-default-797c6dc855-jxpd8\" (UID: \"9d24572d-ec8f-4108-a83e-704c5ae976ff\") " pod="openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-jxpd8"
Oct 31 17:36:31.477708 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:31.477463    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9d24572d-ec8f-4108-a83e-704c5ae976ff-istio-data\") pod \"e2e-worker-w2-5c446c-openshift-default-797c6dc855-jxpd8\" (UID: \"9d24572d-ec8f-4108-a83e-704c5ae976ff\") " pod="openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-jxpd8"
Oct 31 17:36:31.478300 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:31.478263    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9d24572d-ec8f-4108-a83e-704c5ae976ff-istiod-ca-cert\") pod \"e2e-worker-w2-5c446c-openshift-default-797c6dc855-jxpd8\" (UID: \"9d24572d-ec8f-4108-a83e-704c5ae976ff\") " pod="openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-jxpd8"
Oct 31 17:36:31.479110 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:31.479074    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9d24572d-ec8f-4108-a83e-704c5ae976ff-istio-envoy\") pod \"e2e-worker-w2-5c446c-openshift-default-797c6dc855-jxpd8\" (UID: \"9d24572d-ec8f-4108-a83e-704c5ae976ff\") " pod="openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-jxpd8"
Oct 31 17:36:31.485691 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:31.485667    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9d24572d-ec8f-4108-a83e-704c5ae976ff-istio-podinfo\") pod \"e2e-worker-w2-5c446c-openshift-default-797c6dc855-jxpd8\" (UID: \"9d24572d-ec8f-4108-a83e-704c5ae976ff\") " pod="openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-jxpd8"
Oct 31 17:36:31.494245 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:31.494163    4276 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-f8xx7"]
Oct 31 17:36:31.500077 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:31.500056    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-f8xx7"
Oct 31 17:36:31.510537 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:31.509560    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sfs6z\" (UniqueName: \"kubernetes.io/projected/9d24572d-ec8f-4108-a83e-704c5ae976ff-kube-api-access-sfs6z\") pod \"e2e-worker-w2-5c446c-openshift-default-797c6dc855-jxpd8\" (UID: \"9d24572d-ec8f-4108-a83e-704c5ae976ff\") " pod="openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-jxpd8"
Oct 31 17:36:31.513877 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:31.511894    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9d24572d-ec8f-4108-a83e-704c5ae976ff-istio-token\") pod \"e2e-worker-w2-5c446c-openshift-default-797c6dc855-jxpd8\" (UID: \"9d24572d-ec8f-4108-a83e-704c5ae976ff\") " pod="openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-jxpd8"
Oct 31 17:36:31.523547 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:31.521193    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-f8xx7"]
Oct 31 17:36:31.523779 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:31.523184    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w1-0e59ca-openshift-default-dockercfg-75hsv\""
Oct 31 17:36:31.544822 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:31.544798    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-jxpd8"
Oct 31 17:36:31.546177 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:31.546152    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:36:31.577369 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:31.577340    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6418b389-728b-4040-9b73-50cf013c00f3-istio-token\") pod \"e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-f8xx7\" (UID: \"6418b389-728b-4040-9b73-50cf013c00f3\") " pod="openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-f8xx7"
Oct 31 17:36:31.577508 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:31.577403    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6418b389-728b-4040-9b73-50cf013c00f3-istiod-ca-cert\") pod \"e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-f8xx7\" (UID: \"6418b389-728b-4040-9b73-50cf013c00f3\") " pod="openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-f8xx7"
Oct 31 17:36:31.577508 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:31.577431    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6418b389-728b-4040-9b73-50cf013c00f3-workload-socket\") pod \"e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-f8xx7\" (UID: \"6418b389-728b-4040-9b73-50cf013c00f3\") " pod="openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-f8xx7"
Oct 31 17:36:31.577508 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:31.577468    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6418b389-728b-4040-9b73-50cf013c00f3-credential-socket\") pod \"e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-f8xx7\" (UID: \"6418b389-728b-4040-9b73-50cf013c00f3\") " pod="openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-f8xx7"
Oct 31 17:36:31.577947 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:31.577917    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6418b389-728b-4040-9b73-50cf013c00f3-istio-podinfo\") pod \"e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-f8xx7\" (UID: \"6418b389-728b-4040-9b73-50cf013c00f3\") " pod="openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-f8xx7"
Oct 31 17:36:31.578045 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:31.577977    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6418b389-728b-4040-9b73-50cf013c00f3-istio-envoy\") pod \"e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-f8xx7\" (UID: \"6418b389-728b-4040-9b73-50cf013c00f3\") " pod="openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-f8xx7"
Oct 31 17:36:31.578045 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:31.578015    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tlzgw\" (UniqueName: \"kubernetes.io/projected/6418b389-728b-4040-9b73-50cf013c00f3-kube-api-access-tlzgw\") pod \"e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-f8xx7\" (UID: \"6418b389-728b-4040-9b73-50cf013c00f3\") " pod="openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-f8xx7"
Oct 31 17:36:31.578155 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:31.578052    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6418b389-728b-4040-9b73-50cf013c00f3-istio-data\") pod \"e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-f8xx7\" (UID: \"6418b389-728b-4040-9b73-50cf013c00f3\") " pod="openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-f8xx7"
Oct 31 17:36:31.578155 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:31.578097    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6418b389-728b-4040-9b73-50cf013c00f3-workload-certs\") pod \"e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-f8xx7\" (UID: \"6418b389-728b-4040-9b73-50cf013c00f3\") " pod="openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-f8xx7"
Oct 31 17:36:31.587915 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:36:31.587879    4276 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33671198_f448_4f99_9bc9_6a1ed6980c5b.slice/crio-4558c1ed5122febee619605d8afdf42296fb026613c51bfe4bfbe4374ed469aa WatchSource:0}: Error finding container 4558c1ed5122febee619605d8afdf42296fb026613c51bfe4bfbe4374ed469aa: Status 404 returned error can't find the container with id 4558c1ed5122febee619605d8afdf42296fb026613c51bfe4bfbe4374ed469aa
Oct 31 17:36:31.588987 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:31.588291    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-ghwks"]
Oct 31 17:36:31.592340 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:31.591199    4276 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 17:36:31.592340 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:31.591269    4276 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 17:36:31.592340 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:31.591309    4276 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 17:36:31.610139 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:31.610109    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-c5dc6" event={"ID":"22af22dd-1130-409b-9a17-05229dc0de11","Type":"ContainerStarted","Data":"5563c62e742bdd643b7b91521fa78ea55c3acbe6686b95695605135a70068dcc"}
Oct 31 17:36:31.610253 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:31.610211    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-c5dc6" event={"ID":"22af22dd-1130-409b-9a17-05229dc0de11","Type":"ContainerStarted","Data":"43fcf2d064994ee73dff7f5da6a887f6283bbe413a689094db8a54614c485a06"}
Oct 31 17:36:31.612601 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:31.612486    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-ghwks" event={"ID":"33671198-f448-4f99-9bc9-6a1ed6980c5b","Type":"ContainerStarted","Data":"4558c1ed5122febee619605d8afdf42296fb026613c51bfe4bfbe4374ed469aa"}
Oct 31 17:36:31.670377 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:31.670345    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:36:31.695044 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:31.695007    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6418b389-728b-4040-9b73-50cf013c00f3-istio-podinfo\") pod \"e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-f8xx7\" (UID: \"6418b389-728b-4040-9b73-50cf013c00f3\") " pod="openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-f8xx7"
Oct 31 17:36:31.697161 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:31.697135    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6418b389-728b-4040-9b73-50cf013c00f3-istio-envoy\") pod \"e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-f8xx7\" (UID: \"6418b389-728b-4040-9b73-50cf013c00f3\") " pod="openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-f8xx7"
Oct 31 17:36:31.697368 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:31.697351    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tlzgw\" (UniqueName: \"kubernetes.io/projected/6418b389-728b-4040-9b73-50cf013c00f3-kube-api-access-tlzgw\") pod \"e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-f8xx7\" (UID: \"6418b389-728b-4040-9b73-50cf013c00f3\") " pod="openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-f8xx7"
Oct 31 17:36:31.698244 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:31.698215    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6418b389-728b-4040-9b73-50cf013c00f3-istio-data\") pod \"e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-f8xx7\" (UID: \"6418b389-728b-4040-9b73-50cf013c00f3\") " pod="openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-f8xx7"
Oct 31 17:36:31.698800 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:31.698651    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6418b389-728b-4040-9b73-50cf013c00f3-istio-data\") pod \"e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-f8xx7\" (UID: \"6418b389-728b-4040-9b73-50cf013c00f3\") " pod="openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-f8xx7"
Oct 31 17:36:31.699205 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:31.699163    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6418b389-728b-4040-9b73-50cf013c00f3-workload-certs\") pod \"e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-f8xx7\" (UID: \"6418b389-728b-4040-9b73-50cf013c00f3\") " pod="openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-f8xx7"
Oct 31 17:36:31.699328 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:31.699312    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6418b389-728b-4040-9b73-50cf013c00f3-workload-certs\") pod \"e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-f8xx7\" (UID: \"6418b389-728b-4040-9b73-50cf013c00f3\") " pod="openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-f8xx7"
Oct 31 17:36:31.699743 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:31.699437    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6418b389-728b-4040-9b73-50cf013c00f3-istio-token\") pod \"e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-f8xx7\" (UID: \"6418b389-728b-4040-9b73-50cf013c00f3\") " pod="openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-f8xx7"
Oct 31 17:36:31.700165 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:31.700145    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6418b389-728b-4040-9b73-50cf013c00f3-istiod-ca-cert\") pod \"e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-f8xx7\" (UID: \"6418b389-728b-4040-9b73-50cf013c00f3\") " pod="openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-f8xx7"
Oct 31 17:36:31.701256 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:31.700987    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6418b389-728b-4040-9b73-50cf013c00f3-workload-socket\") pod \"e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-f8xx7\" (UID: \"6418b389-728b-4040-9b73-50cf013c00f3\") " pod="openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-f8xx7"
Oct 31 17:36:31.701487 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:31.700919    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6418b389-728b-4040-9b73-50cf013c00f3-istiod-ca-cert\") pod \"e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-f8xx7\" (UID: \"6418b389-728b-4040-9b73-50cf013c00f3\") " pod="openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-f8xx7"
Oct 31 17:36:31.701487 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:31.701237    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6418b389-728b-4040-9b73-50cf013c00f3-workload-socket\") pod \"e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-f8xx7\" (UID: \"6418b389-728b-4040-9b73-50cf013c00f3\") " pod="openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-f8xx7"
Oct 31 17:36:31.701898 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:31.701860    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6418b389-728b-4040-9b73-50cf013c00f3-credential-socket\") pod \"e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-f8xx7\" (UID: \"6418b389-728b-4040-9b73-50cf013c00f3\") " pod="openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-f8xx7"
Oct 31 17:36:31.702000 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:31.701985    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6418b389-728b-4040-9b73-50cf013c00f3-credential-socket\") pod \"e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-f8xx7\" (UID: \"6418b389-728b-4040-9b73-50cf013c00f3\") " pod="openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-f8xx7"
Oct 31 17:36:31.712542 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:31.712483    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6418b389-728b-4040-9b73-50cf013c00f3-istio-podinfo\") pod \"e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-f8xx7\" (UID: \"6418b389-728b-4040-9b73-50cf013c00f3\") " pod="openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-f8xx7"
Oct 31 17:36:31.713983 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:31.713957    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6418b389-728b-4040-9b73-50cf013c00f3-istio-envoy\") pod \"e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-f8xx7\" (UID: \"6418b389-728b-4040-9b73-50cf013c00f3\") " pod="openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-f8xx7"
Oct 31 17:36:31.721691 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:31.721618    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tlzgw\" (UniqueName: \"kubernetes.io/projected/6418b389-728b-4040-9b73-50cf013c00f3-kube-api-access-tlzgw\") pod \"e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-f8xx7\" (UID: \"6418b389-728b-4040-9b73-50cf013c00f3\") " pod="openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-f8xx7"
Oct 31 17:36:31.728824 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:31.728798    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6418b389-728b-4040-9b73-50cf013c00f3-istio-token\") pod \"e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-f8xx7\" (UID: \"6418b389-728b-4040-9b73-50cf013c00f3\") " pod="openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-f8xx7"
Oct 31 17:36:31.799422 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:31.799271    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:36:31.823458 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:31.823411    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-f8xx7"
Oct 31 17:36:31.874627 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:31.874560    4276 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-c5dc6" podStartSLOduration=1.8745381330000002 podStartE2EDuration="1.874538133s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 17:36:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 17:36:31.66819089 +0000 UTC m=+554.465935851" watchObservedRunningTime="2026-10-31 17:36:31.874538133 +0000 UTC m=+554.672283064"
Oct 31 17:36:31.875985 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:31.875957    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-jxpd8"]
Oct 31 17:36:31.886681 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:31.886648    4276 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 17:36:31.886790 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:31.886733    4276 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 17:36:31.886790 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:31.886778    4276 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 17:36:32.003131 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:32.002986    4276 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-c5dc6"
Oct 31 17:36:32.008975 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:32.008949    4276 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-c5dc6"
Oct 31 17:36:32.138927 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:32.138903    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-f8xx7"]
Oct 31 17:36:32.141122 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:36:32.141097    4276 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6418b389_728b_4040_9b73_50cf013c00f3.slice/crio-b4f7da10e113f7e2313e27cdadbe89772b76ca704549a277cb7c5505ff4c0465 WatchSource:0}: Error finding container b4f7da10e113f7e2313e27cdadbe89772b76ca704549a277cb7c5505ff4c0465: Status 404 returned error can't find the container with id b4f7da10e113f7e2313e27cdadbe89772b76ca704549a277cb7c5505ff4c0465
Oct 31 17:36:32.143992 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:32.143957    4276 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 17:36:32.144097 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:32.144032    4276 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 17:36:32.144097 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:32.144075    4276 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 17:36:32.619763 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:32.619734    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-ghwks" event={"ID":"33671198-f448-4f99-9bc9-6a1ed6980c5b","Type":"ContainerStarted","Data":"7aec3c6fddcf595c3e7e91ec3662c167ee611c0ab99dc0d5a11652e39a6c6d26"}
Oct 31 17:36:32.621436 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:32.621409    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-jxpd8" event={"ID":"9d24572d-ec8f-4108-a83e-704c5ae976ff","Type":"ContainerStarted","Data":"e75c910ccf22e78e7ef8f45fcb517c57021f73f93f4dbca44088425c83d8ae28"}
Oct 31 17:36:32.621601 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:32.621475    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-jxpd8" event={"ID":"9d24572d-ec8f-4108-a83e-704c5ae976ff","Type":"ContainerStarted","Data":"38671b61e1c1f0b72d1a7065c7eaac7e7a051a87dd9c87cbfb3e321ec89ade6f"}
Oct 31 17:36:32.622951 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:32.622918    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-f8xx7" event={"ID":"6418b389-728b-4040-9b73-50cf013c00f3","Type":"ContainerStarted","Data":"ab10caa28f333e0be2ae6f6f40c4d874ad5fcc690e27588b9729c82508d8e8a0"}
Oct 31 17:36:32.623058 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:32.622966    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-f8xx7" event={"ID":"6418b389-728b-4040-9b73-50cf013c00f3","Type":"ContainerStarted","Data":"b4f7da10e113f7e2313e27cdadbe89772b76ca704549a277cb7c5505ff4c0465"}
Oct 31 17:36:32.623166 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:32.623147    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-c5dc6"
Oct 31 17:36:32.623937 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:32.623919    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-c5dc6"
Oct 31 17:36:32.663544 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:32.663491    4276 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-ghwks" podStartSLOduration=2.66347492 podStartE2EDuration="2.66347492s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 17:36:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 17:36:32.662861558 +0000 UTC m=+555.460606485" watchObservedRunningTime="2026-10-31 17:36:32.66347492 +0000 UTC m=+555.461219841"
Oct 31 17:36:32.750605 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:32.750553    4276 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-jxpd8" podStartSLOduration=1.75053909 podStartE2EDuration="1.75053909s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 17:36:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 17:36:32.748484274 +0000 UTC m=+555.546229201" watchObservedRunningTime="2026-10-31 17:36:32.75053909 +0000 UTC m=+555.548284015"
Oct 31 17:36:32.793524 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:32.793474    4276 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-f8xx7" podStartSLOduration=1.793437272 podStartE2EDuration="1.793437272s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 17:36:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 17:36:32.791601208 +0000 UTC m=+555.589346136" watchObservedRunningTime="2026-10-31 17:36:32.793437272 +0000 UTC m=+555.591182197"
Oct 31 17:36:32.824377 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:32.824349    4276 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-f8xx7"
Oct 31 17:36:33.276516 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:33.276484    4276 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-ghwks"
Oct 31 17:36:33.281638 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:33.281617    4276 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-ghwks"
Oct 31 17:36:33.545808 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:33.545730    4276 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-jxpd8"
Oct 31 17:36:33.551349 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:33.551318    4276 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-jxpd8"
Oct 31 17:36:33.628426 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:33.628389    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-jxpd8"
Oct 31 17:36:33.628901 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:33.628739    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-ghwks"
Oct 31 17:36:33.629521 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:33.629498    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-ghwks"
Oct 31 17:36:33.629622 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:33.629559    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-jxpd8"
Oct 31 17:36:33.832581 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:33.832553    4276 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-f8xx7"
Oct 31 17:36:34.052885 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:34.052851    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:36:34.169192 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:34.169116    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:36:34.605908 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:34.605877    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:36:34.633599 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:34.633564    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-f8xx7"
Oct 31 17:36:34.634140 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:34.634121    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-f8xx7"
Oct 31 17:36:34.728610 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:34.728563    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:36:34.861781 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:34.861705    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:36:34.987860 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:34.987832    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:36:35.121073 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:35.120996    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:36:35.644827 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:35.644778    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:36:35.838047 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:35.838018    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:36:35.955639 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:35.955569    4276 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-cf0a86-7bb4c5d6c5-8p8r5"]
Oct 31 17:36:35.958881 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:35.958864    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-cf0a86-7bb4c5d6c5-8p8r5"
Oct 31 17:36:35.978134 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:35.978107    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:36:35.979077 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:35.979055    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-cf0a86-7bb4c5d6c5-8p8r5"]
Oct 31 17:36:35.990710 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:35.990682    4276 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w3-cf0a86\""
Oct 31 17:36:36.078789 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:36.078755    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w3-cf0a86-dockercfg-xtfx8\""
Oct 31 17:36:36.120707 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:36.120675    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:36:36.162853 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:36.162820    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1bd48c9b-b035-4931-8cf6-dbb821d3f6d5-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w3-cf0a86-7bb4c5d6c5-8p8r5\" (UID: \"1bd48c9b-b035-4931-8cf6-dbb821d3f6d5\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-cf0a86-7bb4c5d6c5-8p8r5"
Oct 31 17:36:36.162994 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:36.162926    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nj7st\" (UniqueName: \"kubernetes.io/projected/1bd48c9b-b035-4931-8cf6-dbb821d3f6d5-kube-api-access-nj7st\") pod \"payload-pre-processing-e2e-worker-w3-cf0a86-7bb4c5d6c5-8p8r5\" (UID: \"1bd48c9b-b035-4931-8cf6-dbb821d3f6d5\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-cf0a86-7bb4c5d6c5-8p8r5"
Oct 31 17:36:36.259979 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:36.259874    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:36:36.264124 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:36.264087    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nj7st\" (UniqueName: \"kubernetes.io/projected/1bd48c9b-b035-4931-8cf6-dbb821d3f6d5-kube-api-access-nj7st\") pod \"payload-pre-processing-e2e-worker-w3-cf0a86-7bb4c5d6c5-8p8r5\" (UID: \"1bd48c9b-b035-4931-8cf6-dbb821d3f6d5\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-cf0a86-7bb4c5d6c5-8p8r5"
Oct 31 17:36:36.264275 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:36.264257    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1bd48c9b-b035-4931-8cf6-dbb821d3f6d5-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w3-cf0a86-7bb4c5d6c5-8p8r5\" (UID: \"1bd48c9b-b035-4931-8cf6-dbb821d3f6d5\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-cf0a86-7bb4c5d6c5-8p8r5"
Oct 31 17:36:36.264958 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:36.264934    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1bd48c9b-b035-4931-8cf6-dbb821d3f6d5-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w3-cf0a86-7bb4c5d6c5-8p8r5\" (UID: \"1bd48c9b-b035-4931-8cf6-dbb821d3f6d5\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-cf0a86-7bb4c5d6c5-8p8r5"
Oct 31 17:36:36.293009 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:36.292985    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nj7st\" (UniqueName: \"kubernetes.io/projected/1bd48c9b-b035-4931-8cf6-dbb821d3f6d5-kube-api-access-nj7st\") pod \"payload-pre-processing-e2e-worker-w3-cf0a86-7bb4c5d6c5-8p8r5\" (UID: \"1bd48c9b-b035-4931-8cf6-dbb821d3f6d5\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-cf0a86-7bb4c5d6c5-8p8r5"
Oct 31 17:36:36.398620 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:36.398589    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:36:36.571430 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:36.571397    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-cf0a86-7bb4c5d6c5-8p8r5"
Oct 31 17:36:36.879024 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:36.879001    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-cf0a86-7bb4c5d6c5-8p8r5"]
Oct 31 17:36:36.881150 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:36:36.881119    4276 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1bd48c9b_b035_4931_8cf6_dbb821d3f6d5.slice/crio-66b7496e33f41a6800ca712c00b0a7967ff7539d57fe57eea77cbb80f482877f WatchSource:0}: Error finding container 66b7496e33f41a6800ca712c00b0a7967ff7539d57fe57eea77cbb80f482877f: Status 404 returned error can't find the container with id 66b7496e33f41a6800ca712c00b0a7967ff7539d57fe57eea77cbb80f482877f
Oct 31 17:36:37.133131 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:37.133058    4276 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-d80080-869f895d55-s62hv"]
Oct 31 17:36:37.136505 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:37.136488    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-d80080-869f895d55-s62hv"
Oct 31 17:36:37.156311 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:37.156281    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-d80080-869f895d55-s62hv"]
Oct 31 17:36:37.171895 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:37.171869    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/175639e5-138e-428e-bb53-7fe484543496-maas-api-tls\") pod \"maas-api-e2e-worker-w6-d80080-869f895d55-s62hv\" (UID: \"175639e5-138e-428e-bb53-7fe484543496\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-d80080-869f895d55-s62hv"
Oct 31 17:36:37.172016 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:37.171921    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dw6tr\" (UniqueName: \"kubernetes.io/projected/175639e5-138e-428e-bb53-7fe484543496-kube-api-access-dw6tr\") pod \"maas-api-e2e-worker-w6-d80080-869f895d55-s62hv\" (UID: \"175639e5-138e-428e-bb53-7fe484543496\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-d80080-869f895d55-s62hv"
Oct 31 17:36:37.172016 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:37.171994    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/175639e5-138e-428e-bb53-7fe484543496-metrics-certs\") pod \"maas-api-e2e-worker-w6-d80080-869f895d55-s62hv\" (UID: \"175639e5-138e-428e-bb53-7fe484543496\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-d80080-869f895d55-s62hv"
Oct 31 17:36:37.172095 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:37.172036    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/175639e5-138e-428e-bb53-7fe484543496-openshift-service-ca\") pod \"maas-api-e2e-worker-w6-d80080-869f895d55-s62hv\" (UID: \"175639e5-138e-428e-bb53-7fe484543496\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-d80080-869f895d55-s62hv"
Oct 31 17:36:37.226964 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:37.226934    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w6-d80080-serving-cert\""
Oct 31 17:36:37.235927 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:37.235900    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:36:37.272565 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:37.272544    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/175639e5-138e-428e-bb53-7fe484543496-maas-api-tls\") pod \"maas-api-e2e-worker-w6-d80080-869f895d55-s62hv\" (UID: \"175639e5-138e-428e-bb53-7fe484543496\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-d80080-869f895d55-s62hv"
Oct 31 17:36:37.272659 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:37.272597    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dw6tr\" (UniqueName: \"kubernetes.io/projected/175639e5-138e-428e-bb53-7fe484543496-kube-api-access-dw6tr\") pod \"maas-api-e2e-worker-w6-d80080-869f895d55-s62hv\" (UID: \"175639e5-138e-428e-bb53-7fe484543496\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-d80080-869f895d55-s62hv"
Oct 31 17:36:37.272741 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:37.272720    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/175639e5-138e-428e-bb53-7fe484543496-metrics-certs\") pod \"maas-api-e2e-worker-w6-d80080-869f895d55-s62hv\" (UID: \"175639e5-138e-428e-bb53-7fe484543496\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-d80080-869f895d55-s62hv"
Oct 31 17:36:37.272799 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:37.272778    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/175639e5-138e-428e-bb53-7fe484543496-openshift-service-ca\") pod \"maas-api-e2e-worker-w6-d80080-869f895d55-s62hv\" (UID: \"175639e5-138e-428e-bb53-7fe484543496\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-d80080-869f895d55-s62hv"
Oct 31 17:36:37.273398 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:37.273380    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/175639e5-138e-428e-bb53-7fe484543496-openshift-service-ca\") pod \"maas-api-e2e-worker-w6-d80080-869f895d55-s62hv\" (UID: \"175639e5-138e-428e-bb53-7fe484543496\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-d80080-869f895d55-s62hv"
Oct 31 17:36:37.275033 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:37.275010    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/175639e5-138e-428e-bb53-7fe484543496-metrics-certs\") pod \"maas-api-e2e-worker-w6-d80080-869f895d55-s62hv\" (UID: \"175639e5-138e-428e-bb53-7fe484543496\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-d80080-869f895d55-s62hv"
Oct 31 17:36:37.275123 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:37.275011    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/175639e5-138e-428e-bb53-7fe484543496-maas-api-tls\") pod \"maas-api-e2e-worker-w6-d80080-869f895d55-s62hv\" (UID: \"175639e5-138e-428e-bb53-7fe484543496\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-d80080-869f895d55-s62hv"
Oct 31 17:36:37.292720 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:37.292677    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dw6tr\" (UniqueName: \"kubernetes.io/projected/175639e5-138e-428e-bb53-7fe484543496-kube-api-access-dw6tr\") pod \"maas-api-e2e-worker-w6-d80080-869f895d55-s62hv\" (UID: \"175639e5-138e-428e-bb53-7fe484543496\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-d80080-869f895d55-s62hv"
Oct 31 17:36:37.449941 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:37.449876    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-d80080-869f895d55-s62hv"
Oct 31 17:36:37.651155 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:37.651121    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-cf0a86-7bb4c5d6c5-8p8r5" event={"ID":"1bd48c9b-b035-4931-8cf6-dbb821d3f6d5","Type":"ContainerStarted","Data":"fe5960e6e112614bcdb6c6cd41714ed850389b563a08d6c938b3db43b17ebfc4"}
Oct 31 17:36:37.651339 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:37.651173    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-cf0a86-7bb4c5d6c5-8p8r5" event={"ID":"1bd48c9b-b035-4931-8cf6-dbb821d3f6d5","Type":"ContainerStarted","Data":"66b7496e33f41a6800ca712c00b0a7967ff7539d57fe57eea77cbb80f482877f"}
Oct 31 17:36:37.651339 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:37.651255    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-cf0a86-7bb4c5d6c5-8p8r5"
Oct 31 17:36:37.695424 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:37.695378    4276 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-cf0a86-7bb4c5d6c5-8p8r5" podStartSLOduration=2.6953642970000002 podStartE2EDuration="2.695364297s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 17:36:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 17:36:37.693200354 +0000 UTC m=+560.490945282" watchObservedRunningTime="2026-10-31 17:36:37.695364297 +0000 UTC m=+560.493109225"
Oct 31 17:36:37.704062 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:37.703999    4276 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w6-d80080-59b65f4495-h6gz8"]
Oct 31 17:36:37.708646 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:37.708625    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-d80080-59b65f4495-h6gz8"
Oct 31 17:36:37.724535 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:37.724506    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w6-d80080-dockercfg-fwh59\""
Oct 31 17:36:37.724810 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:37.724789    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w6-d80080-59b65f4495-h6gz8"]
Oct 31 17:36:37.748872 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:37.748845    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-d80080-869f895d55-s62hv"]
Oct 31 17:36:37.751292 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:36:37.751265    4276 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod175639e5_138e_428e_bb53_7fe484543496.slice/crio-6c2a417b1d597e443b6fd947fbcb2525e4a016d06869c5062e2b501da39dbe7c WatchSource:0}: Error finding container 6c2a417b1d597e443b6fd947fbcb2525e4a016d06869c5062e2b501da39dbe7c: Status 404 returned error can't find the container with id 6c2a417b1d597e443b6fd947fbcb2525e4a016d06869c5062e2b501da39dbe7c
Oct 31 17:36:37.777654 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:37.777632    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jw7mz\" (UniqueName: \"kubernetes.io/projected/6e4109e9-1a59-48c7-9143-e14a01758994-kube-api-access-jw7mz\") pod \"payload-pre-processing-e2e-worker-w6-d80080-59b65f4495-h6gz8\" (UID: \"6e4109e9-1a59-48c7-9143-e14a01758994\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-d80080-59b65f4495-h6gz8"
Oct 31 17:36:37.777751 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:37.777665    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6e4109e9-1a59-48c7-9143-e14a01758994-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w6-d80080-59b65f4495-h6gz8\" (UID: \"6e4109e9-1a59-48c7-9143-e14a01758994\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-d80080-59b65f4495-h6gz8"
Oct 31 17:36:37.819751 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:37.819517    4276 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w6-d80080\""
Oct 31 17:36:37.878435 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:37.878401    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jw7mz\" (UniqueName: \"kubernetes.io/projected/6e4109e9-1a59-48c7-9143-e14a01758994-kube-api-access-jw7mz\") pod \"payload-pre-processing-e2e-worker-w6-d80080-59b65f4495-h6gz8\" (UID: \"6e4109e9-1a59-48c7-9143-e14a01758994\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-d80080-59b65f4495-h6gz8"
Oct 31 17:36:37.878435 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:37.878467    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6e4109e9-1a59-48c7-9143-e14a01758994-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w6-d80080-59b65f4495-h6gz8\" (UID: \"6e4109e9-1a59-48c7-9143-e14a01758994\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-d80080-59b65f4495-h6gz8"
Oct 31 17:36:37.879167 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:37.879146    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6e4109e9-1a59-48c7-9143-e14a01758994-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w6-d80080-59b65f4495-h6gz8\" (UID: \"6e4109e9-1a59-48c7-9143-e14a01758994\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-d80080-59b65f4495-h6gz8"
Oct 31 17:36:37.899345 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:37.899315    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jw7mz\" (UniqueName: \"kubernetes.io/projected/6e4109e9-1a59-48c7-9143-e14a01758994-kube-api-access-jw7mz\") pod \"payload-pre-processing-e2e-worker-w6-d80080-59b65f4495-h6gz8\" (UID: \"6e4109e9-1a59-48c7-9143-e14a01758994\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-d80080-59b65f4495-h6gz8"
Oct 31 17:36:38.022955 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:38.022861    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-d80080-59b65f4495-h6gz8"
Oct 31 17:36:38.321363 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:38.321333    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w6-d80080-59b65f4495-h6gz8"]
Oct 31 17:36:38.323460 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:36:38.323422    4276 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e4109e9_1a59_48c7_9143_e14a01758994.slice/crio-eeb05c6ad1920207575b6847a78388cb26783dda8e423046763f70217f81ee85 WatchSource:0}: Error finding container eeb05c6ad1920207575b6847a78388cb26783dda8e423046763f70217f81ee85: Status 404 returned error can't find the container with id eeb05c6ad1920207575b6847a78388cb26783dda8e423046763f70217f81ee85
Oct 31 17:36:38.657648 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:38.657549    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-d80080-59b65f4495-h6gz8" event={"ID":"6e4109e9-1a59-48c7-9143-e14a01758994","Type":"ContainerStarted","Data":"4a0dc01c7432bd39e6e6f75d3eac48c4c6452840954242f79abbd2bfb8d4600e"}
Oct 31 17:36:38.657648 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:38.657617    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-d80080-59b65f4495-h6gz8" event={"ID":"6e4109e9-1a59-48c7-9143-e14a01758994","Type":"ContainerStarted","Data":"eeb05c6ad1920207575b6847a78388cb26783dda8e423046763f70217f81ee85"}
Oct 31 17:36:38.657843 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:38.657685    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-d80080-59b65f4495-h6gz8"
Oct 31 17:36:38.659351 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:38.659324    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-d80080-869f895d55-s62hv" event={"ID":"175639e5-138e-428e-bb53-7fe484543496","Type":"ContainerStarted","Data":"9f14cf231e12847b2388f04f42d866775667f02335e3fc4c7e81924a5b3a16d9"}
Oct 31 17:36:38.659499 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:38.659368    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-d80080-869f895d55-s62hv" event={"ID":"175639e5-138e-428e-bb53-7fe484543496","Type":"ContainerStarted","Data":"6c2a417b1d597e443b6fd947fbcb2525e4a016d06869c5062e2b501da39dbe7c"}
Oct 31 17:36:38.659499 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:38.659480    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-d80080-869f895d55-s62hv"
Oct 31 17:36:38.701829 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:38.700793    4276 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-d80080-59b65f4495-h6gz8" podStartSLOduration=1.700777506 podStartE2EDuration="1.700777506s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 17:36:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 17:36:38.697591706 +0000 UTC m=+561.495336633" watchObservedRunningTime="2026-10-31 17:36:38.700777506 +0000 UTC m=+561.498522435"
Oct 31 17:36:38.743104 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:38.743061    4276 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-d80080-869f895d55-s62hv" podStartSLOduration=1.421811442 podStartE2EDuration="1.74304829s" totalImagesPullingTime="321.236848ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 17:36:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 17:36:38.7421655 +0000 UTC m=+561.539910428" watchObservedRunningTime="2026-10-31 17:36:38.74304829 +0000 UTC m=+561.540793218"
Oct 31 17:36:39.025907 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:39.025751    4276 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-cf0a86-86c86d974b-hkgh2"]
Oct 31 17:36:39.069846 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:39.030238    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-cf0a86-86c86d974b-hkgh2"
Oct 31 17:36:39.069846 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:39.047213    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w3-cf0a86-serving-cert\""
Oct 31 17:36:39.069846 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:39.050366    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-cf0a86-86c86d974b-hkgh2"]
Oct 31 17:36:39.103571 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:39.103543    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-skb6n\" (UniqueName: \"kubernetes.io/projected/d5dd5b19-e930-4afa-abea-db914b9f7e89-kube-api-access-skb6n\") pod \"maas-api-e2e-worker-w3-cf0a86-86c86d974b-hkgh2\" (UID: \"d5dd5b19-e930-4afa-abea-db914b9f7e89\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-cf0a86-86c86d974b-hkgh2"
Oct 31 17:36:39.103742 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:39.103588    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d5dd5b19-e930-4afa-abea-db914b9f7e89-maas-api-tls\") pod \"maas-api-e2e-worker-w3-cf0a86-86c86d974b-hkgh2\" (UID: \"d5dd5b19-e930-4afa-abea-db914b9f7e89\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-cf0a86-86c86d974b-hkgh2"
Oct 31 17:36:39.103742 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:39.103709    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d5dd5b19-e930-4afa-abea-db914b9f7e89-metrics-certs\") pod \"maas-api-e2e-worker-w3-cf0a86-86c86d974b-hkgh2\" (UID: \"d5dd5b19-e930-4afa-abea-db914b9f7e89\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-cf0a86-86c86d974b-hkgh2"
Oct 31 17:36:39.103874 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:39.103786    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d5dd5b19-e930-4afa-abea-db914b9f7e89-openshift-service-ca\") pod \"maas-api-e2e-worker-w3-cf0a86-86c86d974b-hkgh2\" (UID: \"d5dd5b19-e930-4afa-abea-db914b9f7e89\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-cf0a86-86c86d974b-hkgh2"
Oct 31 17:36:39.225465 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:39.216961    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d5dd5b19-e930-4afa-abea-db914b9f7e89-openshift-service-ca\") pod \"maas-api-e2e-worker-w3-cf0a86-86c86d974b-hkgh2\" (UID: \"d5dd5b19-e930-4afa-abea-db914b9f7e89\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-cf0a86-86c86d974b-hkgh2"
Oct 31 17:36:39.225465 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:39.217095    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-skb6n\" (UniqueName: \"kubernetes.io/projected/d5dd5b19-e930-4afa-abea-db914b9f7e89-kube-api-access-skb6n\") pod \"maas-api-e2e-worker-w3-cf0a86-86c86d974b-hkgh2\" (UID: \"d5dd5b19-e930-4afa-abea-db914b9f7e89\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-cf0a86-86c86d974b-hkgh2"
Oct 31 17:36:39.225465 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:39.217151    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d5dd5b19-e930-4afa-abea-db914b9f7e89-maas-api-tls\") pod \"maas-api-e2e-worker-w3-cf0a86-86c86d974b-hkgh2\" (UID: \"d5dd5b19-e930-4afa-abea-db914b9f7e89\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-cf0a86-86c86d974b-hkgh2"
Oct 31 17:36:39.225465 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:39.217231    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d5dd5b19-e930-4afa-abea-db914b9f7e89-metrics-certs\") pod \"maas-api-e2e-worker-w3-cf0a86-86c86d974b-hkgh2\" (UID: \"d5dd5b19-e930-4afa-abea-db914b9f7e89\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-cf0a86-86c86d974b-hkgh2"
Oct 31 17:36:39.225465 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:39.222118    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d5dd5b19-e930-4afa-abea-db914b9f7e89-openshift-service-ca\") pod \"maas-api-e2e-worker-w3-cf0a86-86c86d974b-hkgh2\" (UID: \"d5dd5b19-e930-4afa-abea-db914b9f7e89\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-cf0a86-86c86d974b-hkgh2"
Oct 31 17:36:39.225465 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:39.225207    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d5dd5b19-e930-4afa-abea-db914b9f7e89-maas-api-tls\") pod \"maas-api-e2e-worker-w3-cf0a86-86c86d974b-hkgh2\" (UID: \"d5dd5b19-e930-4afa-abea-db914b9f7e89\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-cf0a86-86c86d974b-hkgh2"
Oct 31 17:36:39.227111 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:39.226178    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d5dd5b19-e930-4afa-abea-db914b9f7e89-metrics-certs\") pod \"maas-api-e2e-worker-w3-cf0a86-86c86d974b-hkgh2\" (UID: \"d5dd5b19-e930-4afa-abea-db914b9f7e89\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-cf0a86-86c86d974b-hkgh2"
Oct 31 17:36:39.238086 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:39.238060    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-skb6n\" (UniqueName: \"kubernetes.io/projected/d5dd5b19-e930-4afa-abea-db914b9f7e89-kube-api-access-skb6n\") pod \"maas-api-e2e-worker-w3-cf0a86-86c86d974b-hkgh2\" (UID: \"d5dd5b19-e930-4afa-abea-db914b9f7e89\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-cf0a86-86c86d974b-hkgh2"
Oct 31 17:36:39.343263 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:39.343239    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-cf0a86-86c86d974b-hkgh2"
Oct 31 17:36:39.619730 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:39.619646    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:36:39.671966 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:39.671945    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-cf0a86-86c86d974b-hkgh2"]
Oct 31 17:36:39.673427 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:36:39.673399    4276 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5dd5b19_e930_4afa_abea_db914b9f7e89.slice/crio-66503d2e55b63126fd0bbc4e38537377362e4e52228f06c247036f35fba1683f WatchSource:0}: Error finding container 66503d2e55b63126fd0bbc4e38537377362e4e52228f06c247036f35fba1683f: Status 404 returned error can't find the container with id 66503d2e55b63126fd0bbc4e38537377362e4e52228f06c247036f35fba1683f
Oct 31 17:36:40.670800 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:40.670768    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-cf0a86-86c86d974b-hkgh2" event={"ID":"d5dd5b19-e930-4afa-abea-db914b9f7e89","Type":"ContainerStarted","Data":"b06e1d7ad773d7d8c5bad3f3c6087145a9d0b7fbdae52d69468f818ee0f48a59"}
Oct 31 17:36:40.671206 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:40.670818    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-cf0a86-86c86d974b-hkgh2" event={"ID":"d5dd5b19-e930-4afa-abea-db914b9f7e89","Type":"ContainerStarted","Data":"66503d2e55b63126fd0bbc4e38537377362e4e52228f06c247036f35fba1683f"}
Oct 31 17:36:40.671206 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:40.670867    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-cf0a86-86c86d974b-hkgh2"
Oct 31 17:36:40.713026 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:40.712963    4276 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-cf0a86-86c86d974b-hkgh2" podStartSLOduration=1.385263772 podStartE2EDuration="1.712946467s" totalImagesPullingTime="327.682695ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 17:36:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 17:36:40.711741699 +0000 UTC m=+563.509486628" watchObservedRunningTime="2026-10-31 17:36:40.712946467 +0000 UTC m=+563.510691398"
Oct 31 17:36:40.978101 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:40.978016    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:36:41.102888 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:41.102858    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:36:41.124481 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:41.124452    4276 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w5-b78c31-6b5448c9d7-rbhfz"]
Oct 31 17:36:41.127956 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:41.127939    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-b78c31-6b5448c9d7-rbhfz"
Oct 31 17:36:41.144515 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:41.144422    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w5-b78c31-dockercfg-pzbjt\""
Oct 31 17:36:41.144515 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:41.144431    4276 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w5-b78c31\""
Oct 31 17:36:41.146343 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:41.146322    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w5-b78c31-6b5448c9d7-rbhfz"]
Oct 31 17:36:41.239042 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:41.238935    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qk78z\" (UniqueName: \"kubernetes.io/projected/f7894ac3-7f53-4a3f-8d20-17359e24f69e-kube-api-access-qk78z\") pod \"payload-pre-processing-e2e-worker-w5-b78c31-6b5448c9d7-rbhfz\" (UID: \"f7894ac3-7f53-4a3f-8d20-17359e24f69e\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-b78c31-6b5448c9d7-rbhfz"
Oct 31 17:36:41.239230 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:41.239092    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f7894ac3-7f53-4a3f-8d20-17359e24f69e-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w5-b78c31-6b5448c9d7-rbhfz\" (UID: \"f7894ac3-7f53-4a3f-8d20-17359e24f69e\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-b78c31-6b5448c9d7-rbhfz"
Oct 31 17:36:41.340189 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:41.340154    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qk78z\" (UniqueName: \"kubernetes.io/projected/f7894ac3-7f53-4a3f-8d20-17359e24f69e-kube-api-access-qk78z\") pod \"payload-pre-processing-e2e-worker-w5-b78c31-6b5448c9d7-rbhfz\" (UID: \"f7894ac3-7f53-4a3f-8d20-17359e24f69e\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-b78c31-6b5448c9d7-rbhfz"
Oct 31 17:36:41.340384 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:41.340238    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f7894ac3-7f53-4a3f-8d20-17359e24f69e-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w5-b78c31-6b5448c9d7-rbhfz\" (UID: \"f7894ac3-7f53-4a3f-8d20-17359e24f69e\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-b78c31-6b5448c9d7-rbhfz"
Oct 31 17:36:41.340913 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:41.340892    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f7894ac3-7f53-4a3f-8d20-17359e24f69e-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w5-b78c31-6b5448c9d7-rbhfz\" (UID: \"f7894ac3-7f53-4a3f-8d20-17359e24f69e\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-b78c31-6b5448c9d7-rbhfz"
Oct 31 17:36:41.364055 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:41.364022    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qk78z\" (UniqueName: \"kubernetes.io/projected/f7894ac3-7f53-4a3f-8d20-17359e24f69e-kube-api-access-qk78z\") pod \"payload-pre-processing-e2e-worker-w5-b78c31-6b5448c9d7-rbhfz\" (UID: \"f7894ac3-7f53-4a3f-8d20-17359e24f69e\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-b78c31-6b5448c9d7-rbhfz"
Oct 31 17:36:41.440662 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:41.440628    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-b78c31-6b5448c9d7-rbhfz"
Oct 31 17:36:41.739423 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:41.739397    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w5-b78c31-6b5448c9d7-rbhfz"]
Oct 31 17:36:41.740634 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:36:41.740606    4276 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7894ac3_7f53_4a3f_8d20_17359e24f69e.slice/crio-fe2323db58dbc676e651f9d07ce0d826eee4ced26901b1fa88931e8bb5eb551e WatchSource:0}: Error finding container fe2323db58dbc676e651f9d07ce0d826eee4ced26901b1fa88931e8bb5eb551e: Status 404 returned error can't find the container with id fe2323db58dbc676e651f9d07ce0d826eee4ced26901b1fa88931e8bb5eb551e
Oct 31 17:36:42.234525 ip-10-0-129-127 kubenswrapper[4276]: E1031 17:36:42.234491    4276 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tg5qk" containerName="extractor"
Oct 31 17:36:42.651724 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:42.651691    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:36:42.658134 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:42.658109    4276 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-1c0bfd-6569c7d6f-l29ff"]
Oct 31 17:36:42.662385 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:42.662365    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-1c0bfd-6569c7d6f-l29ff"
Oct 31 17:36:42.689186 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:42.689158    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-b78c31-6b5448c9d7-rbhfz" event={"ID":"f7894ac3-7f53-4a3f-8d20-17359e24f69e","Type":"ContainerStarted","Data":"9d88d61861ce7bf6e5f3c5befa55815e5e0b74178a89eccd2406911943fbe6b5"}
Oct 31 17:36:42.689351 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:42.689204    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-b78c31-6b5448c9d7-rbhfz" event={"ID":"f7894ac3-7f53-4a3f-8d20-17359e24f69e","Type":"ContainerStarted","Data":"fe2323db58dbc676e651f9d07ce0d826eee4ced26901b1fa88931e8bb5eb551e"}
Oct 31 17:36:42.689738 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:42.689722    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-b78c31-6b5448c9d7-rbhfz"
Oct 31 17:36:42.690227 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:42.690212    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w4-1c0bfd-serving-cert\""
Oct 31 17:36:42.692961 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:42.692939    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-1c0bfd-6569c7d6f-l29ff"]
Oct 31 17:36:42.753873 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:42.753814    4276 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-b78c31-6b5448c9d7-rbhfz" podStartSLOduration=1.753802528 podStartE2EDuration="1.753802528s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 17:36:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 17:36:42.752474988 +0000 UTC m=+565.550219918" watchObservedRunningTime="2026-10-31 17:36:42.753802528 +0000 UTC m=+565.551547458"
Oct 31 17:36:42.753873 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:42.753867    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vsfjr\" (UniqueName: \"kubernetes.io/projected/95755676-9f02-4b03-a7e5-f6c8ab4fa965-kube-api-access-vsfjr\") pod \"maas-api-e2e-worker-w4-1c0bfd-6569c7d6f-l29ff\" (UID: \"95755676-9f02-4b03-a7e5-f6c8ab4fa965\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-1c0bfd-6569c7d6f-l29ff"
Oct 31 17:36:42.754391 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:42.754067    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/95755676-9f02-4b03-a7e5-f6c8ab4fa965-maas-api-tls\") pod \"maas-api-e2e-worker-w4-1c0bfd-6569c7d6f-l29ff\" (UID: \"95755676-9f02-4b03-a7e5-f6c8ab4fa965\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-1c0bfd-6569c7d6f-l29ff"
Oct 31 17:36:42.754391 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:42.754122    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/95755676-9f02-4b03-a7e5-f6c8ab4fa965-metrics-certs\") pod \"maas-api-e2e-worker-w4-1c0bfd-6569c7d6f-l29ff\" (UID: \"95755676-9f02-4b03-a7e5-f6c8ab4fa965\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-1c0bfd-6569c7d6f-l29ff"
Oct 31 17:36:42.754391 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:42.754165    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/95755676-9f02-4b03-a7e5-f6c8ab4fa965-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-1c0bfd-6569c7d6f-l29ff\" (UID: \"95755676-9f02-4b03-a7e5-f6c8ab4fa965\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-1c0bfd-6569c7d6f-l29ff"
Oct 31 17:36:42.772003 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:42.771974    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:36:42.855503 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:42.855467    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vsfjr\" (UniqueName: \"kubernetes.io/projected/95755676-9f02-4b03-a7e5-f6c8ab4fa965-kube-api-access-vsfjr\") pod \"maas-api-e2e-worker-w4-1c0bfd-6569c7d6f-l29ff\" (UID: \"95755676-9f02-4b03-a7e5-f6c8ab4fa965\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-1c0bfd-6569c7d6f-l29ff"
Oct 31 17:36:42.855688 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:42.855604    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/95755676-9f02-4b03-a7e5-f6c8ab4fa965-maas-api-tls\") pod \"maas-api-e2e-worker-w4-1c0bfd-6569c7d6f-l29ff\" (UID: \"95755676-9f02-4b03-a7e5-f6c8ab4fa965\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-1c0bfd-6569c7d6f-l29ff"
Oct 31 17:36:42.855688 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:42.855639    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/95755676-9f02-4b03-a7e5-f6c8ab4fa965-metrics-certs\") pod \"maas-api-e2e-worker-w4-1c0bfd-6569c7d6f-l29ff\" (UID: \"95755676-9f02-4b03-a7e5-f6c8ab4fa965\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-1c0bfd-6569c7d6f-l29ff"
Oct 31 17:36:42.855688 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:42.855676    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/95755676-9f02-4b03-a7e5-f6c8ab4fa965-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-1c0bfd-6569c7d6f-l29ff\" (UID: \"95755676-9f02-4b03-a7e5-f6c8ab4fa965\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-1c0bfd-6569c7d6f-l29ff"
Oct 31 17:36:42.856530 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:42.856499    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/95755676-9f02-4b03-a7e5-f6c8ab4fa965-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-1c0bfd-6569c7d6f-l29ff\" (UID: \"95755676-9f02-4b03-a7e5-f6c8ab4fa965\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-1c0bfd-6569c7d6f-l29ff"
Oct 31 17:36:42.858226 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:42.858197    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/95755676-9f02-4b03-a7e5-f6c8ab4fa965-metrics-certs\") pod \"maas-api-e2e-worker-w4-1c0bfd-6569c7d6f-l29ff\" (UID: \"95755676-9f02-4b03-a7e5-f6c8ab4fa965\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-1c0bfd-6569c7d6f-l29ff"
Oct 31 17:36:42.858351 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:42.858203    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/95755676-9f02-4b03-a7e5-f6c8ab4fa965-maas-api-tls\") pod \"maas-api-e2e-worker-w4-1c0bfd-6569c7d6f-l29ff\" (UID: \"95755676-9f02-4b03-a7e5-f6c8ab4fa965\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-1c0bfd-6569c7d6f-l29ff"
Oct 31 17:36:42.875635 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:42.875608    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vsfjr\" (UniqueName: \"kubernetes.io/projected/95755676-9f02-4b03-a7e5-f6c8ab4fa965-kube-api-access-vsfjr\") pod \"maas-api-e2e-worker-w4-1c0bfd-6569c7d6f-l29ff\" (UID: \"95755676-9f02-4b03-a7e5-f6c8ab4fa965\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-1c0bfd-6569c7d6f-l29ff"
Oct 31 17:36:42.889664 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:42.889638    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:36:42.978170 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:42.978071    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-1c0bfd-6569c7d6f-l29ff"
Oct 31 17:36:43.009983 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:43.009956    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:36:43.275957 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:43.275931    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-1c0bfd-6569c7d6f-l29ff"]
Oct 31 17:36:43.276787 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:36:43.276760    4276 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95755676_9f02_4b03_a7e5_f6c8ab4fa965.slice/crio-073e0d34c64ac886838643937844ea2d1afcb25d2319af383326e4e0c00b22f5 WatchSource:0}: Error finding container 073e0d34c64ac886838643937844ea2d1afcb25d2319af383326e4e0c00b22f5: Status 404 returned error can't find the container with id 073e0d34c64ac886838643937844ea2d1afcb25d2319af383326e4e0c00b22f5
Oct 31 17:36:43.588001 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:43.587970    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:36:43.696127 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:43.696095    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-1c0bfd-6569c7d6f-l29ff" event={"ID":"95755676-9f02-4b03-a7e5-f6c8ab4fa965","Type":"ContainerStarted","Data":"073e0d34c64ac886838643937844ea2d1afcb25d2319af383326e4e0c00b22f5"}
Oct 31 17:36:43.698118 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:43.698091    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:36:44.420338 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:44.420300    4276 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-0e59ca-77546c9db5-b8b5g"]
Oct 31 17:36:44.422287 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:44.422260    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:36:44.425871 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:44.425852    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-0e59ca-77546c9db5-b8b5g"
Oct 31 17:36:44.440406 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:44.440383    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-0e59ca-77546c9db5-b8b5g"]
Oct 31 17:36:44.451903 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:44.451879    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w1-0e59ca-serving-cert\""
Oct 31 17:36:44.478833 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:44.478793    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ca2c3199-efa6-450a-8c46-343e39810896-metrics-certs\") pod \"maas-api-e2e-worker-w1-0e59ca-77546c9db5-b8b5g\" (UID: \"ca2c3199-efa6-450a-8c46-343e39810896\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-0e59ca-77546c9db5-b8b5g"
Oct 31 17:36:44.478989 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:44.478885    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca2c3199-efa6-450a-8c46-343e39810896-openshift-service-ca\") pod \"maas-api-e2e-worker-w1-0e59ca-77546c9db5-b8b5g\" (UID: \"ca2c3199-efa6-450a-8c46-343e39810896\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-0e59ca-77546c9db5-b8b5g"
Oct 31 17:36:44.478989 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:44.478960    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xgvc4\" (UniqueName: \"kubernetes.io/projected/ca2c3199-efa6-450a-8c46-343e39810896-kube-api-access-xgvc4\") pod \"maas-api-e2e-worker-w1-0e59ca-77546c9db5-b8b5g\" (UID: \"ca2c3199-efa6-450a-8c46-343e39810896\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-0e59ca-77546c9db5-b8b5g"
Oct 31 17:36:44.479124 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:44.478989    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ca2c3199-efa6-450a-8c46-343e39810896-maas-api-tls\") pod \"maas-api-e2e-worker-w1-0e59ca-77546c9db5-b8b5g\" (UID: \"ca2c3199-efa6-450a-8c46-343e39810896\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-0e59ca-77546c9db5-b8b5g"
Oct 31 17:36:44.541374 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:44.541336    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:36:44.579829 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:44.579789    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ca2c3199-efa6-450a-8c46-343e39810896-metrics-certs\") pod \"maas-api-e2e-worker-w1-0e59ca-77546c9db5-b8b5g\" (UID: \"ca2c3199-efa6-450a-8c46-343e39810896\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-0e59ca-77546c9db5-b8b5g"
Oct 31 17:36:44.579829 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:44.579827    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca2c3199-efa6-450a-8c46-343e39810896-openshift-service-ca\") pod \"maas-api-e2e-worker-w1-0e59ca-77546c9db5-b8b5g\" (UID: \"ca2c3199-efa6-450a-8c46-343e39810896\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-0e59ca-77546c9db5-b8b5g"
Oct 31 17:36:44.580069 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:44.579846    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xgvc4\" (UniqueName: \"kubernetes.io/projected/ca2c3199-efa6-450a-8c46-343e39810896-kube-api-access-xgvc4\") pod \"maas-api-e2e-worker-w1-0e59ca-77546c9db5-b8b5g\" (UID: \"ca2c3199-efa6-450a-8c46-343e39810896\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-0e59ca-77546c9db5-b8b5g"
Oct 31 17:36:44.580069 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:44.579873    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ca2c3199-efa6-450a-8c46-343e39810896-maas-api-tls\") pod \"maas-api-e2e-worker-w1-0e59ca-77546c9db5-b8b5g\" (UID: \"ca2c3199-efa6-450a-8c46-343e39810896\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-0e59ca-77546c9db5-b8b5g"
Oct 31 17:36:44.580069 ip-10-0-129-127 kubenswrapper[4276]: E1031 17:36:44.579974    4276 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w1-0e59ca-serving-cert: secret "maas-api-e2e-worker-w1-0e59ca-serving-cert" not found
Oct 31 17:36:44.580069 ip-10-0-129-127 kubenswrapper[4276]: E1031 17:36:44.580041    4276 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ca2c3199-efa6-450a-8c46-343e39810896-maas-api-tls podName:ca2c3199-efa6-450a-8c46-343e39810896 nodeName:}" failed. No retries permitted until 2026-10-31 17:36:45.080016191 +0000 UTC m=+567.877761104 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/ca2c3199-efa6-450a-8c46-343e39810896-maas-api-tls") pod "maas-api-e2e-worker-w1-0e59ca-77546c9db5-b8b5g" (UID: "ca2c3199-efa6-450a-8c46-343e39810896") : secret "maas-api-e2e-worker-w1-0e59ca-serving-cert" not found
Oct 31 17:36:44.580485 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:44.580433    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca2c3199-efa6-450a-8c46-343e39810896-openshift-service-ca\") pod \"maas-api-e2e-worker-w1-0e59ca-77546c9db5-b8b5g\" (UID: \"ca2c3199-efa6-450a-8c46-343e39810896\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-0e59ca-77546c9db5-b8b5g"
Oct 31 17:36:44.582705 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:44.582683    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ca2c3199-efa6-450a-8c46-343e39810896-metrics-certs\") pod \"maas-api-e2e-worker-w1-0e59ca-77546c9db5-b8b5g\" (UID: \"ca2c3199-efa6-450a-8c46-343e39810896\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-0e59ca-77546c9db5-b8b5g"
Oct 31 17:36:44.610336 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:44.610310    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xgvc4\" (UniqueName: \"kubernetes.io/projected/ca2c3199-efa6-450a-8c46-343e39810896-kube-api-access-xgvc4\") pod \"maas-api-e2e-worker-w1-0e59ca-77546c9db5-b8b5g\" (UID: \"ca2c3199-efa6-450a-8c46-343e39810896\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-0e59ca-77546c9db5-b8b5g"
Oct 31 17:36:44.658118 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:44.658089    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:36:44.670030 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:44.670005    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-d80080-869f895d55-s62hv"
Oct 31 17:36:44.771869 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:44.771770    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-1c0bfd-6569c7d6f-l29ff" event={"ID":"95755676-9f02-4b03-a7e5-f6c8ab4fa965","Type":"ContainerStarted","Data":"b1508c7bbf6311fa84ae2faa759301a8f5a726bee2c9b402767cd3fd5ef15bf1"}
Oct 31 17:36:44.772143 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:44.772123    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-1c0bfd-6569c7d6f-l29ff"
Oct 31 17:36:44.781363 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:44.781331    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:36:44.817795 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:44.817749    4276 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-1c0bfd-6569c7d6f-l29ff" podStartSLOduration=2.423151419 podStartE2EDuration="2.817735329s" totalImagesPullingTime="394.58391ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 17:36:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 17:36:44.816524729 +0000 UTC m=+567.614269656" watchObservedRunningTime="2026-10-31 17:36:44.817735329 +0000 UTC m=+567.615480257"
Oct 31 17:36:45.085156 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:45.085116    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ca2c3199-efa6-450a-8c46-343e39810896-maas-api-tls\") pod \"maas-api-e2e-worker-w1-0e59ca-77546c9db5-b8b5g\" (UID: \"ca2c3199-efa6-450a-8c46-343e39810896\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-0e59ca-77546c9db5-b8b5g"
Oct 31 17:36:45.087498 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:45.087470    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ca2c3199-efa6-450a-8c46-343e39810896-maas-api-tls\") pod \"maas-api-e2e-worker-w1-0e59ca-77546c9db5-b8b5g\" (UID: \"ca2c3199-efa6-450a-8c46-343e39810896\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-0e59ca-77546c9db5-b8b5g"
Oct 31 17:36:45.342253 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:45.342167    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-0e59ca-77546c9db5-b8b5g"
Oct 31 17:36:45.642006 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:45.641984    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-0e59ca-77546c9db5-b8b5g"]
Oct 31 17:36:45.643869 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:36:45.643831    4276 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca2c3199_efa6_450a_8c46_343e39810896.slice/crio-a3ec4129734520d4222c317c30fb76919db56d61b26949f87723c35e00661a44 WatchSource:0}: Error finding container a3ec4129734520d4222c317c30fb76919db56d61b26949f87723c35e00661a44: Status 404 returned error can't find the container with id a3ec4129734520d4222c317c30fb76919db56d61b26949f87723c35e00661a44
Oct 31 17:36:45.777615 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:45.777579    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-0e59ca-77546c9db5-b8b5g" event={"ID":"ca2c3199-efa6-450a-8c46-343e39810896","Type":"ContainerStarted","Data":"a3ec4129734520d4222c317c30fb76919db56d61b26949f87723c35e00661a44"}
Oct 31 17:36:45.943776 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:45.943686    4276 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-5c446c-56f77ff564-dd7dx"]
Oct 31 17:36:45.948719 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:45.948702    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-5c446c-56f77ff564-dd7dx"
Oct 31 17:36:45.967030 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:45.967000    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-5c446c-56f77ff564-dd7dx"]
Oct 31 17:36:45.977864 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:45.977837    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w2-5c446c-serving-cert\""
Oct 31 17:36:45.994927 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:45.994898    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/67250413-5beb-44da-9925-71b1735d723b-openshift-service-ca\") pod \"maas-api-e2e-worker-w2-5c446c-56f77ff564-dd7dx\" (UID: \"67250413-5beb-44da-9925-71b1735d723b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-5c446c-56f77ff564-dd7dx"
Oct 31 17:36:45.995079 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:45.995063    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/67250413-5beb-44da-9925-71b1735d723b-maas-api-tls\") pod \"maas-api-e2e-worker-w2-5c446c-56f77ff564-dd7dx\" (UID: \"67250413-5beb-44da-9925-71b1735d723b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-5c446c-56f77ff564-dd7dx"
Oct 31 17:36:45.995146 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:45.995109    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/67250413-5beb-44da-9925-71b1735d723b-metrics-certs\") pod \"maas-api-e2e-worker-w2-5c446c-56f77ff564-dd7dx\" (UID: \"67250413-5beb-44da-9925-71b1735d723b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-5c446c-56f77ff564-dd7dx"
Oct 31 17:36:45.995203 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:45.995139    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zpjt2\" (UniqueName: \"kubernetes.io/projected/67250413-5beb-44da-9925-71b1735d723b-kube-api-access-zpjt2\") pod \"maas-api-e2e-worker-w2-5c446c-56f77ff564-dd7dx\" (UID: \"67250413-5beb-44da-9925-71b1735d723b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-5c446c-56f77ff564-dd7dx"
Oct 31 17:36:46.053254 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:46.053224    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:36:46.096018 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:46.095989    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/67250413-5beb-44da-9925-71b1735d723b-maas-api-tls\") pod \"maas-api-e2e-worker-w2-5c446c-56f77ff564-dd7dx\" (UID: \"67250413-5beb-44da-9925-71b1735d723b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-5c446c-56f77ff564-dd7dx"
Oct 31 17:36:46.096173 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:46.096034    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/67250413-5beb-44da-9925-71b1735d723b-metrics-certs\") pod \"maas-api-e2e-worker-w2-5c446c-56f77ff564-dd7dx\" (UID: \"67250413-5beb-44da-9925-71b1735d723b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-5c446c-56f77ff564-dd7dx"
Oct 31 17:36:46.096173 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:46.096056    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zpjt2\" (UniqueName: \"kubernetes.io/projected/67250413-5beb-44da-9925-71b1735d723b-kube-api-access-zpjt2\") pod \"maas-api-e2e-worker-w2-5c446c-56f77ff564-dd7dx\" (UID: \"67250413-5beb-44da-9925-71b1735d723b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-5c446c-56f77ff564-dd7dx"
Oct 31 17:36:46.096293 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:46.096177    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/67250413-5beb-44da-9925-71b1735d723b-openshift-service-ca\") pod \"maas-api-e2e-worker-w2-5c446c-56f77ff564-dd7dx\" (UID: \"67250413-5beb-44da-9925-71b1735d723b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-5c446c-56f77ff564-dd7dx"
Oct 31 17:36:46.096810 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:46.096790    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/67250413-5beb-44da-9925-71b1735d723b-openshift-service-ca\") pod \"maas-api-e2e-worker-w2-5c446c-56f77ff564-dd7dx\" (UID: \"67250413-5beb-44da-9925-71b1735d723b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-5c446c-56f77ff564-dd7dx"
Oct 31 17:36:46.098566 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:46.098542    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/67250413-5beb-44da-9925-71b1735d723b-metrics-certs\") pod \"maas-api-e2e-worker-w2-5c446c-56f77ff564-dd7dx\" (UID: \"67250413-5beb-44da-9925-71b1735d723b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-5c446c-56f77ff564-dd7dx"
Oct 31 17:36:46.098654 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:46.098592    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/67250413-5beb-44da-9925-71b1735d723b-maas-api-tls\") pod \"maas-api-e2e-worker-w2-5c446c-56f77ff564-dd7dx\" (UID: \"67250413-5beb-44da-9925-71b1735d723b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-5c446c-56f77ff564-dd7dx"
Oct 31 17:36:46.121930 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:46.121904    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zpjt2\" (UniqueName: \"kubernetes.io/projected/67250413-5beb-44da-9925-71b1735d723b-kube-api-access-zpjt2\") pod \"maas-api-e2e-worker-w2-5c446c-56f77ff564-dd7dx\" (UID: \"67250413-5beb-44da-9925-71b1735d723b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-5c446c-56f77ff564-dd7dx"
Oct 31 17:36:46.180174 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:46.180147    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:36:46.266246 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:46.264979    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-5c446c-56f77ff564-dd7dx"
Oct 31 17:36:46.324471 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:46.323019    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:36:46.616007 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:46.615980    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-5c446c-56f77ff564-dd7dx"]
Oct 31 17:36:46.617499 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:36:46.617472    4276 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67250413_5beb_44da_9925_71b1735d723b.slice/crio-7a3d5ec74400f3bbfb541d4986fe636adf3255e27eb600316dac2994eef9bdaa WatchSource:0}: Error finding container 7a3d5ec74400f3bbfb541d4986fe636adf3255e27eb600316dac2994eef9bdaa: Status 404 returned error can't find the container with id 7a3d5ec74400f3bbfb541d4986fe636adf3255e27eb600316dac2994eef9bdaa
Oct 31 17:36:46.680449 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:46.680426    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-cf0a86-86c86d974b-hkgh2"
Oct 31 17:36:46.789085 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:46.789052    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-5c446c-56f77ff564-dd7dx" event={"ID":"67250413-5beb-44da-9925-71b1735d723b","Type":"ContainerStarted","Data":"7a3d5ec74400f3bbfb541d4986fe636adf3255e27eb600316dac2994eef9bdaa"}
Oct 31 17:36:46.791037 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:46.790995    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-0e59ca-77546c9db5-b8b5g" event={"ID":"ca2c3199-efa6-450a-8c46-343e39810896","Type":"ContainerStarted","Data":"d825eea5b72ce9edfb3548fad4864d78c62c96b9faa074322379ffa82a993849"}
Oct 31 17:36:46.791180 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:46.791135    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-0e59ca-77546c9db5-b8b5g"
Oct 31 17:36:46.833121 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:46.833062    4276 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-0e59ca-77546c9db5-b8b5g" podStartSLOduration=2.483895949 podStartE2EDuration="2.833048002s" totalImagesPullingTime="349.152053ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 17:36:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 17:36:46.83232615 +0000 UTC m=+569.630071078" watchObservedRunningTime="2026-10-31 17:36:46.833048002 +0000 UTC m=+569.630792929"
Oct 31 17:36:47.747373 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:47.747338    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:36:47.809521 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:47.809492    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-5c446c-56f77ff564-dd7dx" event={"ID":"67250413-5beb-44da-9925-71b1735d723b","Type":"ContainerStarted","Data":"c1c500d0691dd556927f205943d108dc8d79ce455dc8c8c53edc3223075e4385"}
Oct 31 17:36:47.809769 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:47.809751    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-5c446c-56f77ff564-dd7dx"
Oct 31 17:36:47.872563 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:47.872528    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:36:48.170466 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:48.169990    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:36:48.686467 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:48.685631    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-cf0a86-7bb4c5d6c5-8p8r5"
Oct 31 17:36:48.730341 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:48.730279    4276 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-5c446c-56f77ff564-dd7dx" podStartSLOduration=3.421423537 podStartE2EDuration="3.730258946s" totalImagesPullingTime="308.835409ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 17:36:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 17:36:47.863396455 +0000 UTC m=+570.661141383" watchObservedRunningTime="2026-10-31 17:36:48.730258946 +0000 UTC m=+571.528003875"
Oct 31 17:36:49.399539 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:49.391918    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:36:49.690681 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:49.690600    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-d80080-59b65f4495-h6gz8"
Oct 31 17:36:50.813654 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:50.813622    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-1c0bfd-6569c7d6f-l29ff"
Oct 31 17:36:50.842354 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:50.842327    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:36:51.005346 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:51.005147    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:36:51.122797 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:51.122691    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:36:52.637923 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:52.637813    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:36:52.824730 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:52.824702    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-0e59ca-77546c9db5-b8b5g"
Oct 31 17:36:53.820093 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:53.820065    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-5c446c-56f77ff564-dd7dx"
Oct 31 17:36:54.415776 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:54.415728    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:36:54.612012 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:54.611930    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:36:54.809468 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:54.808639    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-b78c31-6b5448c9d7-rbhfz"
Oct 31 17:36:55.191525 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:55.191354    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:36:55.311142 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:55.311076    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:36:55.431224 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:55.431189    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:36:56.114876 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:56.114828    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:36:56.225002 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:56.224907    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:36:56.714918 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:56.714881    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:36:57.811428 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:57.811382    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:36:58.621791 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:58.621738    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:36:59.165338 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:59.165305    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:36:59.475355 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:59.475302    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:36:59.751682 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:59.751641    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:36:59.885525 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:59.885491    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:36:59.939932 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:59.939899    4276 generic.go:356] "Generic (PLEG): container finished" podID="22af22dd-1130-409b-9a17-05229dc0de11" containerID="5563c62e742bdd643b7b91521fa78ea55c3acbe6686b95695605135a70068dcc" exitCode=0
Oct 31 17:36:59.940113 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:59.939989    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-c5dc6" event={"ID":"22af22dd-1130-409b-9a17-05229dc0de11","Type":"ContainerDied","Data":"5563c62e742bdd643b7b91521fa78ea55c3acbe6686b95695605135a70068dcc"}
Oct 31 17:36:59.940474 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:59.940434    4276 scope.go:122] "RemoveContainer" containerID="5563c62e742bdd643b7b91521fa78ea55c3acbe6686b95695605135a70068dcc"
Oct 31 17:36:59.942053 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:59.942020    4276 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 17:36:59.942139 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:59.942104    4276 generic.go:356] "Generic (PLEG): container finished" podID="33671198-f448-4f99-9bc9-6a1ed6980c5b" containerID="7aec3c6fddcf595c3e7e91ec3662c167ee611c0ab99dc0d5a11652e39a6c6d26" exitCode=0
Oct 31 17:36:59.942139 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:59.942128    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-ghwks" event={"ID":"33671198-f448-4f99-9bc9-6a1ed6980c5b","Type":"ContainerDied","Data":"7aec3c6fddcf595c3e7e91ec3662c167ee611c0ab99dc0d5a11652e39a6c6d26"}
Oct 31 17:36:59.942225 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:59.942109    4276 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 17:36:59.942225 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:59.942207    4276 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 17:36:59.942522 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:59.942506    4276 scope.go:122] "RemoveContainer" containerID="7aec3c6fddcf595c3e7e91ec3662c167ee611c0ab99dc0d5a11652e39a6c6d26"
Oct 31 17:36:59.943747 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:59.943722    4276 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 17:36:59.943855 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:59.943812    4276 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 17:36:59.943920 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:36:59.943864    4276 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 17:37:00.014237 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:00.014180    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:37:00.262066 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:00.261971    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:37:00.380432 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:00.380381    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:37:00.885464 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:00.881914    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:37:00.969277 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:00.968311    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-ghwks" event={"ID":"33671198-f448-4f99-9bc9-6a1ed6980c5b","Type":"ContainerStarted","Data":"09b8ee7e51fe351cc0bcc6f9d3dbcf11853e736da058d8d30cb89388f02fbd0c"}
Oct 31 17:37:00.991488 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:00.990700    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-c5dc6" event={"ID":"22af22dd-1130-409b-9a17-05229dc0de11","Type":"ContainerStarted","Data":"bc9404ff4e456fc89595255741cc75406b890990ce42d69b922760a2527d1feb"}
Oct 31 17:37:01.010182 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:01.010152    4276 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-c5dc6"
Oct 31 17:37:01.015378 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:01.015062    4276 patch_prober.go:29] interesting pod/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-c5dc6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" start-of-body=
Oct 31 17:37:01.015378 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:01.015116    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-c5dc6" podUID="22af22dd-1130-409b-9a17-05229dc0de11" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused"
Oct 31 17:37:01.016525 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:01.016428    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:37:01.210534 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:01.210273    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:37:01.310150 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:01.309528    4276 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-ghwks"
Oct 31 17:37:01.310150 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:01.310017    4276 patch_prober.go:29] interesting pod/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-ghwks container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" start-of-body=
Oct 31 17:37:01.310150 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:01.310068    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-ghwks" podUID="33671198-f448-4f99-9bc9-6a1ed6980c5b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused"
Oct 31 17:37:01.416581 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:01.413894    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:37:02.008319 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:02.007879    4276 patch_prober.go:29] interesting pod/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-c5dc6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" start-of-body=
Oct 31 17:37:02.008319 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:02.007935    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-c5dc6" podUID="22af22dd-1130-409b-9a17-05229dc0de11" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused"
Oct 31 17:37:02.301524 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:02.276057    4276 patch_prober.go:29] interesting pod/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-ghwks container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" start-of-body=
Oct 31 17:37:02.301524 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:02.276125    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-ghwks" podUID="33671198-f448-4f99-9bc9-6a1ed6980c5b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused"
Oct 31 17:37:02.312863 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:02.312804    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:37:02.623919 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:02.623739    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-c5dc6"
Oct 31 17:37:03.007324 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:03.006554    4276 patch_prober.go:29] interesting pod/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-c5dc6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" start-of-body=
Oct 31 17:37:03.007324 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:03.006615    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-c5dc6" podUID="22af22dd-1130-409b-9a17-05229dc0de11" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused"
Oct 31 17:37:03.086535 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:03.086457    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:37:03.276287 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:03.275801    4276 patch_prober.go:29] interesting pod/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-ghwks container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" start-of-body=
Oct 31 17:37:03.276287 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:03.275860    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-ghwks" podUID="33671198-f448-4f99-9bc9-6a1ed6980c5b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused"
Oct 31 17:37:03.628809 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:03.628781    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-ghwks"
Oct 31 17:37:04.003746 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:04.003706    4276 patch_prober.go:29] interesting pod/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-c5dc6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" start-of-body=
Oct 31 17:37:04.003957 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:04.003769    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-c5dc6" podUID="22af22dd-1130-409b-9a17-05229dc0de11" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused"
Oct 31 17:37:04.023590 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:04.023557    4276 generic.go:356] "Generic (PLEG): container finished" podID="6418b389-728b-4040-9b73-50cf013c00f3" containerID="ab10caa28f333e0be2ae6f6f40c4d874ad5fcc690e27588b9729c82508d8e8a0" exitCode=0
Oct 31 17:37:04.023730 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:04.023596    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-f8xx7" event={"ID":"6418b389-728b-4040-9b73-50cf013c00f3","Type":"ContainerDied","Data":"ab10caa28f333e0be2ae6f6f40c4d874ad5fcc690e27588b9729c82508d8e8a0"}
Oct 31 17:37:04.023945 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:04.023932    4276 scope.go:122] "RemoveContainer" containerID="ab10caa28f333e0be2ae6f6f40c4d874ad5fcc690e27588b9729c82508d8e8a0"
Oct 31 17:37:04.025206 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:04.025181    4276 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 17:37:04.025283 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:04.025269    4276 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 17:37:04.025320 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:04.025312    4276 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 17:37:04.277520 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:04.276895    4276 patch_prober.go:29] interesting pod/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-ghwks container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" start-of-body=
Oct 31 17:37:04.277520 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:04.276969    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-ghwks" podUID="33671198-f448-4f99-9bc9-6a1ed6980c5b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused"
Oct 31 17:37:04.348344 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:04.348242    4276 patch_prober.go:29] interesting pod/e2e-worker-w2-5c446c-openshift-default-797c6dc855-jxpd8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.38:15021/healthz/ready\": read tcp 10.133.0.2:46360->10.133.0.38:15021: read: connection reset by peer" start-of-body=
Oct 31 17:37:04.348344 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:04.348303    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-jxpd8" podUID="9d24572d-ec8f-4108-a83e-704c5ae976ff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.38:15021/healthz/ready\": read tcp 10.133.0.2:46360->10.133.0.38:15021: read: connection reset by peer"
Oct 31 17:37:04.634065 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:04.634029    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-f8xx7"
Oct 31 17:37:04.696551 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:04.696002    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:37:04.924579 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:04.924344    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:37:05.004465 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:05.003947    4276 patch_prober.go:29] interesting pod/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-c5dc6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" start-of-body=
Oct 31 17:37:05.004465 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:05.004021    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-c5dc6" podUID="22af22dd-1130-409b-9a17-05229dc0de11" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused"
Oct 31 17:37:05.033495 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:05.033437    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-f8xx7" event={"ID":"6418b389-728b-4040-9b73-50cf013c00f3","Type":"ContainerStarted","Data":"704cbbdc75a171c7759a0446b73f665cfcb7e7ae497a3c319af6a38912b2391a"}
Oct 31 17:37:05.035389 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:05.035369    4276 generic.go:356] "Generic (PLEG): container finished" podID="9d24572d-ec8f-4108-a83e-704c5ae976ff" containerID="e75c910ccf22e78e7ef8f45fcb517c57021f73f93f4dbca44088425c83d8ae28" exitCode=0
Oct 31 17:37:05.035472 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:05.035404    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-jxpd8" event={"ID":"9d24572d-ec8f-4108-a83e-704c5ae976ff","Type":"ContainerDied","Data":"e75c910ccf22e78e7ef8f45fcb517c57021f73f93f4dbca44088425c83d8ae28"}
Oct 31 17:37:05.035690 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:05.035678    4276 scope.go:122] "RemoveContainer" containerID="e75c910ccf22e78e7ef8f45fcb517c57021f73f93f4dbca44088425c83d8ae28"
Oct 31 17:37:05.036795 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:05.036769    4276 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 17:37:05.036849 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:05.036830    4276 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 17:37:05.036882 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:05.036866    4276 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 17:37:05.280468 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:05.275827    4276 patch_prober.go:29] interesting pod/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-ghwks container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" start-of-body=
Oct 31 17:37:05.280468 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:05.275905    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-ghwks" podUID="33671198-f448-4f99-9bc9-6a1ed6980c5b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused"
Oct 31 17:37:05.538545 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:05.531591    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:37:05.720467 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:05.717437    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:37:05.828468 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:05.824768    4276 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-f8xx7"
Oct 31 17:37:05.828669 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:05.828544    4276 patch_prober.go:29] interesting pod/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-f8xx7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body=
Oct 31 17:37:05.828669 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:05.828598    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-f8xx7" podUID="6418b389-728b-4040-9b73-50cf013c00f3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused"
Oct 31 17:37:06.008471 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:06.008188    4276 patch_prober.go:29] interesting pod/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-c5dc6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" start-of-body=
Oct 31 17:37:06.008471 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:06.008251    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-c5dc6" podUID="22af22dd-1130-409b-9a17-05229dc0de11" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused"
Oct 31 17:37:06.095525 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:06.091720    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:37:06.095525 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:06.094628    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-jxpd8" event={"ID":"9d24572d-ec8f-4108-a83e-704c5ae976ff","Type":"ContainerStarted","Data":"0fd95e192f8f3065b709ebc370709b230b43de2bd09f51864596dd4bad244101"}
Oct 31 17:37:06.237109 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:06.237074    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:37:06.276203 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:06.276161    4276 patch_prober.go:29] interesting pod/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-ghwks container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" start-of-body=
Oct 31 17:37:06.276520 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:06.276487    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-ghwks" podUID="33671198-f448-4f99-9bc9-6a1ed6980c5b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused"
Oct 31 17:37:06.426551 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:06.425166    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:37:06.558395 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:06.558366    4276 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-jxpd8"
Oct 31 17:37:06.559036 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:06.559011    4276 patch_prober.go:29] interesting pod/e2e-worker-w2-5c446c-openshift-default-797c6dc855-jxpd8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" start-of-body=
Oct 31 17:37:06.559243 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:06.559220    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-jxpd8" podUID="9d24572d-ec8f-4108-a83e-704c5ae976ff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused"
Oct 31 17:37:06.563280 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:06.563255    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:37:06.825465 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:06.824487    4276 patch_prober.go:29] interesting pod/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-f8xx7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body=
Oct 31 17:37:06.825465 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:06.824536    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-f8xx7" podUID="6418b389-728b-4040-9b73-50cf013c00f3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused"
Oct 31 17:37:07.014390 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:07.013339    4276 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-c5dc6"
Oct 31 17:37:07.015323 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:07.015287    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-c5dc6"
Oct 31 17:37:07.298518 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:07.298119    4276 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-ghwks"
Oct 31 17:37:07.301455 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:07.301414    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-ghwks"
Oct 31 17:37:07.545743 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:07.545703    4276 patch_prober.go:29] interesting pod/e2e-worker-w2-5c446c-openshift-default-797c6dc855-jxpd8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" start-of-body=
Oct 31 17:37:07.545910 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:07.545763    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-jxpd8" podUID="9d24572d-ec8f-4108-a83e-704c5ae976ff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused"
Oct 31 17:37:07.824619 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:07.824587    4276 patch_prober.go:29] interesting pod/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-f8xx7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body=
Oct 31 17:37:07.824785 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:07.824632    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-f8xx7" podUID="6418b389-728b-4040-9b73-50cf013c00f3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused"
Oct 31 17:37:08.076519 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:08.071515    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:37:08.546048 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:08.545968    4276 patch_prober.go:29] interesting pod/e2e-worker-w2-5c446c-openshift-default-797c6dc855-jxpd8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" start-of-body=
Oct 31 17:37:08.546048 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:08.546018    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-jxpd8" podUID="9d24572d-ec8f-4108-a83e-704c5ae976ff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused"
Oct 31 17:37:08.823907 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:08.823877    4276 patch_prober.go:29] interesting pod/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-f8xx7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body=
Oct 31 17:37:08.824067 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:08.823931    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-f8xx7" podUID="6418b389-728b-4040-9b73-50cf013c00f3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused"
Oct 31 17:37:09.546114 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:09.546081    4276 patch_prober.go:29] interesting pod/e2e-worker-w2-5c446c-openshift-default-797c6dc855-jxpd8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" start-of-body=
Oct 31 17:37:09.546531 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:09.546132    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-jxpd8" podUID="9d24572d-ec8f-4108-a83e-704c5ae976ff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused"
Oct 31 17:37:09.575320 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:09.575288    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:37:09.824286 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:09.824253    4276 patch_prober.go:29] interesting pod/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-f8xx7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body=
Oct 31 17:37:09.824438 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:09.824309    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-f8xx7" podUID="6418b389-728b-4040-9b73-50cf013c00f3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused"
Oct 31 17:37:10.115905 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:10.115796    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:37:10.241579 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:10.241543    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:37:10.545485 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:10.545388    4276 patch_prober.go:29] interesting pod/e2e-worker-w2-5c446c-openshift-default-797c6dc855-jxpd8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" start-of-body=
Oct 31 17:37:10.545485 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:10.545455    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-jxpd8" podUID="9d24572d-ec8f-4108-a83e-704c5ae976ff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused"
Oct 31 17:37:10.691249 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:10.691212    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:37:10.819115 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:10.819078    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:37:10.828390 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:10.828360    4276 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-f8xx7"
Oct 31 17:37:10.828697 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:10.828678    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-f8xx7"
Oct 31 17:37:10.829267 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:10.829250    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-f8xx7"
Oct 31 17:37:10.937092 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:10.937059    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:37:11.284527 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:11.284434    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:37:11.546303 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:11.546227    4276 patch_prober.go:29] interesting pod/e2e-worker-w2-5c446c-openshift-default-797c6dc855-jxpd8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" start-of-body=
Oct 31 17:37:11.546303 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:11.546279    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-jxpd8" podUID="9d24572d-ec8f-4108-a83e-704c5ae976ff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused"
Oct 31 17:37:12.207801 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:12.207763    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:37:12.233925 ip-10-0-129-127 kubenswrapper[4276]: E1031 17:37:12.233886    4276 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tg5qk" containerName="extractor"
Oct 31 17:37:12.549856 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:12.549832    4276 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-jxpd8"
Oct 31 17:37:12.550146 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:12.550121    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-jxpd8"
Oct 31 17:37:12.550717 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:12.550703    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-jxpd8"
Oct 31 17:37:12.876808 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:12.876724    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:37:14.480862 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:14.480830    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:37:15.973219 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:15.973187    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:37:16.258006 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:16.257921    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:37:17.408423 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:17.408395    4276 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zpf4d_5819fd64-9ac0-4436-b8fe-dcd3a27827ac/ovn-acl-logging/0.log"
Oct 31 17:37:17.409462 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:17.409424    4276 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zpf4d_5819fd64-9ac0-4436-b8fe-dcd3a27827ac/ovn-acl-logging/0.log"
Oct 31 17:37:17.582035 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:17.581998    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:37:19.051231 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:19.051194    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:37:20.604186 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:20.604151    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:37:20.733679 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:20.733645    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:37:22.252888 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:22.252849    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:37:23.757348 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:23.757303    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:37:25.168611 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:25.168575    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:37:25.751972 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:25.751932    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:37:26.807597 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:26.807567    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:37:28.263247 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:28.263214    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:37:29.779205 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:29.779156    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:37:31.375792 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:31.375755    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:37:32.978747 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:32.978689    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:37:33.367802 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:33.367769    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:37:34.672328 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:34.672297    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:37:36.256138 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:36.256105    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:37:37.748974 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:37.748944    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:37:39.303324 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:39.303289    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:37:40.857574 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:40.857540    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:37:41.502591 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:41.502549    4276 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w1-0e59ca/premium-w1-0e59ca-kserve-58765f9746-fqwq7"]
Oct 31 17:37:41.506866 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:41.506847    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-0e59ca/premium-w1-0e59ca-kserve-58765f9746-fqwq7"
Oct 31 17:37:41.521799 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:41.521771    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-0e59ca/premium-w1-0e59ca-kserve-58765f9746-fqwq7"]
Oct 31 17:37:41.533043 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:41.533016    4276 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w1-0e59ca\"/\"kube-root-ca.crt\""
Oct 31 17:37:41.533149 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:41.533056    4276 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w1-0e59ca\"/\"openshift-service-ca.crt\""
Oct 31 17:37:41.536432 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:41.536414    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-0e59ca\"/\"premium-w1-0e59ca-kserve-self-signed-certs\""
Oct 31 17:37:41.536502 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:41.536415    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-0e59ca\"/\"default-dockercfg-pwznp\""
Oct 31 17:37:41.574091 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:41.574061    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:37:41.611109 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:41.611083    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8cf1e52e-70f8-42e2-bdf3-a315e17f10f5-model-cache\") pod \"premium-w1-0e59ca-kserve-58765f9746-fqwq7\" (UID: \"8cf1e52e-70f8-42e2-bdf3-a315e17f10f5\") " pod="e2e-models-e2e-worker-w1-0e59ca/premium-w1-0e59ca-kserve-58765f9746-fqwq7"
Oct 31 17:37:41.611228 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:41.611127    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8cf1e52e-70f8-42e2-bdf3-a315e17f10f5-tmp-dir\") pod \"premium-w1-0e59ca-kserve-58765f9746-fqwq7\" (UID: \"8cf1e52e-70f8-42e2-bdf3-a315e17f10f5\") " pod="e2e-models-e2e-worker-w1-0e59ca/premium-w1-0e59ca-kserve-58765f9746-fqwq7"
Oct 31 17:37:41.611228 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:41.611201    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8cf1e52e-70f8-42e2-bdf3-a315e17f10f5-tls-certs\") pod \"premium-w1-0e59ca-kserve-58765f9746-fqwq7\" (UID: \"8cf1e52e-70f8-42e2-bdf3-a315e17f10f5\") " pod="e2e-models-e2e-worker-w1-0e59ca/premium-w1-0e59ca-kserve-58765f9746-fqwq7"
Oct 31 17:37:41.611299 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:41.611263    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8cf1e52e-70f8-42e2-bdf3-a315e17f10f5-home\") pod \"premium-w1-0e59ca-kserve-58765f9746-fqwq7\" (UID: \"8cf1e52e-70f8-42e2-bdf3-a315e17f10f5\") " pod="e2e-models-e2e-worker-w1-0e59ca/premium-w1-0e59ca-kserve-58765f9746-fqwq7"
Oct 31 17:37:41.611350 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:41.611329    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8cf1e52e-70f8-42e2-bdf3-a315e17f10f5-dshm\") pod \"premium-w1-0e59ca-kserve-58765f9746-fqwq7\" (UID: \"8cf1e52e-70f8-42e2-bdf3-a315e17f10f5\") " pod="e2e-models-e2e-worker-w1-0e59ca/premium-w1-0e59ca-kserve-58765f9746-fqwq7"
Oct 31 17:37:41.611432 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:41.611418    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5lc4s\" (UniqueName: \"kubernetes.io/projected/8cf1e52e-70f8-42e2-bdf3-a315e17f10f5-kube-api-access-5lc4s\") pod \"premium-w1-0e59ca-kserve-58765f9746-fqwq7\" (UID: \"8cf1e52e-70f8-42e2-bdf3-a315e17f10f5\") " pod="e2e-models-e2e-worker-w1-0e59ca/premium-w1-0e59ca-kserve-58765f9746-fqwq7"
Oct 31 17:37:41.712849 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:41.712809    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5lc4s\" (UniqueName: \"kubernetes.io/projected/8cf1e52e-70f8-42e2-bdf3-a315e17f10f5-kube-api-access-5lc4s\") pod \"premium-w1-0e59ca-kserve-58765f9746-fqwq7\" (UID: \"8cf1e52e-70f8-42e2-bdf3-a315e17f10f5\") " pod="e2e-models-e2e-worker-w1-0e59ca/premium-w1-0e59ca-kserve-58765f9746-fqwq7"
Oct 31 17:37:41.713506 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:41.713478    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8cf1e52e-70f8-42e2-bdf3-a315e17f10f5-model-cache\") pod \"premium-w1-0e59ca-kserve-58765f9746-fqwq7\" (UID: \"8cf1e52e-70f8-42e2-bdf3-a315e17f10f5\") " pod="e2e-models-e2e-worker-w1-0e59ca/premium-w1-0e59ca-kserve-58765f9746-fqwq7"
Oct 31 17:37:41.713697 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:41.713682    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8cf1e52e-70f8-42e2-bdf3-a315e17f10f5-tmp-dir\") pod \"premium-w1-0e59ca-kserve-58765f9746-fqwq7\" (UID: \"8cf1e52e-70f8-42e2-bdf3-a315e17f10f5\") " pod="e2e-models-e2e-worker-w1-0e59ca/premium-w1-0e59ca-kserve-58765f9746-fqwq7"
Oct 31 17:37:41.713812 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:41.713799    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8cf1e52e-70f8-42e2-bdf3-a315e17f10f5-tls-certs\") pod \"premium-w1-0e59ca-kserve-58765f9746-fqwq7\" (UID: \"8cf1e52e-70f8-42e2-bdf3-a315e17f10f5\") " pod="e2e-models-e2e-worker-w1-0e59ca/premium-w1-0e59ca-kserve-58765f9746-fqwq7"
Oct 31 17:37:41.713992 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:41.713957    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8cf1e52e-70f8-42e2-bdf3-a315e17f10f5-home\") pod \"premium-w1-0e59ca-kserve-58765f9746-fqwq7\" (UID: \"8cf1e52e-70f8-42e2-bdf3-a315e17f10f5\") " pod="e2e-models-e2e-worker-w1-0e59ca/premium-w1-0e59ca-kserve-58765f9746-fqwq7"
Oct 31 17:37:41.714116 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:41.714102    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8cf1e52e-70f8-42e2-bdf3-a315e17f10f5-dshm\") pod \"premium-w1-0e59ca-kserve-58765f9746-fqwq7\" (UID: \"8cf1e52e-70f8-42e2-bdf3-a315e17f10f5\") " pod="e2e-models-e2e-worker-w1-0e59ca/premium-w1-0e59ca-kserve-58765f9746-fqwq7"
Oct 31 17:37:41.714707 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:41.714648    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8cf1e52e-70f8-42e2-bdf3-a315e17f10f5-model-cache\") pod \"premium-w1-0e59ca-kserve-58765f9746-fqwq7\" (UID: \"8cf1e52e-70f8-42e2-bdf3-a315e17f10f5\") " pod="e2e-models-e2e-worker-w1-0e59ca/premium-w1-0e59ca-kserve-58765f9746-fqwq7"
Oct 31 17:37:41.714707 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:41.714682    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8cf1e52e-70f8-42e2-bdf3-a315e17f10f5-home\") pod \"premium-w1-0e59ca-kserve-58765f9746-fqwq7\" (UID: \"8cf1e52e-70f8-42e2-bdf3-a315e17f10f5\") " pod="e2e-models-e2e-worker-w1-0e59ca/premium-w1-0e59ca-kserve-58765f9746-fqwq7"
Oct 31 17:37:41.715359 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:41.715334    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8cf1e52e-70f8-42e2-bdf3-a315e17f10f5-tmp-dir\") pod \"premium-w1-0e59ca-kserve-58765f9746-fqwq7\" (UID: \"8cf1e52e-70f8-42e2-bdf3-a315e17f10f5\") " pod="e2e-models-e2e-worker-w1-0e59ca/premium-w1-0e59ca-kserve-58765f9746-fqwq7"
Oct 31 17:37:41.718027 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:41.717995    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8cf1e52e-70f8-42e2-bdf3-a315e17f10f5-dshm\") pod \"premium-w1-0e59ca-kserve-58765f9746-fqwq7\" (UID: \"8cf1e52e-70f8-42e2-bdf3-a315e17f10f5\") " pod="e2e-models-e2e-worker-w1-0e59ca/premium-w1-0e59ca-kserve-58765f9746-fqwq7"
Oct 31 17:37:41.718262 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:41.718238    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8cf1e52e-70f8-42e2-bdf3-a315e17f10f5-tls-certs\") pod \"premium-w1-0e59ca-kserve-58765f9746-fqwq7\" (UID: \"8cf1e52e-70f8-42e2-bdf3-a315e17f10f5\") " pod="e2e-models-e2e-worker-w1-0e59ca/premium-w1-0e59ca-kserve-58765f9746-fqwq7"
Oct 31 17:37:41.733202 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:41.733177    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5lc4s\" (UniqueName: \"kubernetes.io/projected/8cf1e52e-70f8-42e2-bdf3-a315e17f10f5-kube-api-access-5lc4s\") pod \"premium-w1-0e59ca-kserve-58765f9746-fqwq7\" (UID: \"8cf1e52e-70f8-42e2-bdf3-a315e17f10f5\") " pod="e2e-models-e2e-worker-w1-0e59ca/premium-w1-0e59ca-kserve-58765f9746-fqwq7"
Oct 31 17:37:41.821118 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:41.821095    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-0e59ca/premium-w1-0e59ca-kserve-58765f9746-fqwq7"
Oct 31 17:37:42.120587 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:42.120564    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-0e59ca/premium-w1-0e59ca-kserve-58765f9746-fqwq7"]
Oct 31 17:37:42.122407 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:37:42.122382    4276 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8cf1e52e_70f8_42e2_bdf3_a315e17f10f5.slice/crio-6d777ed7b88cd6d8beed8d3294233a5004202e03ff061f5a09410d34ca782e5d WatchSource:0}: Error finding container 6d777ed7b88cd6d8beed8d3294233a5004202e03ff061f5a09410d34ca782e5d: Status 404 returned error can't find the container with id 6d777ed7b88cd6d8beed8d3294233a5004202e03ff061f5a09410d34ca782e5d
Oct 31 17:37:42.234002 ip-10-0-129-127 kubenswrapper[4276]: E1031 17:37:42.233969    4276 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tg5qk" containerName="extractor"
Oct 31 17:37:42.332875 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:42.332831    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-0e59ca/premium-w1-0e59ca-kserve-58765f9746-fqwq7" event={"ID":"8cf1e52e-70f8-42e2-bdf3-a315e17f10f5","Type":"ContainerStarted","Data":"da6e91bd6c1fc2162a5b6d8134f06ad43a10ec61ac7d785a236ab9032e95cc3c"}
Oct 31 17:37:42.332875 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:42.332881    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-0e59ca/premium-w1-0e59ca-kserve-58765f9746-fqwq7" event={"ID":"8cf1e52e-70f8-42e2-bdf3-a315e17f10f5","Type":"ContainerStarted","Data":"6d777ed7b88cd6d8beed8d3294233a5004202e03ff061f5a09410d34ca782e5d"}
Oct 31 17:37:42.362547 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:42.362516    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:37:42.374123 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:42.374046    4276 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w1-0e59ca/premium-w1-0e59ca-kserve-58765f9746-fqwq7" podStartSLOduration=1.374033052 podStartE2EDuration="1.374033052s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 17:37:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 17:37:42.372507853 +0000 UTC m=+625.170252782" watchObservedRunningTime="2026-10-31 17:37:42.374033052 +0000 UTC m=+625.171777974"
Oct 31 17:37:42.821369 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:42.821341    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-0e59ca/premium-w1-0e59ca-kserve-58765f9746-fqwq7"
Oct 31 17:37:42.937973 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:42.937922    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:37:43.917693 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:43.917656    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:37:45.424312 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:45.424277    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:37:46.848518 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:46.848482    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:37:48.344976 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:48.344941    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:37:49.240146 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:49.240109    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:37:49.957730 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:49.957699    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:37:51.378931 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:51.378899    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:37:51.821898 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:51.821872    4276 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w1-0e59ca/premium-w1-0e59ca-kserve-58765f9746-fqwq7"
Oct 31 17:37:51.834565 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:51.834541    4276 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w1-0e59ca/premium-w1-0e59ca-kserve-58765f9746-fqwq7"
Oct 31 17:37:52.413915 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:52.413884    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-0e59ca/premium-w1-0e59ca-kserve-58765f9746-fqwq7"
Oct 31 17:37:52.855364 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:52.855328    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:37:53.055087 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:53.055047    4276 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-ee816e/premium-w0-ee816e-kserve-65c86cd6db-lnt9p"]
Oct 31 17:37:53.059158 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:53.059136    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-ee816e/premium-w0-ee816e-kserve-65c86cd6db-lnt9p"
Oct 31 17:37:53.080639 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:53.080610    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-ee816e/premium-w0-ee816e-kserve-65c86cd6db-lnt9p"]
Oct 31 17:37:53.090585 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:53.090560    4276 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-ee816e\"/\"openshift-service-ca.crt\""
Oct 31 17:37:53.090715 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:53.090594    4276 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-ee816e\"/\"kube-root-ca.crt\""
Oct 31 17:37:53.090715 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:53.090653    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-ee816e\"/\"default-dockercfg-7pq72\""
Oct 31 17:37:53.090798 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:53.090708    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-ee816e\"/\"premium-w0-ee816e-kserve-self-signed-certs\""
Oct 31 17:37:53.132276 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:53.132207    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:37:53.156457 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:53.156412    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e536c902-52a9-4b36-a2ad-eb5500781a5e-home\") pod \"premium-w0-ee816e-kserve-65c86cd6db-lnt9p\" (UID: \"e536c902-52a9-4b36-a2ad-eb5500781a5e\") " pod="e2e-models-e2e-worker-w0-ee816e/premium-w0-ee816e-kserve-65c86cd6db-lnt9p"
Oct 31 17:37:53.156593 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:53.156571    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e536c902-52a9-4b36-a2ad-eb5500781a5e-model-cache\") pod \"premium-w0-ee816e-kserve-65c86cd6db-lnt9p\" (UID: \"e536c902-52a9-4b36-a2ad-eb5500781a5e\") " pod="e2e-models-e2e-worker-w0-ee816e/premium-w0-ee816e-kserve-65c86cd6db-lnt9p"
Oct 31 17:37:53.156654 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:53.156632    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e536c902-52a9-4b36-a2ad-eb5500781a5e-tmp-dir\") pod \"premium-w0-ee816e-kserve-65c86cd6db-lnt9p\" (UID: \"e536c902-52a9-4b36-a2ad-eb5500781a5e\") " pod="e2e-models-e2e-worker-w0-ee816e/premium-w0-ee816e-kserve-65c86cd6db-lnt9p"
Oct 31 17:37:53.156783 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:53.156765    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e536c902-52a9-4b36-a2ad-eb5500781a5e-dshm\") pod \"premium-w0-ee816e-kserve-65c86cd6db-lnt9p\" (UID: \"e536c902-52a9-4b36-a2ad-eb5500781a5e\") " pod="e2e-models-e2e-worker-w0-ee816e/premium-w0-ee816e-kserve-65c86cd6db-lnt9p"
Oct 31 17:37:53.156832 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:53.156798    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e536c902-52a9-4b36-a2ad-eb5500781a5e-tls-certs\") pod \"premium-w0-ee816e-kserve-65c86cd6db-lnt9p\" (UID: \"e536c902-52a9-4b36-a2ad-eb5500781a5e\") " pod="e2e-models-e2e-worker-w0-ee816e/premium-w0-ee816e-kserve-65c86cd6db-lnt9p"
Oct 31 17:37:53.156910 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:53.156885    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4jwzd\" (UniqueName: \"kubernetes.io/projected/e536c902-52a9-4b36-a2ad-eb5500781a5e-kube-api-access-4jwzd\") pod \"premium-w0-ee816e-kserve-65c86cd6db-lnt9p\" (UID: \"e536c902-52a9-4b36-a2ad-eb5500781a5e\") " pod="e2e-models-e2e-worker-w0-ee816e/premium-w0-ee816e-kserve-65c86cd6db-lnt9p"
Oct 31 17:37:53.258057 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:53.258010    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e536c902-52a9-4b36-a2ad-eb5500781a5e-home\") pod \"premium-w0-ee816e-kserve-65c86cd6db-lnt9p\" (UID: \"e536c902-52a9-4b36-a2ad-eb5500781a5e\") " pod="e2e-models-e2e-worker-w0-ee816e/premium-w0-ee816e-kserve-65c86cd6db-lnt9p"
Oct 31 17:37:53.258254 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:53.258151    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e536c902-52a9-4b36-a2ad-eb5500781a5e-model-cache\") pod \"premium-w0-ee816e-kserve-65c86cd6db-lnt9p\" (UID: \"e536c902-52a9-4b36-a2ad-eb5500781a5e\") " pod="e2e-models-e2e-worker-w0-ee816e/premium-w0-ee816e-kserve-65c86cd6db-lnt9p"
Oct 31 17:37:53.258254 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:53.258202    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e536c902-52a9-4b36-a2ad-eb5500781a5e-tmp-dir\") pod \"premium-w0-ee816e-kserve-65c86cd6db-lnt9p\" (UID: \"e536c902-52a9-4b36-a2ad-eb5500781a5e\") " pod="e2e-models-e2e-worker-w0-ee816e/premium-w0-ee816e-kserve-65c86cd6db-lnt9p"
Oct 31 17:37:53.258381 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:53.258264    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e536c902-52a9-4b36-a2ad-eb5500781a5e-dshm\") pod \"premium-w0-ee816e-kserve-65c86cd6db-lnt9p\" (UID: \"e536c902-52a9-4b36-a2ad-eb5500781a5e\") " pod="e2e-models-e2e-worker-w0-ee816e/premium-w0-ee816e-kserve-65c86cd6db-lnt9p"
Oct 31 17:37:53.258381 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:53.258296    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e536c902-52a9-4b36-a2ad-eb5500781a5e-tls-certs\") pod \"premium-w0-ee816e-kserve-65c86cd6db-lnt9p\" (UID: \"e536c902-52a9-4b36-a2ad-eb5500781a5e\") " pod="e2e-models-e2e-worker-w0-ee816e/premium-w0-ee816e-kserve-65c86cd6db-lnt9p"
Oct 31 17:37:53.258381 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:53.258362    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4jwzd\" (UniqueName: \"kubernetes.io/projected/e536c902-52a9-4b36-a2ad-eb5500781a5e-kube-api-access-4jwzd\") pod \"premium-w0-ee816e-kserve-65c86cd6db-lnt9p\" (UID: \"e536c902-52a9-4b36-a2ad-eb5500781a5e\") " pod="e2e-models-e2e-worker-w0-ee816e/premium-w0-ee816e-kserve-65c86cd6db-lnt9p"
Oct 31 17:37:53.258978 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:53.258940    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e536c902-52a9-4b36-a2ad-eb5500781a5e-tmp-dir\") pod \"premium-w0-ee816e-kserve-65c86cd6db-lnt9p\" (UID: \"e536c902-52a9-4b36-a2ad-eb5500781a5e\") " pod="e2e-models-e2e-worker-w0-ee816e/premium-w0-ee816e-kserve-65c86cd6db-lnt9p"
Oct 31 17:37:53.259456 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:53.259421    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e536c902-52a9-4b36-a2ad-eb5500781a5e-model-cache\") pod \"premium-w0-ee816e-kserve-65c86cd6db-lnt9p\" (UID: \"e536c902-52a9-4b36-a2ad-eb5500781a5e\") " pod="e2e-models-e2e-worker-w0-ee816e/premium-w0-ee816e-kserve-65c86cd6db-lnt9p"
Oct 31 17:37:53.259601 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:53.259433    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e536c902-52a9-4b36-a2ad-eb5500781a5e-home\") pod \"premium-w0-ee816e-kserve-65c86cd6db-lnt9p\" (UID: \"e536c902-52a9-4b36-a2ad-eb5500781a5e\") " pod="e2e-models-e2e-worker-w0-ee816e/premium-w0-ee816e-kserve-65c86cd6db-lnt9p"
Oct 31 17:37:53.264303 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:53.263035    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e536c902-52a9-4b36-a2ad-eb5500781a5e-dshm\") pod \"premium-w0-ee816e-kserve-65c86cd6db-lnt9p\" (UID: \"e536c902-52a9-4b36-a2ad-eb5500781a5e\") " pod="e2e-models-e2e-worker-w0-ee816e/premium-w0-ee816e-kserve-65c86cd6db-lnt9p"
Oct 31 17:37:53.265015 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:53.264995    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e536c902-52a9-4b36-a2ad-eb5500781a5e-tls-certs\") pod \"premium-w0-ee816e-kserve-65c86cd6db-lnt9p\" (UID: \"e536c902-52a9-4b36-a2ad-eb5500781a5e\") " pod="e2e-models-e2e-worker-w0-ee816e/premium-w0-ee816e-kserve-65c86cd6db-lnt9p"
Oct 31 17:37:53.282036 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:53.282006    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4jwzd\" (UniqueName: \"kubernetes.io/projected/e536c902-52a9-4b36-a2ad-eb5500781a5e-kube-api-access-4jwzd\") pod \"premium-w0-ee816e-kserve-65c86cd6db-lnt9p\" (UID: \"e536c902-52a9-4b36-a2ad-eb5500781a5e\") " pod="e2e-models-e2e-worker-w0-ee816e/premium-w0-ee816e-kserve-65c86cd6db-lnt9p"
Oct 31 17:37:53.372544 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:53.372514    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-ee816e/premium-w0-ee816e-kserve-65c86cd6db-lnt9p"
Oct 31 17:37:53.669733 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:53.669705    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-ee816e/premium-w0-ee816e-kserve-65c86cd6db-lnt9p"]
Oct 31 17:37:53.670893 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:37:53.670867    4276 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode536c902_52a9_4b36_a2ad_eb5500781a5e.slice/crio-c61fb67c93b42d3898d2cde64e58720bfd198d0cace91429f7ded9237cc9c4ab WatchSource:0}: Error finding container c61fb67c93b42d3898d2cde64e58720bfd198d0cace91429f7ded9237cc9c4ab: Status 404 returned error can't find the container with id c61fb67c93b42d3898d2cde64e58720bfd198d0cace91429f7ded9237cc9c4ab
Oct 31 17:37:54.273226 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:54.273187    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:37:54.415600 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:54.415566    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-ee816e/premium-w0-ee816e-kserve-65c86cd6db-lnt9p" event={"ID":"e536c902-52a9-4b36-a2ad-eb5500781a5e","Type":"ContainerStarted","Data":"36c6d2d02f285224be746e52aa2bbb9d39a797d025543f7867fdd95f94cb5696"}
Oct 31 17:37:54.415743 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:54.415609    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-ee816e/premium-w0-ee816e-kserve-65c86cd6db-lnt9p" event={"ID":"e536c902-52a9-4b36-a2ad-eb5500781a5e","Type":"ContainerStarted","Data":"c61fb67c93b42d3898d2cde64e58720bfd198d0cace91429f7ded9237cc9c4ab"}
Oct 31 17:37:54.460515 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:54.460469    4276 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-ee816e/premium-w0-ee816e-kserve-65c86cd6db-lnt9p" podStartSLOduration=1.460455727 podStartE2EDuration="1.460455727s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 17:37:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 17:37:54.459197868 +0000 UTC m=+637.256942797" watchObservedRunningTime="2026-10-31 17:37:54.460455727 +0000 UTC m=+637.258200649"
Oct 31 17:37:55.373141 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:55.373115    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-ee816e/premium-w0-ee816e-kserve-65c86cd6db-lnt9p"
Oct 31 17:37:55.888229 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:55.888189    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:37:56.936186 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:56.936150    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:37:57.359365 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:57.359317    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:37:58.942303 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:37:58.942270    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:38:00.362688 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:38:00.362621    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:38:01.374342 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:38:01.374312    4276 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-25kdb"]
Oct 31 17:38:01.378462 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:38:01.378430    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-25kdb"
Oct 31 17:38:01.401680 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:38:01.401654    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-25kdb"]
Oct 31 17:38:01.409687 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:38:01.409667    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w6-d80080-openshift-default-dockercfg-hqzw7\""
Oct 31 17:38:01.517066 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:38:01.517032    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:38:01.553982 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:38:01.553950    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6ec6accb-4b2a-45a8-bab2-fe65f2f5df2d-istio-podinfo\") pod \"e2e-worker-w6-d80080-openshift-default-956cddf57-25kdb\" (UID: \"6ec6accb-4b2a-45a8-bab2-fe65f2f5df2d\") " pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-25kdb"
Oct 31 17:38:01.554145 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:38:01.553992    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6ec6accb-4b2a-45a8-bab2-fe65f2f5df2d-istio-token\") pod \"e2e-worker-w6-d80080-openshift-default-956cddf57-25kdb\" (UID: \"6ec6accb-4b2a-45a8-bab2-fe65f2f5df2d\") " pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-25kdb"
Oct 31 17:38:01.554145 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:38:01.554130    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kp64p\" (UniqueName: \"kubernetes.io/projected/6ec6accb-4b2a-45a8-bab2-fe65f2f5df2d-kube-api-access-kp64p\") pod \"e2e-worker-w6-d80080-openshift-default-956cddf57-25kdb\" (UID: \"6ec6accb-4b2a-45a8-bab2-fe65f2f5df2d\") " pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-25kdb"
Oct 31 17:38:01.554262 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:38:01.554172    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ec6accb-4b2a-45a8-bab2-fe65f2f5df2d-credential-socket\") pod \"e2e-worker-w6-d80080-openshift-default-956cddf57-25kdb\" (UID: \"6ec6accb-4b2a-45a8-bab2-fe65f2f5df2d\") " pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-25kdb"
Oct 31 17:38:01.554262 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:38:01.554230    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6ec6accb-4b2a-45a8-bab2-fe65f2f5df2d-istio-data\") pod \"e2e-worker-w6-d80080-openshift-default-956cddf57-25kdb\" (UID: \"6ec6accb-4b2a-45a8-bab2-fe65f2f5df2d\") " pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-25kdb"
Oct 31 17:38:01.554340 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:38:01.554263    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6ec6accb-4b2a-45a8-bab2-fe65f2f5df2d-istiod-ca-cert\") pod \"e2e-worker-w6-d80080-openshift-default-956cddf57-25kdb\" (UID: \"6ec6accb-4b2a-45a8-bab2-fe65f2f5df2d\") " pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-25kdb"
Oct 31 17:38:01.554383 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:38:01.554356    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ec6accb-4b2a-45a8-bab2-fe65f2f5df2d-workload-socket\") pod \"e2e-worker-w6-d80080-openshift-default-956cddf57-25kdb\" (UID: \"6ec6accb-4b2a-45a8-bab2-fe65f2f5df2d\") " pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-25kdb"
Oct 31 17:38:01.554509 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:38:01.554428    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6ec6accb-4b2a-45a8-bab2-fe65f2f5df2d-istio-envoy\") pod \"e2e-worker-w6-d80080-openshift-default-956cddf57-25kdb\" (UID: \"6ec6accb-4b2a-45a8-bab2-fe65f2f5df2d\") " pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-25kdb"
Oct 31 17:38:01.554589 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:38:01.554574    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6ec6accb-4b2a-45a8-bab2-fe65f2f5df2d-workload-certs\") pod \"e2e-worker-w6-d80080-openshift-default-956cddf57-25kdb\" (UID: \"6ec6accb-4b2a-45a8-bab2-fe65f2f5df2d\") " pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-25kdb"
Oct 31 17:38:01.655391 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:38:01.655303    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6ec6accb-4b2a-45a8-bab2-fe65f2f5df2d-istio-envoy\") pod \"e2e-worker-w6-d80080-openshift-default-956cddf57-25kdb\" (UID: \"6ec6accb-4b2a-45a8-bab2-fe65f2f5df2d\") " pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-25kdb"
Oct 31 17:38:01.655391 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:38:01.655351    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6ec6accb-4b2a-45a8-bab2-fe65f2f5df2d-workload-certs\") pod \"e2e-worker-w6-d80080-openshift-default-956cddf57-25kdb\" (UID: \"6ec6accb-4b2a-45a8-bab2-fe65f2f5df2d\") " pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-25kdb"
Oct 31 17:38:01.655391 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:38:01.655393    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6ec6accb-4b2a-45a8-bab2-fe65f2f5df2d-istio-podinfo\") pod \"e2e-worker-w6-d80080-openshift-default-956cddf57-25kdb\" (UID: \"6ec6accb-4b2a-45a8-bab2-fe65f2f5df2d\") " pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-25kdb"
Oct 31 17:38:01.655696 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:38:01.655407    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6ec6accb-4b2a-45a8-bab2-fe65f2f5df2d-istio-token\") pod \"e2e-worker-w6-d80080-openshift-default-956cddf57-25kdb\" (UID: \"6ec6accb-4b2a-45a8-bab2-fe65f2f5df2d\") " pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-25kdb"
Oct 31 17:38:01.655696 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:38:01.655498    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kp64p\" (UniqueName: \"kubernetes.io/projected/6ec6accb-4b2a-45a8-bab2-fe65f2f5df2d-kube-api-access-kp64p\") pod \"e2e-worker-w6-d80080-openshift-default-956cddf57-25kdb\" (UID: \"6ec6accb-4b2a-45a8-bab2-fe65f2f5df2d\") " pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-25kdb"
Oct 31 17:38:01.655696 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:38:01.655538    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ec6accb-4b2a-45a8-bab2-fe65f2f5df2d-credential-socket\") pod \"e2e-worker-w6-d80080-openshift-default-956cddf57-25kdb\" (UID: \"6ec6accb-4b2a-45a8-bab2-fe65f2f5df2d\") " pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-25kdb"
Oct 31 17:38:01.655696 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:38:01.655599    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6ec6accb-4b2a-45a8-bab2-fe65f2f5df2d-istio-data\") pod \"e2e-worker-w6-d80080-openshift-default-956cddf57-25kdb\" (UID: \"6ec6accb-4b2a-45a8-bab2-fe65f2f5df2d\") " pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-25kdb"
Oct 31 17:38:01.655696 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:38:01.655621    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6ec6accb-4b2a-45a8-bab2-fe65f2f5df2d-istiod-ca-cert\") pod \"e2e-worker-w6-d80080-openshift-default-956cddf57-25kdb\" (UID: \"6ec6accb-4b2a-45a8-bab2-fe65f2f5df2d\") " pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-25kdb"
Oct 31 17:38:01.655901 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:38:01.655703    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ec6accb-4b2a-45a8-bab2-fe65f2f5df2d-workload-socket\") pod \"e2e-worker-w6-d80080-openshift-default-956cddf57-25kdb\" (UID: \"6ec6accb-4b2a-45a8-bab2-fe65f2f5df2d\") " pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-25kdb"
Oct 31 17:38:01.655901 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:38:01.655869    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6ec6accb-4b2a-45a8-bab2-fe65f2f5df2d-workload-certs\") pod \"e2e-worker-w6-d80080-openshift-default-956cddf57-25kdb\" (UID: \"6ec6accb-4b2a-45a8-bab2-fe65f2f5df2d\") " pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-25kdb"
Oct 31 17:38:01.655970 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:38:01.655941    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6ec6accb-4b2a-45a8-bab2-fe65f2f5df2d-istio-data\") pod \"e2e-worker-w6-d80080-openshift-default-956cddf57-25kdb\" (UID: \"6ec6accb-4b2a-45a8-bab2-fe65f2f5df2d\") " pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-25kdb"
Oct 31 17:38:01.656070 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:38:01.656049    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ec6accb-4b2a-45a8-bab2-fe65f2f5df2d-workload-socket\") pod \"e2e-worker-w6-d80080-openshift-default-956cddf57-25kdb\" (UID: \"6ec6accb-4b2a-45a8-bab2-fe65f2f5df2d\") " pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-25kdb"
Oct 31 17:38:01.656245 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:38:01.656216    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ec6accb-4b2a-45a8-bab2-fe65f2f5df2d-credential-socket\") pod \"e2e-worker-w6-d80080-openshift-default-956cddf57-25kdb\" (UID: \"6ec6accb-4b2a-45a8-bab2-fe65f2f5df2d\") " pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-25kdb"
Oct 31 17:38:01.656374 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:38:01.656356    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6ec6accb-4b2a-45a8-bab2-fe65f2f5df2d-istiod-ca-cert\") pod \"e2e-worker-w6-d80080-openshift-default-956cddf57-25kdb\" (UID: \"6ec6accb-4b2a-45a8-bab2-fe65f2f5df2d\") " pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-25kdb"
Oct 31 17:38:01.657793 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:38:01.657774    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6ec6accb-4b2a-45a8-bab2-fe65f2f5df2d-istio-envoy\") pod \"e2e-worker-w6-d80080-openshift-default-956cddf57-25kdb\" (UID: \"6ec6accb-4b2a-45a8-bab2-fe65f2f5df2d\") " pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-25kdb"
Oct 31 17:38:01.657876 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:38:01.657858    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6ec6accb-4b2a-45a8-bab2-fe65f2f5df2d-istio-podinfo\") pod \"e2e-worker-w6-d80080-openshift-default-956cddf57-25kdb\" (UID: \"6ec6accb-4b2a-45a8-bab2-fe65f2f5df2d\") " pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-25kdb"
Oct 31 17:38:01.675427 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:38:01.675406    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6ec6accb-4b2a-45a8-bab2-fe65f2f5df2d-istio-token\") pod \"e2e-worker-w6-d80080-openshift-default-956cddf57-25kdb\" (UID: \"6ec6accb-4b2a-45a8-bab2-fe65f2f5df2d\") " pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-25kdb"
Oct 31 17:38:01.675805 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:38:01.675784    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kp64p\" (UniqueName: \"kubernetes.io/projected/6ec6accb-4b2a-45a8-bab2-fe65f2f5df2d-kube-api-access-kp64p\") pod \"e2e-worker-w6-d80080-openshift-default-956cddf57-25kdb\" (UID: \"6ec6accb-4b2a-45a8-bab2-fe65f2f5df2d\") " pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-25kdb"
Oct 31 17:38:01.691989 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:38:01.691971    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-25kdb"
Oct 31 17:38:01.859047 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:38:01.859010    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:38:02.005178 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:38:02.005153    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-25kdb"]
Oct 31 17:38:02.005893 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:38:02.005863    4276 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ec6accb_4b2a_45a8_bab2_fe65f2f5df2d.slice/crio-c7b22e980dc77eb919d97c899ab99174f89c58c05a99a1385ff91d76d8886cd1 WatchSource:0}: Error finding container c7b22e980dc77eb919d97c899ab99174f89c58c05a99a1385ff91d76d8886cd1: Status 404 returned error can't find the container with id c7b22e980dc77eb919d97c899ab99174f89c58c05a99a1385ff91d76d8886cd1
Oct 31 17:38:02.008281 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:38:02.008254    4276 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 17:38:02.008359 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:38:02.008317    4276 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 17:38:02.008359 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:38:02.008348    4276 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 17:38:02.430236 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:38:02.430197    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:38:02.477512 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:38:02.477466    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-25kdb" event={"ID":"6ec6accb-4b2a-45a8-bab2-fe65f2f5df2d","Type":"ContainerStarted","Data":"a1287b2dab22e7f754ce2563530796e068f02327c3f3a8993b891d470489e006"}
Oct 31 17:38:02.477512 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:38:02.477518    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-25kdb" event={"ID":"6ec6accb-4b2a-45a8-bab2-fe65f2f5df2d","Type":"ContainerStarted","Data":"c7b22e980dc77eb919d97c899ab99174f89c58c05a99a1385ff91d76d8886cd1"}
Oct 31 17:38:02.522870 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:38:02.522819    4276 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-25kdb" podStartSLOduration=1.522805162 podStartE2EDuration="1.522805162s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 17:38:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 17:38:02.520020974 +0000 UTC m=+645.317765903" watchObservedRunningTime="2026-10-31 17:38:02.522805162 +0000 UTC m=+645.320550089"
Oct 31 17:38:02.692988 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:38:02.692923    4276 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-25kdb"
Oct 31 17:38:03.381523 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:38:03.379738    4276 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-ee816e/premium-w0-ee816e-kserve-65c86cd6db-lnt9p"
Oct 31 17:38:03.392798 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:38:03.392771    4276 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-ee816e/premium-w0-ee816e-kserve-65c86cd6db-lnt9p"
Oct 31 17:38:03.456956 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:38:03.456924    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:38:03.522633 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:38:03.522609    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-ee816e/premium-w0-ee816e-kserve-65c86cd6db-lnt9p"
Oct 31 17:38:03.694586 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:38:03.694497    4276 patch_prober.go:29] interesting pod/e2e-worker-w6-d80080-openshift-default-956cddf57-25kdb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body=
Oct 31 17:38:03.694586 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:38:03.694549    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-25kdb" podUID="6ec6accb-4b2a-45a8-bab2-fe65f2f5df2d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused"
Oct 31 17:38:04.692962 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:38:04.692929    4276 patch_prober.go:29] interesting pod/e2e-worker-w6-d80080-openshift-default-956cddf57-25kdb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body=
Oct 31 17:38:04.693366 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:38:04.692987    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-25kdb" podUID="6ec6accb-4b2a-45a8-bab2-fe65f2f5df2d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused"
Oct 31 17:38:05.176010 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:38:05.175969    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:38:05.692966 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:38:05.692928    4276 patch_prober.go:29] interesting pod/e2e-worker-w6-d80080-openshift-default-956cddf57-25kdb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body=
Oct 31 17:38:05.693435 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:38:05.692989    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-25kdb" podUID="6ec6accb-4b2a-45a8-bab2-fe65f2f5df2d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused"
Oct 31 17:38:06.670467 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:38:06.667861    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:38:06.693002 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:38:06.692974    4276 patch_prober.go:29] interesting pod/e2e-worker-w6-d80080-openshift-default-956cddf57-25kdb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body=
Oct 31 17:38:06.693581 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:38:06.693540    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-25kdb" podUID="6ec6accb-4b2a-45a8-bab2-fe65f2f5df2d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused"
Oct 31 17:38:07.693240 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:38:07.693208    4276 patch_prober.go:29] interesting pod/e2e-worker-w6-d80080-openshift-default-956cddf57-25kdb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body=
Oct 31 17:38:07.693700 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:38:07.693261    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-25kdb" podUID="6ec6accb-4b2a-45a8-bab2-fe65f2f5df2d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused"
Oct 31 17:38:08.152651 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:38:08.152603    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:38:09.283425 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:38:09.283394    4276 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-25kdb"
Oct 31 17:38:09.283872 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:38:09.283749    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-25kdb"
Oct 31 17:38:09.285046 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:38:09.285024    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-25kdb"
Oct 31 17:38:09.800048 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:38:09.799986    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:38:11.364057 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:38:11.364021    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:38:12.234003 ip-10-0-129-127 kubenswrapper[4276]: E1031 17:38:12.233962    4276 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tg5qk" containerName="extractor"
Oct 31 17:38:12.849856 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:38:12.849823    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:38:14.329646 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:38:14.329588    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:38:15.996068 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:38:15.994725    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:38:16.652001 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:38:16.651919    4276 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-wm2s9"]
Oct 31 17:38:16.655863 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:38:16.655844    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-wm2s9"
Oct 31 17:38:16.672660 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:38:16.672636    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w0-ee816e-openshift-default-dockercfg-mnsfm\""
Oct 31 17:38:16.688884 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:38:16.688858    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-wm2s9"]
Oct 31 17:38:16.736116 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:38:16.736090    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/34243752-91ce-4e92-b0a6-72490b1f7c9e-istiod-ca-cert\") pod \"e2e-worker-w0-ee816e-openshift-default-c4db85b5c-wm2s9\" (UID: \"34243752-91ce-4e92-b0a6-72490b1f7c9e\") " pod="openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-wm2s9"
Oct 31 17:38:16.736224 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:38:16.736152    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/34243752-91ce-4e92-b0a6-72490b1f7c9e-workload-certs\") pod \"e2e-worker-w0-ee816e-openshift-default-c4db85b5c-wm2s9\" (UID: \"34243752-91ce-4e92-b0a6-72490b1f7c9e\") " pod="openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-wm2s9"
Oct 31 17:38:16.736224 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:38:16.736205    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/34243752-91ce-4e92-b0a6-72490b1f7c9e-istio-envoy\") pod \"e2e-worker-w0-ee816e-openshift-default-c4db85b5c-wm2s9\" (UID: \"34243752-91ce-4e92-b0a6-72490b1f7c9e\") " pod="openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-wm2s9"
Oct 31 17:38:16.736322 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:38:16.736307    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/34243752-91ce-4e92-b0a6-72490b1f7c9e-istio-data\") pod \"e2e-worker-w0-ee816e-openshift-default-c4db85b5c-wm2s9\" (UID: \"34243752-91ce-4e92-b0a6-72490b1f7c9e\") " pod="openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-wm2s9"
Oct 31 17:38:16.736357 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:38:16.736331    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/34243752-91ce-4e92-b0a6-72490b1f7c9e-credential-socket\") pod \"e2e-worker-w0-ee816e-openshift-default-c4db85b5c-wm2s9\" (UID: \"34243752-91ce-4e92-b0a6-72490b1f7c9e\") " pod="openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-wm2s9"
Oct 31 17:38:16.736410 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:38:16.736395    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/34243752-91ce-4e92-b0a6-72490b1f7c9e-istio-podinfo\") pod \"e2e-worker-w0-ee816e-openshift-default-c4db85b5c-wm2s9\" (UID: \"34243752-91ce-4e92-b0a6-72490b1f7c9e\") " pod="openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-wm2s9"
Oct 31 17:38:16.736481 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:38:16.736421    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/34243752-91ce-4e92-b0a6-72490b1f7c9e-workload-socket\") pod \"e2e-worker-w0-ee816e-openshift-default-c4db85b5c-wm2s9\" (UID: \"34243752-91ce-4e92-b0a6-72490b1f7c9e\") " pod="openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-wm2s9"
Oct 31 17:38:16.736522 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:38:16.736482    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tzwwp\" (UniqueName: \"kubernetes.io/projected/34243752-91ce-4e92-b0a6-72490b1f7c9e-kube-api-access-tzwwp\") pod \"e2e-worker-w0-ee816e-openshift-default-c4db85b5c-wm2s9\" (UID: \"34243752-91ce-4e92-b0a6-72490b1f7c9e\") " pod="openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-wm2s9"
Oct 31 17:38:16.736556 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:38:16.736518    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/34243752-91ce-4e92-b0a6-72490b1f7c9e-istio-token\") pod \"e2e-worker-w0-ee816e-openshift-default-c4db85b5c-wm2s9\" (UID: \"34243752-91ce-4e92-b0a6-72490b1f7c9e\") " pod="openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-wm2s9"
Oct 31 17:38:16.837045 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:38:16.837013    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/34243752-91ce-4e92-b0a6-72490b1f7c9e-workload-certs\") pod \"e2e-worker-w0-ee816e-openshift-default-c4db85b5c-wm2s9\" (UID: \"34243752-91ce-4e92-b0a6-72490b1f7c9e\") " pod="openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-wm2s9"
Oct 31 17:38:16.837045 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:38:16.837044    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/34243752-91ce-4e92-b0a6-72490b1f7c9e-istio-envoy\") pod \"e2e-worker-w0-ee816e-openshift-default-c4db85b5c-wm2s9\" (UID: \"34243752-91ce-4e92-b0a6-72490b1f7c9e\") " pod="openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-wm2s9"
Oct 31 17:38:16.837241 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:38:16.837068    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/34243752-91ce-4e92-b0a6-72490b1f7c9e-istio-data\") pod \"e2e-worker-w0-ee816e-openshift-default-c4db85b5c-wm2s9\" (UID: \"34243752-91ce-4e92-b0a6-72490b1f7c9e\") " pod="openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-wm2s9"
Oct 31 17:38:16.837241 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:38:16.837087    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/34243752-91ce-4e92-b0a6-72490b1f7c9e-credential-socket\") pod \"e2e-worker-w0-ee816e-openshift-default-c4db85b5c-wm2s9\" (UID: \"34243752-91ce-4e92-b0a6-72490b1f7c9e\") " pod="openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-wm2s9"
Oct 31 17:38:16.837241 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:38:16.837146    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/34243752-91ce-4e92-b0a6-72490b1f7c9e-istio-podinfo\") pod \"e2e-worker-w0-ee816e-openshift-default-c4db85b5c-wm2s9\" (UID: \"34243752-91ce-4e92-b0a6-72490b1f7c9e\") " pod="openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-wm2s9"
Oct 31 17:38:16.837241 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:38:16.837182    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/34243752-91ce-4e92-b0a6-72490b1f7c9e-workload-socket\") pod \"e2e-worker-w0-ee816e-openshift-default-c4db85b5c-wm2s9\" (UID: \"34243752-91ce-4e92-b0a6-72490b1f7c9e\") " pod="openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-wm2s9"
Oct 31 17:38:16.837241 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:38:16.837217    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tzwwp\" (UniqueName: \"kubernetes.io/projected/34243752-91ce-4e92-b0a6-72490b1f7c9e-kube-api-access-tzwwp\") pod \"e2e-worker-w0-ee816e-openshift-default-c4db85b5c-wm2s9\" (UID: \"34243752-91ce-4e92-b0a6-72490b1f7c9e\") " pod="openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-wm2s9"
Oct 31 17:38:16.837521 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:38:16.837279    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/34243752-91ce-4e92-b0a6-72490b1f7c9e-istio-token\") pod \"e2e-worker-w0-ee816e-openshift-default-c4db85b5c-wm2s9\" (UID: \"34243752-91ce-4e92-b0a6-72490b1f7c9e\") " pod="openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-wm2s9"
Oct 31 17:38:16.837521 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:38:16.837392    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/34243752-91ce-4e92-b0a6-72490b1f7c9e-istiod-ca-cert\") pod \"e2e-worker-w0-ee816e-openshift-default-c4db85b5c-wm2s9\" (UID: \"34243752-91ce-4e92-b0a6-72490b1f7c9e\") " pod="openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-wm2s9"
Oct 31 17:38:16.837521 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:38:16.837477    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/34243752-91ce-4e92-b0a6-72490b1f7c9e-workload-certs\") pod \"e2e-worker-w0-ee816e-openshift-default-c4db85b5c-wm2s9\" (UID: \"34243752-91ce-4e92-b0a6-72490b1f7c9e\") " pod="openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-wm2s9"
Oct 31 17:38:16.837673 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:38:16.837550    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/34243752-91ce-4e92-b0a6-72490b1f7c9e-workload-socket\") pod \"e2e-worker-w0-ee816e-openshift-default-c4db85b5c-wm2s9\" (UID: \"34243752-91ce-4e92-b0a6-72490b1f7c9e\") " pod="openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-wm2s9"
Oct 31 17:38:16.837673 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:38:16.837606    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/34243752-91ce-4e92-b0a6-72490b1f7c9e-credential-socket\") pod \"e2e-worker-w0-ee816e-openshift-default-c4db85b5c-wm2s9\" (UID: \"34243752-91ce-4e92-b0a6-72490b1f7c9e\") " pod="openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-wm2s9"
Oct 31 17:38:16.837906 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:38:16.837886    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/34243752-91ce-4e92-b0a6-72490b1f7c9e-istio-data\") pod \"e2e-worker-w0-ee816e-openshift-default-c4db85b5c-wm2s9\" (UID: \"34243752-91ce-4e92-b0a6-72490b1f7c9e\") " pod="openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-wm2s9"
Oct 31 17:38:16.838033 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:38:16.838015    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/34243752-91ce-4e92-b0a6-72490b1f7c9e-istiod-ca-cert\") pod \"e2e-worker-w0-ee816e-openshift-default-c4db85b5c-wm2s9\" (UID: \"34243752-91ce-4e92-b0a6-72490b1f7c9e\") " pod="openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-wm2s9"
Oct 31 17:38:16.839412 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:38:16.839389    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/34243752-91ce-4e92-b0a6-72490b1f7c9e-istio-envoy\") pod \"e2e-worker-w0-ee816e-openshift-default-c4db85b5c-wm2s9\" (UID: \"34243752-91ce-4e92-b0a6-72490b1f7c9e\") " pod="openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-wm2s9"
Oct 31 17:38:16.839585 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:38:16.839571    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/34243752-91ce-4e92-b0a6-72490b1f7c9e-istio-podinfo\") pod \"e2e-worker-w0-ee816e-openshift-default-c4db85b5c-wm2s9\" (UID: \"34243752-91ce-4e92-b0a6-72490b1f7c9e\") " pod="openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-wm2s9"
Oct 31 17:38:16.857641 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:38:16.857617    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tzwwp\" (UniqueName: \"kubernetes.io/projected/34243752-91ce-4e92-b0a6-72490b1f7c9e-kube-api-access-tzwwp\") pod \"e2e-worker-w0-ee816e-openshift-default-c4db85b5c-wm2s9\" (UID: \"34243752-91ce-4e92-b0a6-72490b1f7c9e\") " pod="openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-wm2s9"
Oct 31 17:38:16.857742 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:38:16.857732    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/34243752-91ce-4e92-b0a6-72490b1f7c9e-istio-token\") pod \"e2e-worker-w0-ee816e-openshift-default-c4db85b5c-wm2s9\" (UID: \"34243752-91ce-4e92-b0a6-72490b1f7c9e\") " pod="openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-wm2s9"
Oct 31 17:38:16.969102 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:38:16.969041    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-wm2s9"
Oct 31 17:38:17.282637 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:38:17.282613    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-wm2s9"]
Oct 31 17:38:17.283922 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:38:17.283899    4276 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34243752_91ce_4e92_b0a6_72490b1f7c9e.slice/crio-0a12fed32e7edd00e3ca31f1b5917fc70ded33e995e88ddaddaf85f4e779ba43 WatchSource:0}: Error finding container 0a12fed32e7edd00e3ca31f1b5917fc70ded33e995e88ddaddaf85f4e779ba43: Status 404 returned error can't find the container with id 0a12fed32e7edd00e3ca31f1b5917fc70ded33e995e88ddaddaf85f4e779ba43
Oct 31 17:38:17.286183 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:38:17.286148    4276 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 17:38:17.286292 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:38:17.286229    4276 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 17:38:17.286292 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:38:17.286274    4276 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 17:38:17.544757 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:38:17.544676    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:38:17.609550 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:38:17.609413    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-wm2s9" event={"ID":"34243752-91ce-4e92-b0a6-72490b1f7c9e","Type":"ContainerStarted","Data":"89e2d0c4c006a4779f619c4ee4c592e212bcd6cf3946b5efe9b95513bc818853"}
Oct 31 17:38:17.609550 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:38:17.609505    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-wm2s9" event={"ID":"34243752-91ce-4e92-b0a6-72490b1f7c9e","Type":"ContainerStarted","Data":"0a12fed32e7edd00e3ca31f1b5917fc70ded33e995e88ddaddaf85f4e779ba43"}
Oct 31 17:38:17.651677 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:38:17.651628    4276 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-wm2s9" podStartSLOduration=1.651615095 podStartE2EDuration="1.651615095s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 17:38:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 17:38:17.64985847 +0000 UTC m=+660.447603398" watchObservedRunningTime="2026-10-31 17:38:17.651615095 +0000 UTC m=+660.449360022"
Oct 31 17:38:17.969606 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:38:17.969565    4276 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-wm2s9"
Oct 31 17:38:18.971424 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:38:18.971392    4276 patch_prober.go:29] interesting pod/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-wm2s9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body=
Oct 31 17:38:18.972028 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:38:18.971995    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-wm2s9" podUID="34243752-91ce-4e92-b0a6-72490b1f7c9e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused"
Oct 31 17:38:19.169796 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:38:19.169759    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:38:19.970935 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:38:19.970895    4276 patch_prober.go:29] interesting pod/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-wm2s9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body=
Oct 31 17:38:19.971248 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:38:19.971216    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-wm2s9" podUID="34243752-91ce-4e92-b0a6-72490b1f7c9e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused"
Oct 31 17:38:20.863307 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:38:20.863266    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:38:20.974467 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:38:20.972796    4276 patch_prober.go:29] interesting pod/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-wm2s9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body=
Oct 31 17:38:20.974467 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:38:20.972856    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-wm2s9" podUID="34243752-91ce-4e92-b0a6-72490b1f7c9e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused"
Oct 31 17:38:21.975466 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:38:21.974784    4276 patch_prober.go:29] interesting pod/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-wm2s9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body=
Oct 31 17:38:21.975466 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:38:21.974842    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-wm2s9" podUID="34243752-91ce-4e92-b0a6-72490b1f7c9e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused"
Oct 31 17:38:22.375952 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:38:22.375914    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:38:22.973828 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:38:22.973791    4276 patch_prober.go:29] interesting pod/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-wm2s9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body=
Oct 31 17:38:22.975193 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:38:22.975158    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-wm2s9" podUID="34243752-91ce-4e92-b0a6-72490b1f7c9e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused"
Oct 31 17:38:23.836941 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:38:23.836904    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:38:23.973536 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:38:23.973503    4276 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-wm2s9"
Oct 31 17:38:23.973794 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:38:23.973775    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-wm2s9"
Oct 31 17:38:23.974524 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:38:23.974509    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-wm2s9"
Oct 31 17:38:25.365004 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:38:25.364972    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:38:26.857150 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:38:26.857119    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:38:28.463140 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:38:28.463103    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:38:30.084635 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:38:30.084592    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:38:31.573048 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:38:31.573015    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:38:33.150360 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:38:33.150322    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:38:34.763193 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:38:34.763162    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:38:36.340302 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:38:36.340270    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:38:37.951314 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:38:37.951283    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:38:39.472970 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:38:39.472935    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:38:41.072207 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:38:41.072176    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:38:42.297092 ip-10-0-129-127 kubenswrapper[4276]: E1031 17:38:42.297025    4276 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tg5qk" containerName="extractor"
Oct 31 17:38:42.675826 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:38:42.675741    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:38:44.147090 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:38:44.147057    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:38:45.671382 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:38:45.671338    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:38:47.249903 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:38:47.249875    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:38:48.926473 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:38:48.926426    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:38:50.473201 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:38:50.473170    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:38:52.248494 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:38:52.248416    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:38:53.723381 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:38:53.723344    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:38:55.455485 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:38:55.455423    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:38:56.938514 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:38:56.938485    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:38:58.465090 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:38:58.465046    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:38:59.900555 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:38:59.900520    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:39:01.503629 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:01.503600    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:39:02.544482 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:02.544436    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:39:03.367902 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:03.367868    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:39:05.316467 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:05.311045    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:39:05.465309 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:05.465275    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:39:07.085561 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:07.085526    4276 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7d9897b54c-b8gmm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.34:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 17:39:07.086118 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:07.086083    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-b8gmm" podUID="5530ee8f-be36-43d3-a268-051794343d8b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.34:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 17:39:07.995525 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:07.995488    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:39:09.753024 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:09.752993    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:39:11.336311 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:11.336279    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:39:12.234659 ip-10-0-129-127 kubenswrapper[4276]: E1031 17:39:12.234625    4276 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tg5qk" containerName="extractor"
Oct 31 17:39:12.953067 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:12.953034    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:39:14.477185 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:14.477153    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:39:15.548488 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:15.548410    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:39:15.664720 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:15.664689    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:39:16.156339 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:16.156308    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:39:17.784258 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:17.784221    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:39:19.494138 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:19.494104    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:39:21.405256 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:21.405226    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:39:23.155308 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:23.155271    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:39:24.083400 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:24.083368    4276 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w6-d80080/distinct-w6-d80080-kserve-664576c8f5-6jc4d"]
Oct 31 17:39:24.087722 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:24.087697    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-d80080/distinct-w6-d80080-kserve-664576c8f5-6jc4d"
Oct 31 17:39:24.105724 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:24.105696    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w6-d80080/distinct-w6-d80080-kserve-664576c8f5-6jc4d"]
Oct 31 17:39:24.115666 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:24.115638    4276 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w6-d80080\"/\"kube-root-ca.crt\""
Oct 31 17:39:24.115666 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:24.115638    4276 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w6-d80080\"/\"openshift-service-ca.crt\""
Oct 31 17:39:24.118613 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:24.118596    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w6-d80080\"/\"distinct-w6-d80080-kserve-self-signed-certs\""
Oct 31 17:39:24.118700 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:24.118623    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w6-d80080\"/\"default-dockercfg-7tv7m\""
Oct 31 17:39:24.167650 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:24.167627    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:39:24.205402 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:24.205374    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b9c2f445-a191-4b00-802a-1b0ded068411-dshm\") pod \"distinct-w6-d80080-kserve-664576c8f5-6jc4d\" (UID: \"b9c2f445-a191-4b00-802a-1b0ded068411\") " pod="e2e-models-e2e-worker-w6-d80080/distinct-w6-d80080-kserve-664576c8f5-6jc4d"
Oct 31 17:39:24.205529 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:24.205427    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b9c2f445-a191-4b00-802a-1b0ded068411-tls-certs\") pod \"distinct-w6-d80080-kserve-664576c8f5-6jc4d\" (UID: \"b9c2f445-a191-4b00-802a-1b0ded068411\") " pod="e2e-models-e2e-worker-w6-d80080/distinct-w6-d80080-kserve-664576c8f5-6jc4d"
Oct 31 17:39:24.205529 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:24.205512    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b9c2f445-a191-4b00-802a-1b0ded068411-home\") pod \"distinct-w6-d80080-kserve-664576c8f5-6jc4d\" (UID: \"b9c2f445-a191-4b00-802a-1b0ded068411\") " pod="e2e-models-e2e-worker-w6-d80080/distinct-w6-d80080-kserve-664576c8f5-6jc4d"
Oct 31 17:39:24.205606 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:24.205546    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b9c2f445-a191-4b00-802a-1b0ded068411-model-cache\") pod \"distinct-w6-d80080-kserve-664576c8f5-6jc4d\" (UID: \"b9c2f445-a191-4b00-802a-1b0ded068411\") " pod="e2e-models-e2e-worker-w6-d80080/distinct-w6-d80080-kserve-664576c8f5-6jc4d"
Oct 31 17:39:24.205641 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:24.205600    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-48bv5\" (UniqueName: \"kubernetes.io/projected/b9c2f445-a191-4b00-802a-1b0ded068411-kube-api-access-48bv5\") pod \"distinct-w6-d80080-kserve-664576c8f5-6jc4d\" (UID: \"b9c2f445-a191-4b00-802a-1b0ded068411\") " pod="e2e-models-e2e-worker-w6-d80080/distinct-w6-d80080-kserve-664576c8f5-6jc4d"
Oct 31 17:39:24.205678 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:24.205643    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b9c2f445-a191-4b00-802a-1b0ded068411-tmp-dir\") pod \"distinct-w6-d80080-kserve-664576c8f5-6jc4d\" (UID: \"b9c2f445-a191-4b00-802a-1b0ded068411\") " pod="e2e-models-e2e-worker-w6-d80080/distinct-w6-d80080-kserve-664576c8f5-6jc4d"
Oct 31 17:39:24.312468 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:24.306616    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b9c2f445-a191-4b00-802a-1b0ded068411-dshm\") pod \"distinct-w6-d80080-kserve-664576c8f5-6jc4d\" (UID: \"b9c2f445-a191-4b00-802a-1b0ded068411\") " pod="e2e-models-e2e-worker-w6-d80080/distinct-w6-d80080-kserve-664576c8f5-6jc4d"
Oct 31 17:39:24.312468 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:24.306708    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b9c2f445-a191-4b00-802a-1b0ded068411-tls-certs\") pod \"distinct-w6-d80080-kserve-664576c8f5-6jc4d\" (UID: \"b9c2f445-a191-4b00-802a-1b0ded068411\") " pod="e2e-models-e2e-worker-w6-d80080/distinct-w6-d80080-kserve-664576c8f5-6jc4d"
Oct 31 17:39:24.312468 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:24.306740    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b9c2f445-a191-4b00-802a-1b0ded068411-home\") pod \"distinct-w6-d80080-kserve-664576c8f5-6jc4d\" (UID: \"b9c2f445-a191-4b00-802a-1b0ded068411\") " pod="e2e-models-e2e-worker-w6-d80080/distinct-w6-d80080-kserve-664576c8f5-6jc4d"
Oct 31 17:39:24.312468 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:24.306762    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b9c2f445-a191-4b00-802a-1b0ded068411-model-cache\") pod \"distinct-w6-d80080-kserve-664576c8f5-6jc4d\" (UID: \"b9c2f445-a191-4b00-802a-1b0ded068411\") " pod="e2e-models-e2e-worker-w6-d80080/distinct-w6-d80080-kserve-664576c8f5-6jc4d"
Oct 31 17:39:24.312468 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:24.306807    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-48bv5\" (UniqueName: \"kubernetes.io/projected/b9c2f445-a191-4b00-802a-1b0ded068411-kube-api-access-48bv5\") pod \"distinct-w6-d80080-kserve-664576c8f5-6jc4d\" (UID: \"b9c2f445-a191-4b00-802a-1b0ded068411\") " pod="e2e-models-e2e-worker-w6-d80080/distinct-w6-d80080-kserve-664576c8f5-6jc4d"
Oct 31 17:39:24.312468 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:24.306864    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b9c2f445-a191-4b00-802a-1b0ded068411-tmp-dir\") pod \"distinct-w6-d80080-kserve-664576c8f5-6jc4d\" (UID: \"b9c2f445-a191-4b00-802a-1b0ded068411\") " pod="e2e-models-e2e-worker-w6-d80080/distinct-w6-d80080-kserve-664576c8f5-6jc4d"
Oct 31 17:39:24.312468 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:24.307349    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b9c2f445-a191-4b00-802a-1b0ded068411-tmp-dir\") pod \"distinct-w6-d80080-kserve-664576c8f5-6jc4d\" (UID: \"b9c2f445-a191-4b00-802a-1b0ded068411\") " pod="e2e-models-e2e-worker-w6-d80080/distinct-w6-d80080-kserve-664576c8f5-6jc4d"
Oct 31 17:39:24.312468 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:24.309770    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b9c2f445-a191-4b00-802a-1b0ded068411-home\") pod \"distinct-w6-d80080-kserve-664576c8f5-6jc4d\" (UID: \"b9c2f445-a191-4b00-802a-1b0ded068411\") " pod="e2e-models-e2e-worker-w6-d80080/distinct-w6-d80080-kserve-664576c8f5-6jc4d"
Oct 31 17:39:24.312468 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:24.310436    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b9c2f445-a191-4b00-802a-1b0ded068411-model-cache\") pod \"distinct-w6-d80080-kserve-664576c8f5-6jc4d\" (UID: \"b9c2f445-a191-4b00-802a-1b0ded068411\") " pod="e2e-models-e2e-worker-w6-d80080/distinct-w6-d80080-kserve-664576c8f5-6jc4d"
Oct 31 17:39:24.323098 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:24.323067    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b9c2f445-a191-4b00-802a-1b0ded068411-tls-certs\") pod \"distinct-w6-d80080-kserve-664576c8f5-6jc4d\" (UID: \"b9c2f445-a191-4b00-802a-1b0ded068411\") " pod="e2e-models-e2e-worker-w6-d80080/distinct-w6-d80080-kserve-664576c8f5-6jc4d"
Oct 31 17:39:24.327791 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:24.327761    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b9c2f445-a191-4b00-802a-1b0ded068411-dshm\") pod \"distinct-w6-d80080-kserve-664576c8f5-6jc4d\" (UID: \"b9c2f445-a191-4b00-802a-1b0ded068411\") " pod="e2e-models-e2e-worker-w6-d80080/distinct-w6-d80080-kserve-664576c8f5-6jc4d"
Oct 31 17:39:24.334194 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:24.334116    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-48bv5\" (UniqueName: \"kubernetes.io/projected/b9c2f445-a191-4b00-802a-1b0ded068411-kube-api-access-48bv5\") pod \"distinct-w6-d80080-kserve-664576c8f5-6jc4d\" (UID: \"b9c2f445-a191-4b00-802a-1b0ded068411\") " pod="e2e-models-e2e-worker-w6-d80080/distinct-w6-d80080-kserve-664576c8f5-6jc4d"
Oct 31 17:39:24.402649 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:24.402617    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-d80080/distinct-w6-d80080-kserve-664576c8f5-6jc4d"
Oct 31 17:39:24.703606 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:24.703584    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w6-d80080/distinct-w6-d80080-kserve-664576c8f5-6jc4d"]
Oct 31 17:39:24.705076 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:39:24.705042    4276 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb9c2f445_a191_4b00_802a_1b0ded068411.slice/crio-99d360bd5b1b96dd72f3b1c35791cba7629c1f6416af3b08729d9157d1a97097 WatchSource:0}: Error finding container 99d360bd5b1b96dd72f3b1c35791cba7629c1f6416af3b08729d9157d1a97097: Status 404 returned error can't find the container with id 99d360bd5b1b96dd72f3b1c35791cba7629c1f6416af3b08729d9157d1a97097
Oct 31 17:39:24.830792 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:24.830764    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:39:25.330980 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:25.330947    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-d80080/distinct-w6-d80080-kserve-664576c8f5-6jc4d" event={"ID":"b9c2f445-a191-4b00-802a-1b0ded068411","Type":"ContainerStarted","Data":"423c6b9d0c9fafdfabe87587397a88aa7ad3576dc56f7b8b69c86bceb33fbf23"}
Oct 31 17:39:25.331384 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:25.330991    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-d80080/distinct-w6-d80080-kserve-664576c8f5-6jc4d" event={"ID":"b9c2f445-a191-4b00-802a-1b0ded068411","Type":"ContainerStarted","Data":"99d360bd5b1b96dd72f3b1c35791cba7629c1f6416af3b08729d9157d1a97097"}
Oct 31 17:39:25.373116 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:25.373068    4276 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w6-d80080/distinct-w6-d80080-kserve-664576c8f5-6jc4d" podStartSLOduration=1.373051556 podStartE2EDuration="1.373051556s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 17:39:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 17:39:25.371188429 +0000 UTC m=+728.168933357" watchObservedRunningTime="2026-10-31 17:39:25.373051556 +0000 UTC m=+728.170796484"
Oct 31 17:39:25.403313 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:25.403284    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-d80080/distinct-w6-d80080-kserve-664576c8f5-6jc4d"
Oct 31 17:39:26.520636 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:26.520600    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:39:27.830070 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:27.830040    4276 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-98fcj"]
Oct 31 17:39:27.834109 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:27.834088    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-98fcj"
Oct 31 17:39:27.854249 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:27.854225    4276 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-p4v97"]
Oct 31 17:39:27.857926 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:27.857905    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-p4v97"
Oct 31 17:39:27.869760 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:27.869734    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-98fcj"]
Oct 31 17:39:27.876805 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:27.876781    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-p4v97"]
Oct 31 17:39:27.948391 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:27.948357    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ffeb078-cb4d-4293-b7fb-eb6fd305d96b-credential-socket\") pod \"maas-default-gateway-openshift-default-7d9897b54c-p4v97\" (UID: \"3ffeb078-cb4d-4293-b7fb-eb6fd305d96b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-p4v97"
Oct 31 17:39:27.948550 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:27.948395    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9421021e-be39-4906-8fae-da1ce9ba4047-istio-data\") pod \"maas-default-gateway-openshift-default-7d9897b54c-98fcj\" (UID: \"9421021e-be39-4906-8fae-da1ce9ba4047\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-98fcj"
Oct 31 17:39:27.948550 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:27.948426    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3ffeb078-cb4d-4293-b7fb-eb6fd305d96b-istio-envoy\") pod \"maas-default-gateway-openshift-default-7d9897b54c-p4v97\" (UID: \"3ffeb078-cb4d-4293-b7fb-eb6fd305d96b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-p4v97"
Oct 31 17:39:27.948550 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:27.948535    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9421021e-be39-4906-8fae-da1ce9ba4047-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7d9897b54c-98fcj\" (UID: \"9421021e-be39-4906-8fae-da1ce9ba4047\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-98fcj"
Oct 31 17:39:27.948664 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:27.948587    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4bkjh\" (UniqueName: \"kubernetes.io/projected/9421021e-be39-4906-8fae-da1ce9ba4047-kube-api-access-4bkjh\") pod \"maas-default-gateway-openshift-default-7d9897b54c-98fcj\" (UID: \"9421021e-be39-4906-8fae-da1ce9ba4047\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-98fcj"
Oct 31 17:39:27.948664 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:27.948652    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9421021e-be39-4906-8fae-da1ce9ba4047-istio-envoy\") pod \"maas-default-gateway-openshift-default-7d9897b54c-98fcj\" (UID: \"9421021e-be39-4906-8fae-da1ce9ba4047\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-98fcj"
Oct 31 17:39:27.948728 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:27.948714    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3ffeb078-cb4d-4293-b7fb-eb6fd305d96b-workload-certs\") pod \"maas-default-gateway-openshift-default-7d9897b54c-p4v97\" (UID: \"3ffeb078-cb4d-4293-b7fb-eb6fd305d96b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-p4v97"
Oct 31 17:39:27.948761 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:27.948732    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3ffeb078-cb4d-4293-b7fb-eb6fd305d96b-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7d9897b54c-p4v97\" (UID: \"3ffeb078-cb4d-4293-b7fb-eb6fd305d96b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-p4v97"
Oct 31 17:39:27.948761 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:27.948748    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9421021e-be39-4906-8fae-da1ce9ba4047-workload-certs\") pod \"maas-default-gateway-openshift-default-7d9897b54c-98fcj\" (UID: \"9421021e-be39-4906-8fae-da1ce9ba4047\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-98fcj"
Oct 31 17:39:27.948841 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:27.948764    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9421021e-be39-4906-8fae-da1ce9ba4047-credential-socket\") pod \"maas-default-gateway-openshift-default-7d9897b54c-98fcj\" (UID: \"9421021e-be39-4906-8fae-da1ce9ba4047\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-98fcj"
Oct 31 17:39:27.948841 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:27.948784    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9421021e-be39-4906-8fae-da1ce9ba4047-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7d9897b54c-98fcj\" (UID: \"9421021e-be39-4906-8fae-da1ce9ba4047\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-98fcj"
Oct 31 17:39:27.948906 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:27.948862    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3ffeb078-cb4d-4293-b7fb-eb6fd305d96b-istio-data\") pod \"maas-default-gateway-openshift-default-7d9897b54c-p4v97\" (UID: \"3ffeb078-cb4d-4293-b7fb-eb6fd305d96b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-p4v97"
Oct 31 17:39:27.948906 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:27.948902    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9421021e-be39-4906-8fae-da1ce9ba4047-workload-socket\") pod \"maas-default-gateway-openshift-default-7d9897b54c-98fcj\" (UID: \"9421021e-be39-4906-8fae-da1ce9ba4047\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-98fcj"
Oct 31 17:39:27.949024 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:27.948917    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3ffeb078-cb4d-4293-b7fb-eb6fd305d96b-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7d9897b54c-p4v97\" (UID: \"3ffeb078-cb4d-4293-b7fb-eb6fd305d96b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-p4v97"
Oct 31 17:39:27.949024 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:27.948994    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ffeb078-cb4d-4293-b7fb-eb6fd305d96b-workload-socket\") pod \"maas-default-gateway-openshift-default-7d9897b54c-p4v97\" (UID: \"3ffeb078-cb4d-4293-b7fb-eb6fd305d96b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-p4v97"
Oct 31 17:39:27.949097 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:27.949047    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9421021e-be39-4906-8fae-da1ce9ba4047-istio-token\") pod \"maas-default-gateway-openshift-default-7d9897b54c-98fcj\" (UID: \"9421021e-be39-4906-8fae-da1ce9ba4047\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-98fcj"
Oct 31 17:39:27.949097 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:27.949082    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9ml2z\" (UniqueName: \"kubernetes.io/projected/3ffeb078-cb4d-4293-b7fb-eb6fd305d96b-kube-api-access-9ml2z\") pod \"maas-default-gateway-openshift-default-7d9897b54c-p4v97\" (UID: \"3ffeb078-cb4d-4293-b7fb-eb6fd305d96b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-p4v97"
Oct 31 17:39:27.949176 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:27.949114    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3ffeb078-cb4d-4293-b7fb-eb6fd305d96b-istio-token\") pod \"maas-default-gateway-openshift-default-7d9897b54c-p4v97\" (UID: \"3ffeb078-cb4d-4293-b7fb-eb6fd305d96b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-p4v97"
Oct 31 17:39:28.050014 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:28.049983    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4bkjh\" (UniqueName: \"kubernetes.io/projected/9421021e-be39-4906-8fae-da1ce9ba4047-kube-api-access-4bkjh\") pod \"maas-default-gateway-openshift-default-7d9897b54c-98fcj\" (UID: \"9421021e-be39-4906-8fae-da1ce9ba4047\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-98fcj"
Oct 31 17:39:28.050203 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:28.050022    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9421021e-be39-4906-8fae-da1ce9ba4047-istio-envoy\") pod \"maas-default-gateway-openshift-default-7d9897b54c-98fcj\" (UID: \"9421021e-be39-4906-8fae-da1ce9ba4047\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-98fcj"
Oct 31 17:39:28.050203 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:28.050050    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3ffeb078-cb4d-4293-b7fb-eb6fd305d96b-workload-certs\") pod \"maas-default-gateway-openshift-default-7d9897b54c-p4v97\" (UID: \"3ffeb078-cb4d-4293-b7fb-eb6fd305d96b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-p4v97"
Oct 31 17:39:28.050203 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:28.050169    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3ffeb078-cb4d-4293-b7fb-eb6fd305d96b-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7d9897b54c-p4v97\" (UID: \"3ffeb078-cb4d-4293-b7fb-eb6fd305d96b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-p4v97"
Oct 31 17:39:28.050366 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:28.050206    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9421021e-be39-4906-8fae-da1ce9ba4047-workload-certs\") pod \"maas-default-gateway-openshift-default-7d9897b54c-98fcj\" (UID: \"9421021e-be39-4906-8fae-da1ce9ba4047\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-98fcj"
Oct 31 17:39:28.050366 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:28.050235    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9421021e-be39-4906-8fae-da1ce9ba4047-credential-socket\") pod \"maas-default-gateway-openshift-default-7d9897b54c-98fcj\" (UID: \"9421021e-be39-4906-8fae-da1ce9ba4047\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-98fcj"
Oct 31 17:39:28.050366 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:28.050268    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9421021e-be39-4906-8fae-da1ce9ba4047-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7d9897b54c-98fcj\" (UID: \"9421021e-be39-4906-8fae-da1ce9ba4047\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-98fcj"
Oct 31 17:39:28.050366 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:28.050326    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3ffeb078-cb4d-4293-b7fb-eb6fd305d96b-istio-data\") pod \"maas-default-gateway-openshift-default-7d9897b54c-p4v97\" (UID: \"3ffeb078-cb4d-4293-b7fb-eb6fd305d96b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-p4v97"
Oct 31 17:39:28.050582 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:28.050398    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9421021e-be39-4906-8fae-da1ce9ba4047-workload-socket\") pod \"maas-default-gateway-openshift-default-7d9897b54c-98fcj\" (UID: \"9421021e-be39-4906-8fae-da1ce9ba4047\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-98fcj"
Oct 31 17:39:28.050582 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:28.050412    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3ffeb078-cb4d-4293-b7fb-eb6fd305d96b-workload-certs\") pod \"maas-default-gateway-openshift-default-7d9897b54c-p4v97\" (UID: \"3ffeb078-cb4d-4293-b7fb-eb6fd305d96b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-p4v97"
Oct 31 17:39:28.050582 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:28.050422    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3ffeb078-cb4d-4293-b7fb-eb6fd305d96b-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7d9897b54c-p4v97\" (UID: \"3ffeb078-cb4d-4293-b7fb-eb6fd305d96b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-p4v97"
Oct 31 17:39:28.050582 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:28.050480    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ffeb078-cb4d-4293-b7fb-eb6fd305d96b-workload-socket\") pod \"maas-default-gateway-openshift-default-7d9897b54c-p4v97\" (UID: \"3ffeb078-cb4d-4293-b7fb-eb6fd305d96b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-p4v97"
Oct 31 17:39:28.050582 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:28.050555    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9421021e-be39-4906-8fae-da1ce9ba4047-istio-token\") pod \"maas-default-gateway-openshift-default-7d9897b54c-98fcj\" (UID: \"9421021e-be39-4906-8fae-da1ce9ba4047\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-98fcj"
Oct 31 17:39:28.050582 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:28.050570    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9421021e-be39-4906-8fae-da1ce9ba4047-workload-certs\") pod \"maas-default-gateway-openshift-default-7d9897b54c-98fcj\" (UID: \"9421021e-be39-4906-8fae-da1ce9ba4047\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-98fcj"
Oct 31 17:39:28.050883 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:28.050590    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9ml2z\" (UniqueName: \"kubernetes.io/projected/3ffeb078-cb4d-4293-b7fb-eb6fd305d96b-kube-api-access-9ml2z\") pod \"maas-default-gateway-openshift-default-7d9897b54c-p4v97\" (UID: \"3ffeb078-cb4d-4293-b7fb-eb6fd305d96b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-p4v97"
Oct 31 17:39:28.050883 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:28.050667    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3ffeb078-cb4d-4293-b7fb-eb6fd305d96b-istio-token\") pod \"maas-default-gateway-openshift-default-7d9897b54c-p4v97\" (UID: \"3ffeb078-cb4d-4293-b7fb-eb6fd305d96b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-p4v97"
Oct 31 17:39:28.050883 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:28.050697    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3ffeb078-cb4d-4293-b7fb-eb6fd305d96b-istio-data\") pod \"maas-default-gateway-openshift-default-7d9897b54c-p4v97\" (UID: \"3ffeb078-cb4d-4293-b7fb-eb6fd305d96b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-p4v97"
Oct 31 17:39:28.050883 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:28.050705    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ffeb078-cb4d-4293-b7fb-eb6fd305d96b-credential-socket\") pod \"maas-default-gateway-openshift-default-7d9897b54c-p4v97\" (UID: \"3ffeb078-cb4d-4293-b7fb-eb6fd305d96b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-p4v97"
Oct 31 17:39:28.050883 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:28.050744    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9421021e-be39-4906-8fae-da1ce9ba4047-istio-data\") pod \"maas-default-gateway-openshift-default-7d9897b54c-98fcj\" (UID: \"9421021e-be39-4906-8fae-da1ce9ba4047\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-98fcj"
Oct 31 17:39:28.050883 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:28.050779    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3ffeb078-cb4d-4293-b7fb-eb6fd305d96b-istio-envoy\") pod \"maas-default-gateway-openshift-default-7d9897b54c-p4v97\" (UID: \"3ffeb078-cb4d-4293-b7fb-eb6fd305d96b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-p4v97"
Oct 31 17:39:28.051181 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:28.050955    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9421021e-be39-4906-8fae-da1ce9ba4047-credential-socket\") pod \"maas-default-gateway-openshift-default-7d9897b54c-98fcj\" (UID: \"9421021e-be39-4906-8fae-da1ce9ba4047\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-98fcj"
Oct 31 17:39:28.051181 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:28.050963    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ffeb078-cb4d-4293-b7fb-eb6fd305d96b-credential-socket\") pod \"maas-default-gateway-openshift-default-7d9897b54c-p4v97\" (UID: \"3ffeb078-cb4d-4293-b7fb-eb6fd305d96b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-p4v97"
Oct 31 17:39:28.051181 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:28.051130    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ffeb078-cb4d-4293-b7fb-eb6fd305d96b-workload-socket\") pod \"maas-default-gateway-openshift-default-7d9897b54c-p4v97\" (UID: \"3ffeb078-cb4d-4293-b7fb-eb6fd305d96b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-p4v97"
Oct 31 17:39:28.051181 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:28.051134    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9421021e-be39-4906-8fae-da1ce9ba4047-workload-socket\") pod \"maas-default-gateway-openshift-default-7d9897b54c-98fcj\" (UID: \"9421021e-be39-4906-8fae-da1ce9ba4047\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-98fcj"
Oct 31 17:39:28.051181 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:28.051173    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9421021e-be39-4906-8fae-da1ce9ba4047-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7d9897b54c-98fcj\" (UID: \"9421021e-be39-4906-8fae-da1ce9ba4047\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-98fcj"
Oct 31 17:39:28.051426 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:28.051285    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9421021e-be39-4906-8fae-da1ce9ba4047-istio-data\") pod \"maas-default-gateway-openshift-default-7d9897b54c-98fcj\" (UID: \"9421021e-be39-4906-8fae-da1ce9ba4047\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-98fcj"
Oct 31 17:39:28.051958 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:28.051931    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3ffeb078-cb4d-4293-b7fb-eb6fd305d96b-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7d9897b54c-p4v97\" (UID: \"3ffeb078-cb4d-4293-b7fb-eb6fd305d96b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-p4v97"
Oct 31 17:39:28.052137 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:28.051942    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9421021e-be39-4906-8fae-da1ce9ba4047-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7d9897b54c-98fcj\" (UID: \"9421021e-be39-4906-8fae-da1ce9ba4047\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-98fcj"
Oct 31 17:39:28.053149 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:28.053054    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9421021e-be39-4906-8fae-da1ce9ba4047-istio-envoy\") pod \"maas-default-gateway-openshift-default-7d9897b54c-98fcj\" (UID: \"9421021e-be39-4906-8fae-da1ce9ba4047\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-98fcj"
Oct 31 17:39:28.053340 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:28.053322    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3ffeb078-cb4d-4293-b7fb-eb6fd305d96b-istio-envoy\") pod \"maas-default-gateway-openshift-default-7d9897b54c-p4v97\" (UID: \"3ffeb078-cb4d-4293-b7fb-eb6fd305d96b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-p4v97"
Oct 31 17:39:28.053625 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:28.053603    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9421021e-be39-4906-8fae-da1ce9ba4047-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7d9897b54c-98fcj\" (UID: \"9421021e-be39-4906-8fae-da1ce9ba4047\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-98fcj"
Oct 31 17:39:28.053725 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:28.053607    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3ffeb078-cb4d-4293-b7fb-eb6fd305d96b-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7d9897b54c-p4v97\" (UID: \"3ffeb078-cb4d-4293-b7fb-eb6fd305d96b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-p4v97"
Oct 31 17:39:28.070495 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:28.070474    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4bkjh\" (UniqueName: \"kubernetes.io/projected/9421021e-be39-4906-8fae-da1ce9ba4047-kube-api-access-4bkjh\") pod \"maas-default-gateway-openshift-default-7d9897b54c-98fcj\" (UID: \"9421021e-be39-4906-8fae-da1ce9ba4047\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-98fcj"
Oct 31 17:39:28.071095 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:28.071072    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9ml2z\" (UniqueName: \"kubernetes.io/projected/3ffeb078-cb4d-4293-b7fb-eb6fd305d96b-kube-api-access-9ml2z\") pod \"maas-default-gateway-openshift-default-7d9897b54c-p4v97\" (UID: \"3ffeb078-cb4d-4293-b7fb-eb6fd305d96b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-p4v97"
Oct 31 17:39:28.071163 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:28.071139    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3ffeb078-cb4d-4293-b7fb-eb6fd305d96b-istio-token\") pod \"maas-default-gateway-openshift-default-7d9897b54c-p4v97\" (UID: \"3ffeb078-cb4d-4293-b7fb-eb6fd305d96b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-p4v97"
Oct 31 17:39:28.071738 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:28.071722    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9421021e-be39-4906-8fae-da1ce9ba4047-istio-token\") pod \"maas-default-gateway-openshift-default-7d9897b54c-98fcj\" (UID: \"9421021e-be39-4906-8fae-da1ce9ba4047\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-98fcj"
Oct 31 17:39:28.073433 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:28.073411    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:39:28.148890 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:28.148806    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-98fcj"
Oct 31 17:39:28.181726 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:28.181694    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-p4v97"
Oct 31 17:39:28.468422 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:28.468397    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-98fcj"]
Oct 31 17:39:28.469210 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:39:28.469188    4276 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9421021e_be39_4906_8fae_da1ce9ba4047.slice/crio-4f71fb017120b09237356b1121a762cfcd8630ebc4c21c286da4da1bc83faf51 WatchSource:0}: Error finding container 4f71fb017120b09237356b1121a762cfcd8630ebc4c21c286da4da1bc83faf51: Status 404 returned error can't find the container with id 4f71fb017120b09237356b1121a762cfcd8630ebc4c21c286da4da1bc83faf51
Oct 31 17:39:28.471757 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:28.471723    4276 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 17:39:28.471861 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:28.471783    4276 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 17:39:28.471861 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:28.471816    4276 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 17:39:28.505718 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:28.505696    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-p4v97"]
Oct 31 17:39:28.507514 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:39:28.507489    4276 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ffeb078_cb4d_4293_b7fb_eb6fd305d96b.slice/crio-f6bc08a6cf23ca9722e45d55302c473aedb078e1780cc19d8cf185ba05638f52 WatchSource:0}: Error finding container f6bc08a6cf23ca9722e45d55302c473aedb078e1780cc19d8cf185ba05638f52: Status 404 returned error can't find the container with id f6bc08a6cf23ca9722e45d55302c473aedb078e1780cc19d8cf185ba05638f52
Oct 31 17:39:28.509925 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:28.509895    4276 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 17:39:28.510019 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:28.509968    4276 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 17:39:28.510082 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:28.510014    4276 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 17:39:29.353372 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:29.353331    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-98fcj" event={"ID":"9421021e-be39-4906-8fae-da1ce9ba4047","Type":"ContainerStarted","Data":"a2ff7a0152cad8c7f557ef56464fcb6a4193add625040de70f61fef059507b85"}
Oct 31 17:39:29.354088 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:29.353464    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-98fcj" event={"ID":"9421021e-be39-4906-8fae-da1ce9ba4047","Type":"ContainerStarted","Data":"4f71fb017120b09237356b1121a762cfcd8630ebc4c21c286da4da1bc83faf51"}
Oct 31 17:39:29.354957 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:29.354938    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-p4v97" event={"ID":"3ffeb078-cb4d-4293-b7fb-eb6fd305d96b","Type":"ContainerStarted","Data":"65391aa19fe9c932e942257d5942895467ba014301c465dc91345b6962a56301"}
Oct 31 17:39:29.355053 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:29.354972    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-p4v97" event={"ID":"3ffeb078-cb4d-4293-b7fb-eb6fd305d96b","Type":"ContainerStarted","Data":"f6bc08a6cf23ca9722e45d55302c473aedb078e1780cc19d8cf185ba05638f52"}
Oct 31 17:39:29.411262 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:29.411214    4276 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-98fcj" podStartSLOduration=2.411202492 podStartE2EDuration="2.411202492s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 17:39:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 17:39:29.410734927 +0000 UTC m=+732.208479854" watchObservedRunningTime="2026-10-31 17:39:29.411202492 +0000 UTC m=+732.208947420"
Oct 31 17:39:29.454003 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:29.453957    4276 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-p4v97" podStartSLOduration=2.453944799 podStartE2EDuration="2.453944799s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 17:39:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 17:39:29.453379323 +0000 UTC m=+732.251124250" watchObservedRunningTime="2026-10-31 17:39:29.453944799 +0000 UTC m=+732.251689727"
Oct 31 17:39:29.759480 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:29.759360    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:39:30.149924 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:30.149888    4276 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-98fcj"
Oct 31 17:39:30.167469 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:30.163896    4276 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7d9897b54c-98fcj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body=
Oct 31 17:39:30.167469 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:30.163968    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-98fcj" podUID="9421021e-be39-4906-8fae-da1ce9ba4047" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused"
Oct 31 17:39:30.200468 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:30.189790    4276 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-p4v97"
Oct 31 17:39:30.200468 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:30.191301    4276 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7d9897b54c-p4v97 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body=
Oct 31 17:39:30.200468 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:30.191367    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-p4v97" podUID="3ffeb078-cb4d-4293-b7fb-eb6fd305d96b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused"
Oct 31 17:39:31.149667 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:31.149638    4276 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7d9897b54c-98fcj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body=
Oct 31 17:39:31.150049 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:31.149688    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-98fcj" podUID="9421021e-be39-4906-8fae-da1ce9ba4047" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused"
Oct 31 17:39:31.198210 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:31.198168    4276 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-rfwx5"]
Oct 31 17:39:31.198525 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:31.198496    4276 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7d9897b54c-p4v97 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body=
Oct 31 17:39:31.198729 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:31.198706    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-p4v97" podUID="3ffeb078-cb4d-4293-b7fb-eb6fd305d96b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused"
Oct 31 17:39:31.203331 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:31.203308    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-rfwx5"
Oct 31 17:39:31.215091 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:31.215062    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5155d374-e42a-4c40-9485-935443ede2c5-workload-certs\") pod \"e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-rfwx5\" (UID: \"5155d374-e42a-4c40-9485-935443ede2c5\") " pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-rfwx5"
Oct 31 17:39:31.215288 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:31.215266    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5155d374-e42a-4c40-9485-935443ede2c5-workload-socket\") pod \"e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-rfwx5\" (UID: \"5155d374-e42a-4c40-9485-935443ede2c5\") " pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-rfwx5"
Oct 31 17:39:31.215413 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:31.215396    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5155d374-e42a-4c40-9485-935443ede2c5-credential-socket\") pod \"e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-rfwx5\" (UID: \"5155d374-e42a-4c40-9485-935443ede2c5\") " pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-rfwx5"
Oct 31 17:39:31.215575 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:31.215555    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5155d374-e42a-4c40-9485-935443ede2c5-istio-data\") pod \"e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-rfwx5\" (UID: \"5155d374-e42a-4c40-9485-935443ede2c5\") " pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-rfwx5"
Oct 31 17:39:31.215700 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:31.215683    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pcq9f\" (UniqueName: \"kubernetes.io/projected/5155d374-e42a-4c40-9485-935443ede2c5-kube-api-access-pcq9f\") pod \"e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-rfwx5\" (UID: \"5155d374-e42a-4c40-9485-935443ede2c5\") " pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-rfwx5"
Oct 31 17:39:31.215819 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:31.215804    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5155d374-e42a-4c40-9485-935443ede2c5-istio-podinfo\") pod \"e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-rfwx5\" (UID: \"5155d374-e42a-4c40-9485-935443ede2c5\") " pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-rfwx5"
Oct 31 17:39:31.215929 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:31.215915    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5155d374-e42a-4c40-9485-935443ede2c5-istio-token\") pod \"e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-rfwx5\" (UID: \"5155d374-e42a-4c40-9485-935443ede2c5\") " pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-rfwx5"
Oct 31 17:39:31.216035 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:31.216021    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5155d374-e42a-4c40-9485-935443ede2c5-istiod-ca-cert\") pod \"e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-rfwx5\" (UID: \"5155d374-e42a-4c40-9485-935443ede2c5\") " pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-rfwx5"
Oct 31 17:39:31.216220 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:31.216200    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5155d374-e42a-4c40-9485-935443ede2c5-istio-envoy\") pod \"e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-rfwx5\" (UID: \"5155d374-e42a-4c40-9485-935443ede2c5\") " pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-rfwx5"
Oct 31 17:39:31.246904 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:31.246867    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-rfwx5"]
Oct 31 17:39:31.317254 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:31.317223    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5155d374-e42a-4c40-9485-935443ede2c5-workload-certs\") pod \"e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-rfwx5\" (UID: \"5155d374-e42a-4c40-9485-935443ede2c5\") " pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-rfwx5"
Oct 31 17:39:31.317627 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:31.317607    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5155d374-e42a-4c40-9485-935443ede2c5-workload-certs\") pod \"e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-rfwx5\" (UID: \"5155d374-e42a-4c40-9485-935443ede2c5\") " pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-rfwx5"
Oct 31 17:39:31.317700 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:31.317671    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5155d374-e42a-4c40-9485-935443ede2c5-workload-socket\") pod \"e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-rfwx5\" (UID: \"5155d374-e42a-4c40-9485-935443ede2c5\") " pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-rfwx5"
Oct 31 17:39:31.317755 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:31.317701    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5155d374-e42a-4c40-9485-935443ede2c5-credential-socket\") pod \"e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-rfwx5\" (UID: \"5155d374-e42a-4c40-9485-935443ede2c5\") " pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-rfwx5"
Oct 31 17:39:31.317887 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:31.317866    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5155d374-e42a-4c40-9485-935443ede2c5-istio-data\") pod \"e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-rfwx5\" (UID: \"5155d374-e42a-4c40-9485-935443ede2c5\") " pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-rfwx5"
Oct 31 17:39:31.317939 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:31.317909    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pcq9f\" (UniqueName: \"kubernetes.io/projected/5155d374-e42a-4c40-9485-935443ede2c5-kube-api-access-pcq9f\") pod \"e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-rfwx5\" (UID: \"5155d374-e42a-4c40-9485-935443ede2c5\") " pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-rfwx5"
Oct 31 17:39:31.317975 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:31.317957    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5155d374-e42a-4c40-9485-935443ede2c5-istio-podinfo\") pod \"e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-rfwx5\" (UID: \"5155d374-e42a-4c40-9485-935443ede2c5\") " pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-rfwx5"
Oct 31 17:39:31.318008 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:31.317975    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5155d374-e42a-4c40-9485-935443ede2c5-workload-socket\") pod \"e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-rfwx5\" (UID: \"5155d374-e42a-4c40-9485-935443ede2c5\") " pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-rfwx5"
Oct 31 17:39:31.318186 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:31.318160    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5155d374-e42a-4c40-9485-935443ede2c5-credential-socket\") pod \"e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-rfwx5\" (UID: \"5155d374-e42a-4c40-9485-935443ede2c5\") " pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-rfwx5"
Oct 31 17:39:31.318347 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:31.318175    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5155d374-e42a-4c40-9485-935443ede2c5-istio-data\") pod \"e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-rfwx5\" (UID: \"5155d374-e42a-4c40-9485-935443ede2c5\") " pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-rfwx5"
Oct 31 17:39:31.318347 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:31.318272    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5155d374-e42a-4c40-9485-935443ede2c5-istio-token\") pod \"e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-rfwx5\" (UID: \"5155d374-e42a-4c40-9485-935443ede2c5\") " pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-rfwx5"
Oct 31 17:39:31.318569 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:31.318375    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5155d374-e42a-4c40-9485-935443ede2c5-istiod-ca-cert\") pod \"e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-rfwx5\" (UID: \"5155d374-e42a-4c40-9485-935443ede2c5\") " pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-rfwx5"
Oct 31 17:39:31.318569 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:31.318494    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5155d374-e42a-4c40-9485-935443ede2c5-istio-envoy\") pod \"e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-rfwx5\" (UID: \"5155d374-e42a-4c40-9485-935443ede2c5\") " pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-rfwx5"
Oct 31 17:39:31.318918 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:31.318882    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5155d374-e42a-4c40-9485-935443ede2c5-istiod-ca-cert\") pod \"e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-rfwx5\" (UID: \"5155d374-e42a-4c40-9485-935443ede2c5\") " pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-rfwx5"
Oct 31 17:39:31.321817 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:31.321795    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5155d374-e42a-4c40-9485-935443ede2c5-istio-podinfo\") pod \"e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-rfwx5\" (UID: \"5155d374-e42a-4c40-9485-935443ede2c5\") " pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-rfwx5"
Oct 31 17:39:31.322311 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:31.322284    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5155d374-e42a-4c40-9485-935443ede2c5-istio-envoy\") pod \"e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-rfwx5\" (UID: \"5155d374-e42a-4c40-9485-935443ede2c5\") " pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-rfwx5"
Oct 31 17:39:31.342323 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:31.342303    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5155d374-e42a-4c40-9485-935443ede2c5-istio-token\") pod \"e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-rfwx5\" (UID: \"5155d374-e42a-4c40-9485-935443ede2c5\") " pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-rfwx5"
Oct 31 17:39:31.343557 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:31.343532    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pcq9f\" (UniqueName: \"kubernetes.io/projected/5155d374-e42a-4c40-9485-935443ede2c5-kube-api-access-pcq9f\") pod \"e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-rfwx5\" (UID: \"5155d374-e42a-4c40-9485-935443ede2c5\") " pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-rfwx5"
Oct 31 17:39:31.354536 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:31.354496    4276 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-dx9cd"]
Oct 31 17:39:31.392666 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:31.381747    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-dx9cd"
Oct 31 17:39:31.420668 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:31.414777    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w5-b78c31-openshift-default-dockercfg-7pl9w\""
Oct 31 17:39:31.420668 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:31.419580    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3716260a-fa38-4064-991c-48322b1b48fa-workload-certs\") pod \"e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-dx9cd\" (UID: \"3716260a-fa38-4064-991c-48322b1b48fa\") " pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-dx9cd"
Oct 31 17:39:31.420668 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:31.419630    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3716260a-fa38-4064-991c-48322b1b48fa-istio-podinfo\") pod \"e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-dx9cd\" (UID: \"3716260a-fa38-4064-991c-48322b1b48fa\") " pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-dx9cd"
Oct 31 17:39:31.420668 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:31.419664    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3716260a-fa38-4064-991c-48322b1b48fa-istiod-ca-cert\") pod \"e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-dx9cd\" (UID: \"3716260a-fa38-4064-991c-48322b1b48fa\") " pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-dx9cd"
Oct 31 17:39:31.420668 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:31.419770    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3716260a-fa38-4064-991c-48322b1b48fa-istio-token\") pod \"e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-dx9cd\" (UID: \"3716260a-fa38-4064-991c-48322b1b48fa\") " pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-dx9cd"
Oct 31 17:39:31.420668 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:31.419819    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3716260a-fa38-4064-991c-48322b1b48fa-credential-socket\") pod \"e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-dx9cd\" (UID: \"3716260a-fa38-4064-991c-48322b1b48fa\") " pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-dx9cd"
Oct 31 17:39:31.420668 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:31.419844    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3716260a-fa38-4064-991c-48322b1b48fa-istio-envoy\") pod \"e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-dx9cd\" (UID: \"3716260a-fa38-4064-991c-48322b1b48fa\") " pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-dx9cd"
Oct 31 17:39:31.420668 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:31.419878    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3716260a-fa38-4064-991c-48322b1b48fa-istio-data\") pod \"e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-dx9cd\" (UID: \"3716260a-fa38-4064-991c-48322b1b48fa\") " pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-dx9cd"
Oct 31 17:39:31.420668 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:31.419968    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dnl6c\" (UniqueName: \"kubernetes.io/projected/3716260a-fa38-4064-991c-48322b1b48fa-kube-api-access-dnl6c\") pod \"e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-dx9cd\" (UID: \"3716260a-fa38-4064-991c-48322b1b48fa\") " pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-dx9cd"
Oct 31 17:39:31.420668 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:31.420090    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3716260a-fa38-4064-991c-48322b1b48fa-workload-socket\") pod \"e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-dx9cd\" (UID: \"3716260a-fa38-4064-991c-48322b1b48fa\") " pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-dx9cd"
Oct 31 17:39:31.429694 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:31.429104    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-dx9cd"]
Oct 31 17:39:31.519832 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:31.519488    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-rfwx5"
Oct 31 17:39:31.520904 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:31.520876    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dnl6c\" (UniqueName: \"kubernetes.io/projected/3716260a-fa38-4064-991c-48322b1b48fa-kube-api-access-dnl6c\") pod \"e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-dx9cd\" (UID: \"3716260a-fa38-4064-991c-48322b1b48fa\") " pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-dx9cd"
Oct 31 17:39:31.521065 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:31.521049    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3716260a-fa38-4064-991c-48322b1b48fa-workload-socket\") pod \"e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-dx9cd\" (UID: \"3716260a-fa38-4064-991c-48322b1b48fa\") " pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-dx9cd"
Oct 31 17:39:31.521167 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:31.521152    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3716260a-fa38-4064-991c-48322b1b48fa-workload-certs\") pod \"e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-dx9cd\" (UID: \"3716260a-fa38-4064-991c-48322b1b48fa\") " pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-dx9cd"
Oct 31 17:39:31.521222 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:31.521207    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3716260a-fa38-4064-991c-48322b1b48fa-istio-podinfo\") pod \"e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-dx9cd\" (UID: \"3716260a-fa38-4064-991c-48322b1b48fa\") " pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-dx9cd"
Oct 31 17:39:31.521258 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:31.521240    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3716260a-fa38-4064-991c-48322b1b48fa-istiod-ca-cert\") pod \"e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-dx9cd\" (UID: \"3716260a-fa38-4064-991c-48322b1b48fa\") " pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-dx9cd"
Oct 31 17:39:31.521380 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:31.521367    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3716260a-fa38-4064-991c-48322b1b48fa-istio-token\") pod \"e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-dx9cd\" (UID: \"3716260a-fa38-4064-991c-48322b1b48fa\") " pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-dx9cd"
Oct 31 17:39:31.521429 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:31.521415    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3716260a-fa38-4064-991c-48322b1b48fa-credential-socket\") pod \"e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-dx9cd\" (UID: \"3716260a-fa38-4064-991c-48322b1b48fa\") " pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-dx9cd"
Oct 31 17:39:31.521483 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:31.521454    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3716260a-fa38-4064-991c-48322b1b48fa-istio-envoy\") pod \"e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-dx9cd\" (UID: \"3716260a-fa38-4064-991c-48322b1b48fa\") " pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-dx9cd"
Oct 31 17:39:31.521522 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:31.521482    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3716260a-fa38-4064-991c-48322b1b48fa-istio-data\") pod \"e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-dx9cd\" (UID: \"3716260a-fa38-4064-991c-48322b1b48fa\") " pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-dx9cd"
Oct 31 17:39:31.523934 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:31.522114    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:39:31.523934 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:31.522683    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3716260a-fa38-4064-991c-48322b1b48fa-workload-certs\") pod \"e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-dx9cd\" (UID: \"3716260a-fa38-4064-991c-48322b1b48fa\") " pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-dx9cd"
Oct 31 17:39:31.523934 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:31.523083    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3716260a-fa38-4064-991c-48322b1b48fa-istio-data\") pod \"e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-dx9cd\" (UID: \"3716260a-fa38-4064-991c-48322b1b48fa\") " pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-dx9cd"
Oct 31 17:39:31.523934 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:31.523278    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3716260a-fa38-4064-991c-48322b1b48fa-credential-socket\") pod \"e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-dx9cd\" (UID: \"3716260a-fa38-4064-991c-48322b1b48fa\") " pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-dx9cd"
Oct 31 17:39:31.523934 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:31.523693    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3716260a-fa38-4064-991c-48322b1b48fa-workload-socket\") pod \"e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-dx9cd\" (UID: \"3716260a-fa38-4064-991c-48322b1b48fa\") " pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-dx9cd"
Oct 31 17:39:31.523934 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:31.523886    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3716260a-fa38-4064-991c-48322b1b48fa-istiod-ca-cert\") pod \"e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-dx9cd\" (UID: \"3716260a-fa38-4064-991c-48322b1b48fa\") " pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-dx9cd"
Oct 31 17:39:31.528581 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:31.528556    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3716260a-fa38-4064-991c-48322b1b48fa-istio-envoy\") pod \"e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-dx9cd\" (UID: \"3716260a-fa38-4064-991c-48322b1b48fa\") " pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-dx9cd"
Oct 31 17:39:31.529287 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:31.529210    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3716260a-fa38-4064-991c-48322b1b48fa-istio-podinfo\") pod \"e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-dx9cd\" (UID: \"3716260a-fa38-4064-991c-48322b1b48fa\") " pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-dx9cd"
Oct 31 17:39:31.546960 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:31.546261    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dnl6c\" (UniqueName: \"kubernetes.io/projected/3716260a-fa38-4064-991c-48322b1b48fa-kube-api-access-dnl6c\") pod \"e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-dx9cd\" (UID: \"3716260a-fa38-4064-991c-48322b1b48fa\") " pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-dx9cd"
Oct 31 17:39:31.583859 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:31.583790    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3716260a-fa38-4064-991c-48322b1b48fa-istio-token\") pod \"e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-dx9cd\" (UID: \"3716260a-fa38-4064-991c-48322b1b48fa\") " pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-dx9cd"
Oct 31 17:39:31.736916 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:31.736628    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-dx9cd"
Oct 31 17:39:32.013301 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:32.011517    4276 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-fgvsp"]
Oct 31 17:39:32.015234 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:39:32.015208    4276 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5155d374_e42a_4c40_9485_935443ede2c5.slice/crio-3c1a889c48e1f72718fae760d28b15b474c59131109f7a65ea2bfd263e5b4e06 WatchSource:0}: Error finding container 3c1a889c48e1f72718fae760d28b15b474c59131109f7a65ea2bfd263e5b4e06: Status 404 returned error can't find the container with id 3c1a889c48e1f72718fae760d28b15b474c59131109f7a65ea2bfd263e5b4e06
Oct 31 17:39:32.020204 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:32.016986    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-rfwx5"]
Oct 31 17:39:32.020204 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:32.017095    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-fgvsp"
Oct 31 17:39:32.020204 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:32.018453    4276 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 17:39:32.020204 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:32.018525    4276 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 17:39:32.020204 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:32.018601    4276 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 17:39:32.035836 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:32.035232    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g8pph\" (UniqueName: \"kubernetes.io/projected/d46713a5-7bba-4dfb-b77f-8fae0b39271e-kube-api-access-g8pph\") pod \"e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-fgvsp\" (UID: \"d46713a5-7bba-4dfb-b77f-8fae0b39271e\") " pod="openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-fgvsp"
Oct 31 17:39:32.035836 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:32.035314    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d46713a5-7bba-4dfb-b77f-8fae0b39271e-workload-socket\") pod \"e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-fgvsp\" (UID: \"d46713a5-7bba-4dfb-b77f-8fae0b39271e\") " pod="openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-fgvsp"
Oct 31 17:39:32.035836 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:32.035352    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d46713a5-7bba-4dfb-b77f-8fae0b39271e-istio-data\") pod \"e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-fgvsp\" (UID: \"d46713a5-7bba-4dfb-b77f-8fae0b39271e\") " pod="openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-fgvsp"
Oct 31 17:39:32.035836 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:32.035480    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d46713a5-7bba-4dfb-b77f-8fae0b39271e-istio-token\") pod \"e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-fgvsp\" (UID: \"d46713a5-7bba-4dfb-b77f-8fae0b39271e\") " pod="openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-fgvsp"
Oct 31 17:39:32.035836 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:32.035524    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d46713a5-7bba-4dfb-b77f-8fae0b39271e-istiod-ca-cert\") pod \"e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-fgvsp\" (UID: \"d46713a5-7bba-4dfb-b77f-8fae0b39271e\") " pod="openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-fgvsp"
Oct 31 17:39:32.035836 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:32.035566    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d46713a5-7bba-4dfb-b77f-8fae0b39271e-credential-socket\") pod \"e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-fgvsp\" (UID: \"d46713a5-7bba-4dfb-b77f-8fae0b39271e\") " pod="openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-fgvsp"
Oct 31 17:39:32.035836 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:32.035589    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d46713a5-7bba-4dfb-b77f-8fae0b39271e-istio-envoy\") pod \"e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-fgvsp\" (UID: \"d46713a5-7bba-4dfb-b77f-8fae0b39271e\") " pod="openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-fgvsp"
Oct 31 17:39:32.035836 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:32.035650    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d46713a5-7bba-4dfb-b77f-8fae0b39271e-workload-certs\") pod \"e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-fgvsp\" (UID: \"d46713a5-7bba-4dfb-b77f-8fae0b39271e\") " pod="openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-fgvsp"
Oct 31 17:39:32.035836 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:32.035708    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d46713a5-7bba-4dfb-b77f-8fae0b39271e-istio-podinfo\") pod \"e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-fgvsp\" (UID: \"d46713a5-7bba-4dfb-b77f-8fae0b39271e\") " pod="openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-fgvsp"
Oct 31 17:39:32.042002 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:32.040435    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-fgvsp"]
Oct 31 17:39:32.121270 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:32.121219    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:39:32.136683 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:32.136653    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d46713a5-7bba-4dfb-b77f-8fae0b39271e-istiod-ca-cert\") pod \"e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-fgvsp\" (UID: \"d46713a5-7bba-4dfb-b77f-8fae0b39271e\") " pod="openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-fgvsp"
Oct 31 17:39:32.136817 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:32.136724    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d46713a5-7bba-4dfb-b77f-8fae0b39271e-credential-socket\") pod \"e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-fgvsp\" (UID: \"d46713a5-7bba-4dfb-b77f-8fae0b39271e\") " pod="openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-fgvsp"
Oct 31 17:39:32.136817 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:32.136754    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d46713a5-7bba-4dfb-b77f-8fae0b39271e-istio-envoy\") pod \"e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-fgvsp\" (UID: \"d46713a5-7bba-4dfb-b77f-8fae0b39271e\") " pod="openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-fgvsp"
Oct 31 17:39:32.136887 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:32.136813    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d46713a5-7bba-4dfb-b77f-8fae0b39271e-workload-certs\") pod \"e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-fgvsp\" (UID: \"d46713a5-7bba-4dfb-b77f-8fae0b39271e\") " pod="openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-fgvsp"
Oct 31 17:39:32.136887 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:32.136856    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d46713a5-7bba-4dfb-b77f-8fae0b39271e-istio-podinfo\") pod \"e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-fgvsp\" (UID: \"d46713a5-7bba-4dfb-b77f-8fae0b39271e\") " pod="openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-fgvsp"
Oct 31 17:39:32.136887 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:32.136884    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g8pph\" (UniqueName: \"kubernetes.io/projected/d46713a5-7bba-4dfb-b77f-8fae0b39271e-kube-api-access-g8pph\") pod \"e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-fgvsp\" (UID: \"d46713a5-7bba-4dfb-b77f-8fae0b39271e\") " pod="openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-fgvsp"
Oct 31 17:39:32.136985 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:32.136929    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d46713a5-7bba-4dfb-b77f-8fae0b39271e-workload-socket\") pod \"e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-fgvsp\" (UID: \"d46713a5-7bba-4dfb-b77f-8fae0b39271e\") " pod="openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-fgvsp"
Oct 31 17:39:32.136985 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:32.136954    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d46713a5-7bba-4dfb-b77f-8fae0b39271e-istio-data\") pod \"e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-fgvsp\" (UID: \"d46713a5-7bba-4dfb-b77f-8fae0b39271e\") " pod="openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-fgvsp"
Oct 31 17:39:32.137047 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:32.137015    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d46713a5-7bba-4dfb-b77f-8fae0b39271e-istio-token\") pod \"e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-fgvsp\" (UID: \"d46713a5-7bba-4dfb-b77f-8fae0b39271e\") " pod="openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-fgvsp"
Oct 31 17:39:32.137548 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:32.137526    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d46713a5-7bba-4dfb-b77f-8fae0b39271e-credential-socket\") pod \"e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-fgvsp\" (UID: \"d46713a5-7bba-4dfb-b77f-8fae0b39271e\") " pod="openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-fgvsp"
Oct 31 17:39:32.137787 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:32.137687    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d46713a5-7bba-4dfb-b77f-8fae0b39271e-istiod-ca-cert\") pod \"e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-fgvsp\" (UID: \"d46713a5-7bba-4dfb-b77f-8fae0b39271e\") " pod="openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-fgvsp"
Oct 31 17:39:32.138082 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:32.138061    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d46713a5-7bba-4dfb-b77f-8fae0b39271e-workload-socket\") pod \"e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-fgvsp\" (UID: \"d46713a5-7bba-4dfb-b77f-8fae0b39271e\") " pod="openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-fgvsp"
Oct 31 17:39:32.138390 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:32.138164    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d46713a5-7bba-4dfb-b77f-8fae0b39271e-workload-certs\") pod \"e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-fgvsp\" (UID: \"d46713a5-7bba-4dfb-b77f-8fae0b39271e\") " pod="openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-fgvsp"
Oct 31 17:39:32.138555 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:32.138497    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d46713a5-7bba-4dfb-b77f-8fae0b39271e-istio-data\") pod \"e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-fgvsp\" (UID: \"d46713a5-7bba-4dfb-b77f-8fae0b39271e\") " pod="openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-fgvsp"
Oct 31 17:39:32.139823 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:32.139794    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d46713a5-7bba-4dfb-b77f-8fae0b39271e-istio-envoy\") pod \"e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-fgvsp\" (UID: \"d46713a5-7bba-4dfb-b77f-8fae0b39271e\") " pod="openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-fgvsp"
Oct 31 17:39:32.140264 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:32.140242    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d46713a5-7bba-4dfb-b77f-8fae0b39271e-istio-podinfo\") pod \"e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-fgvsp\" (UID: \"d46713a5-7bba-4dfb-b77f-8fae0b39271e\") " pod="openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-fgvsp"
Oct 31 17:39:32.149312 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:32.149284    4276 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7d9897b54c-98fcj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body=
Oct 31 17:39:32.149400 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:32.149331    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-98fcj" podUID="9421021e-be39-4906-8fae-da1ce9ba4047" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused"
Oct 31 17:39:32.189475 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:32.184768    4276 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7d9897b54c-p4v97 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body=
Oct 31 17:39:32.189475 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:32.184823    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-p4v97" podUID="3ffeb078-cb4d-4293-b7fb-eb6fd305d96b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused"
Oct 31 17:39:32.189475 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:32.185659    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-dx9cd"]
Oct 31 17:39:32.196467 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:32.192775    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d46713a5-7bba-4dfb-b77f-8fae0b39271e-istio-token\") pod \"e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-fgvsp\" (UID: \"d46713a5-7bba-4dfb-b77f-8fae0b39271e\") " pod="openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-fgvsp"
Oct 31 17:39:32.197822 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:32.197794    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g8pph\" (UniqueName: \"kubernetes.io/projected/d46713a5-7bba-4dfb-b77f-8fae0b39271e-kube-api-access-g8pph\") pod \"e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-fgvsp\" (UID: \"d46713a5-7bba-4dfb-b77f-8fae0b39271e\") " pod="openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-fgvsp"
Oct 31 17:39:32.209357 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:39:32.209305    4276 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3716260a_fa38_4064_991c_48322b1b48fa.slice/crio-d1130cce08a06c9fc72c9c1126d0154246c0a939b75b11809b8c6f9132738629 WatchSource:0}: Error finding container d1130cce08a06c9fc72c9c1126d0154246c0a939b75b11809b8c6f9132738629: Status 404 returned error can't find the container with id d1130cce08a06c9fc72c9c1126d0154246c0a939b75b11809b8c6f9132738629
Oct 31 17:39:32.212318 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:32.212286    4276 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 17:39:32.212400 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:32.212353    4276 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 17:39:32.212460 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:32.212401    4276 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 17:39:32.348614 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:32.348586    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-fgvsp"
Oct 31 17:39:32.539683 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:32.539626    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-dx9cd" event={"ID":"3716260a-fa38-4064-991c-48322b1b48fa","Type":"ContainerStarted","Data":"d1130cce08a06c9fc72c9c1126d0154246c0a939b75b11809b8c6f9132738629"}
Oct 31 17:39:32.545169 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:32.545140    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-rfwx5" event={"ID":"5155d374-e42a-4c40-9485-935443ede2c5","Type":"ContainerStarted","Data":"0707bf2ff1d341d72693257e316834b57c76ced9cad6fe5df2ec01e0ec9c803d"}
Oct 31 17:39:32.545386 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:32.545372    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-rfwx5" event={"ID":"5155d374-e42a-4c40-9485-935443ede2c5","Type":"ContainerStarted","Data":"3c1a889c48e1f72718fae760d28b15b474c59131109f7a65ea2bfd263e5b4e06"}
Oct 31 17:39:32.604865 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:32.604754    4276 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-rfwx5" podStartSLOduration=1.6047321079999999 podStartE2EDuration="1.604732108s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 17:39:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 17:39:32.600915218 +0000 UTC m=+735.398660147" watchObservedRunningTime="2026-10-31 17:39:32.604732108 +0000 UTC m=+735.402477037"
Oct 31 17:39:32.925268 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:32.925182    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-fgvsp"]
Oct 31 17:39:32.931948 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:32.931914    4276 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 17:39:32.932068 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:32.931990    4276 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 17:39:32.932068 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:32.932034    4276 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 17:39:33.149559 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:33.149525    4276 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7d9897b54c-98fcj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body=
Oct 31 17:39:33.149700 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:33.149576    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-98fcj" podUID="9421021e-be39-4906-8fae-da1ce9ba4047" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused"
Oct 31 17:39:33.182971 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:33.182802    4276 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7d9897b54c-p4v97 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body=
Oct 31 17:39:33.182971 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:33.182858    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-p4v97" podUID="3ffeb078-cb4d-4293-b7fb-eb6fd305d96b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused"
Oct 31 17:39:33.533510 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:33.533408    4276 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-rfwx5"
Oct 31 17:39:33.534131 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:33.534108    4276 patch_prober.go:29] interesting pod/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-rfwx5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Oct 31 17:39:33.534285 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:33.534261    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-rfwx5" podUID="5155d374-e42a-4c40-9485-935443ede2c5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Oct 31 17:39:33.572514 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:33.571896    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:39:33.587287 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:33.587251    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-fgvsp" event={"ID":"d46713a5-7bba-4dfb-b77f-8fae0b39271e","Type":"ContainerStarted","Data":"80478be634138f6b35e4e949b2560185d9c16c8c3466e1bcc2b3320bbff17148"}
Oct 31 17:39:33.587576 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:33.587552    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-fgvsp" event={"ID":"d46713a5-7bba-4dfb-b77f-8fae0b39271e","Type":"ContainerStarted","Data":"216c2271f7cc28d5be2d78234482aabc108227f2ac122befeb33cfac184871ac"}
Oct 31 17:39:33.601944 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:33.598542    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-dx9cd" event={"ID":"3716260a-fa38-4064-991c-48322b1b48fa","Type":"ContainerStarted","Data":"122ad55efb2bd53bf8bad3bce430e571a33aa4bfe0469d290e9c49dc7e3b99a5"}
Oct 31 17:39:33.769496 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:33.761968    4276 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-dx9cd"
Oct 31 17:39:33.792545 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:33.736414    4276 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-fgvsp" podStartSLOduration=2.736390982 podStartE2EDuration="2.736390982s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 17:39:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 17:39:33.653261779 +0000 UTC m=+736.451006709" watchObservedRunningTime="2026-10-31 17:39:33.736390982 +0000 UTC m=+736.534135911"
Oct 31 17:39:33.792545 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:33.784245    4276 patch_prober.go:29] interesting pod/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-dx9cd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body=
Oct 31 17:39:33.792545 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:33.784293    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-dx9cd" podUID="3716260a-fa38-4064-991c-48322b1b48fa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused"
Oct 31 17:39:34.156466 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:34.156408    4276 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7d9897b54c-98fcj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body=
Oct 31 17:39:34.156673 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:34.156500    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-98fcj" podUID="9421021e-be39-4906-8fae-da1ce9ba4047" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused"
Oct 31 17:39:34.192468 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:34.191849    4276 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7d9897b54c-p4v97 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body=
Oct 31 17:39:34.192468 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:34.191909    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-p4v97" podUID="3ffeb078-cb4d-4293-b7fb-eb6fd305d96b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused"
Oct 31 17:39:34.361469 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:34.351572    4276 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-fgvsp"
Oct 31 17:39:34.361469 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:34.351704    4276 patch_prober.go:29] interesting pod/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-fgvsp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body=
Oct 31 17:39:34.361469 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:34.351749    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-fgvsp" podUID="d46713a5-7bba-4dfb-b77f-8fae0b39271e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused"
Oct 31 17:39:34.410520 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:34.405565    4276 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w6-d80080/distinct-w6-d80080-kserve-664576c8f5-6jc4d"
Oct 31 17:39:34.524471 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:34.521236    4276 patch_prober.go:29] interesting pod/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-rfwx5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Oct 31 17:39:34.524471 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:34.521310    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-rfwx5" podUID="5155d374-e42a-4c40-9485-935443ede2c5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Oct 31 17:39:34.744522 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:34.743010    4276 patch_prober.go:29] interesting pod/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-dx9cd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body=
Oct 31 17:39:34.744522 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:34.743090    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-dx9cd" podUID="3716260a-fa38-4064-991c-48322b1b48fa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused"
Oct 31 17:39:35.159850 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:35.157744    4276 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7d9897b54c-98fcj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body=
Oct 31 17:39:35.159850 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:35.157803    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-98fcj" podUID="9421021e-be39-4906-8fae-da1ce9ba4047" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused"
Oct 31 17:39:35.184960 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:35.184881    4276 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7d9897b54c-p4v97 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body=
Oct 31 17:39:35.185270 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:35.185198    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-p4v97" podUID="3ffeb078-cb4d-4293-b7fb-eb6fd305d96b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused"
Oct 31 17:39:35.351924 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:35.351886    4276 patch_prober.go:29] interesting pod/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-fgvsp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body=
Oct 31 17:39:35.352258 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:35.352209    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-fgvsp" podUID="d46713a5-7bba-4dfb-b77f-8fae0b39271e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused"
Oct 31 17:39:35.406790 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:35.406710    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-0e59ca/premium-w1-0e59ca-kserve-58765f9746-fqwq7" podUID="8cf1e52e-70f8-42e2-bdf3-a315e17f10f5" containerName="main" probeResult="failure" output="Get \"https://10.133.0.48:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 17:39:35.409634 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:35.409542    4276 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-models-e2e-worker-w6-d80080/distinct-w6-d80080-kserve-664576c8f5-6jc4d" podUID="b9c2f445-a191-4b00-802a-1b0ded068411" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/health\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 31 17:39:35.523525 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:35.522869    4276 patch_prober.go:29] interesting pod/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-rfwx5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Oct 31 17:39:35.523525 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:35.522950    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-rfwx5" podUID="5155d374-e42a-4c40-9485-935443ede2c5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Oct 31 17:39:35.740469 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:35.739841    4276 patch_prober.go:29] interesting pod/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-dx9cd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body=
Oct 31 17:39:35.740469 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:35.739902    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-dx9cd" podUID="3716260a-fa38-4064-991c-48322b1b48fa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused"
Oct 31 17:39:36.155472 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:36.155113    4276 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7d9897b54c-98fcj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body=
Oct 31 17:39:36.155472 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:36.155170    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-98fcj" podUID="9421021e-be39-4906-8fae-da1ce9ba4047" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused"
Oct 31 17:39:36.187833 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:36.187798    4276 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7d9897b54c-p4v97 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body=
Oct 31 17:39:36.188070 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:36.188044    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-p4v97" podUID="3ffeb078-cb4d-4293-b7fb-eb6fd305d96b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused"
Oct 31 17:39:36.518670 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:36.518574    4276 patch_prober.go:29] interesting pod/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-fgvsp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body=
Oct 31 17:39:36.518942 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:36.518890    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-fgvsp" podUID="d46713a5-7bba-4dfb-b77f-8fae0b39271e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused"
Oct 31 17:39:36.527472 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:36.521140    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ee816e/premium-w0-ee816e-kserve-65c86cd6db-lnt9p" podUID="e536c902-52a9-4b36-a2ad-eb5500781a5e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.49:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 31 17:39:36.527809 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:36.522806    4276 patch_prober.go:29] interesting pod/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-rfwx5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Oct 31 17:39:36.527974 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:36.527952    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-rfwx5" podUID="5155d374-e42a-4c40-9485-935443ede2c5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Oct 31 17:39:36.528305 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:36.523277    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:39:36.790527 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:36.790144    4276 patch_prober.go:29] interesting pod/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-dx9cd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body=
Oct 31 17:39:36.790527 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:36.790207    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-dx9cd" podUID="3716260a-fa38-4064-991c-48322b1b48fa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused"
Oct 31 17:39:36.947477 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:36.947396    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:39:37.151903 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:37.151869    4276 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7d9897b54c-98fcj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body=
Oct 31 17:39:37.152170 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:37.152142    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-98fcj" podUID="9421021e-be39-4906-8fae-da1ce9ba4047" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused"
Oct 31 17:39:37.186469 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:37.185579    4276 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7d9897b54c-p4v97 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body=
Oct 31 17:39:37.186469 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:37.185633    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-p4v97" podUID="3ffeb078-cb4d-4293-b7fb-eb6fd305d96b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused"
Oct 31 17:39:37.360468 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:37.351829    4276 patch_prober.go:29] interesting pod/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-fgvsp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body=
Oct 31 17:39:37.360468 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:37.351896    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-fgvsp" podUID="d46713a5-7bba-4dfb-b77f-8fae0b39271e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused"
Oct 31 17:39:37.522080 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:37.521980    4276 patch_prober.go:29] interesting pod/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-rfwx5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Oct 31 17:39:37.522371 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:37.522338    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-rfwx5" podUID="5155d374-e42a-4c40-9485-935443ede2c5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Oct 31 17:39:37.739467 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:37.738682    4276 patch_prober.go:29] interesting pod/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-dx9cd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body=
Oct 31 17:39:37.739467 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:37.738740    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-dx9cd" podUID="3716260a-fa38-4064-991c-48322b1b48fa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused"
Oct 31 17:39:38.149421 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:38.149373    4276 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7d9897b54c-98fcj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body=
Oct 31 17:39:38.149788 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:38.149755    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-98fcj" podUID="9421021e-be39-4906-8fae-da1ce9ba4047" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused"
Oct 31 17:39:38.187211 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:38.187166    4276 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7d9897b54c-p4v97 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body=
Oct 31 17:39:38.187821 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:38.187775    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-p4v97" podUID="3ffeb078-cb4d-4293-b7fb-eb6fd305d96b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused"
Oct 31 17:39:38.360489 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:38.352626    4276 patch_prober.go:29] interesting pod/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-fgvsp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body=
Oct 31 17:39:38.360489 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:38.352682    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-fgvsp" podUID="d46713a5-7bba-4dfb-b77f-8fae0b39271e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused"
Oct 31 17:39:38.551586 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:38.551552    4276 patch_prober.go:29] interesting pod/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-rfwx5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Oct 31 17:39:38.552279 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:38.552247    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-rfwx5" podUID="5155d374-e42a-4c40-9485-935443ede2c5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Oct 31 17:39:38.738468 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:38.738086    4276 patch_prober.go:29] interesting pod/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-dx9cd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body=
Oct 31 17:39:38.738468 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:38.738154    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-dx9cd" podUID="3716260a-fa38-4064-991c-48322b1b48fa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused"
Oct 31 17:39:39.253467 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:39.221727    4276 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7d9897b54c-p4v97 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body=
Oct 31 17:39:39.254071 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:39.254036    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-p4v97" podUID="3ffeb078-cb4d-4293-b7fb-eb6fd305d96b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused"
Oct 31 17:39:39.254564 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:39.227846    4276 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7d9897b54c-98fcj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body=
Oct 31 17:39:39.272464 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:39.254709    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-98fcj" podUID="9421021e-be39-4906-8fae-da1ce9ba4047" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused"
Oct 31 17:39:39.353922 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:39.353888    4276 patch_prober.go:29] interesting pod/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-fgvsp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body=
Oct 31 17:39:39.354186 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:39.354162    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-fgvsp" podUID="d46713a5-7bba-4dfb-b77f-8fae0b39271e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused"
Oct 31 17:39:39.563529 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:39.531069    4276 patch_prober.go:29] interesting pod/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-rfwx5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Oct 31 17:39:39.563529 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:39.531139    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-rfwx5" podUID="5155d374-e42a-4c40-9485-935443ede2c5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Oct 31 17:39:39.756855 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:39.755555    4276 patch_prober.go:29] interesting pod/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-dx9cd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body=
Oct 31 17:39:39.756855 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:39.755618    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-dx9cd" podUID="3716260a-fa38-4064-991c-48322b1b48fa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused"
Oct 31 17:39:40.358786 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:40.358752    4276 patch_prober.go:29] interesting pod/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-fgvsp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body=
Oct 31 17:39:40.359418 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:40.359384    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-fgvsp" podUID="d46713a5-7bba-4dfb-b77f-8fae0b39271e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused"
Oct 31 17:39:40.548587 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:40.548546    4276 patch_prober.go:29] interesting pod/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-rfwx5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Oct 31 17:39:40.561567 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:40.561524    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-rfwx5" podUID="5155d374-e42a-4c40-9485-935443ede2c5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Oct 31 17:39:40.810713 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:40.810667    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:39:40.838491 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:40.824143    4276 patch_prober.go:29] interesting pod/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-dx9cd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body=
Oct 31 17:39:40.838491 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:40.824199    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-dx9cd" podUID="3716260a-fa38-4064-991c-48322b1b48fa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused"
Oct 31 17:39:41.164707 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:41.164614    4276 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7d9897b54c-98fcj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 17:39:41.164949 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:41.164919    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-98fcj" podUID="9421021e-be39-4906-8fae-da1ce9ba4047" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 17:39:41.247578 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:41.247541    4276 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7d9897b54c-p4v97 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 17:39:41.247846 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:41.247818    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-p4v97" podUID="3ffeb078-cb4d-4293-b7fb-eb6fd305d96b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 17:39:41.353973 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:41.353940    4276 patch_prober.go:29] interesting pod/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-fgvsp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body=
Oct 31 17:39:41.354254 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:41.354209    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-fgvsp" podUID="d46713a5-7bba-4dfb-b77f-8fae0b39271e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused"
Oct 31 17:39:41.543624 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:41.543517    4276 patch_prober.go:29] interesting pod/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-rfwx5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Oct 31 17:39:41.544620 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:41.544588    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-rfwx5" podUID="5155d374-e42a-4c40-9485-935443ede2c5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Oct 31 17:39:41.600510 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:41.582752    4276 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-98fcj"
Oct 31 17:39:41.600510 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:41.584128    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-98fcj"
Oct 31 17:39:41.600510 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:41.594725    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-98fcj"
Oct 31 17:39:41.702623 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:41.702589    4276 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-p4v97"
Oct 31 17:39:41.703600 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:41.703464    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-p4v97"
Oct 31 17:39:41.706492 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:41.706411    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-p4v97"
Oct 31 17:39:41.710005 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:41.709946    4276 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-dx9cd" podStartSLOduration=10.709928738 podStartE2EDuration="10.709928738s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 17:39:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 17:39:33.784149211 +0000 UTC m=+736.581894135" watchObservedRunningTime="2026-10-31 17:39:41.709928738 +0000 UTC m=+744.507673667"
Oct 31 17:39:41.737881 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:41.737847    4276 patch_prober.go:29] interesting pod/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-dx9cd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body=
Oct 31 17:39:41.738027 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:41.737903    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-dx9cd" podUID="3716260a-fa38-4064-991c-48322b1b48fa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused"
Oct 31 17:39:42.236344 ip-10-0-129-127 kubenswrapper[4276]: E1031 17:39:42.236301    4276 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tg5qk" containerName="extractor"
Oct 31 17:39:42.350060 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:42.350026    4276 patch_prober.go:29] interesting pod/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-fgvsp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body=
Oct 31 17:39:42.350243 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:42.350079    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-fgvsp" podUID="d46713a5-7bba-4dfb-b77f-8fae0b39271e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused"
Oct 31 17:39:42.522754 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:42.522583    4276 patch_prober.go:29] interesting pod/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-rfwx5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Oct 31 17:39:42.522754 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:42.522650    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-rfwx5" podUID="5155d374-e42a-4c40-9485-935443ede2c5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Oct 31 17:39:42.743555 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:42.743519    4276 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-dx9cd"
Oct 31 17:39:42.743953 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:42.743929    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-dx9cd"
Oct 31 17:39:42.744714 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:42.744691    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-dx9cd"
Oct 31 17:39:43.082282 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:43.082240    4276 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w2-5c446c/distinct-w2-5c446c-kserve-69d97d6968-gtvjv"]
Oct 31 17:39:43.087091 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:43.087071    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-5c446c/distinct-w2-5c446c-kserve-69d97d6968-gtvjv"
Oct 31 17:39:43.112775 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:43.112747    4276 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w2-5c446c\"/\"openshift-service-ca.crt\""
Oct 31 17:39:43.121479 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:43.113144    4276 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w2-5c446c\"/\"kube-root-ca.crt\""
Oct 31 17:39:43.121479 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:43.117438    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-5c446c/distinct-w2-5c446c-kserve-69d97d6968-gtvjv"]
Oct 31 17:39:43.126548 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:43.126525    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-5c446c\"/\"default-dockercfg-52ml5\""
Oct 31 17:39:43.126656 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:43.126642    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-5c446c\"/\"distinct-w2-5c446c-kserve-self-signed-certs\""
Oct 31 17:39:43.160166 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:43.160140    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:39:43.172419 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:43.172394    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2d1185cc-bac1-45ca-a250-8f44f8589162-home\") pod \"distinct-w2-5c446c-kserve-69d97d6968-gtvjv\" (UID: \"2d1185cc-bac1-45ca-a250-8f44f8589162\") " pod="e2e-models-e2e-worker-w2-5c446c/distinct-w2-5c446c-kserve-69d97d6968-gtvjv"
Oct 31 17:39:43.172533 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:43.172438    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sl555\" (UniqueName: \"kubernetes.io/projected/2d1185cc-bac1-45ca-a250-8f44f8589162-kube-api-access-sl555\") pod \"distinct-w2-5c446c-kserve-69d97d6968-gtvjv\" (UID: \"2d1185cc-bac1-45ca-a250-8f44f8589162\") " pod="e2e-models-e2e-worker-w2-5c446c/distinct-w2-5c446c-kserve-69d97d6968-gtvjv"
Oct 31 17:39:43.172611 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:43.172496    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2d1185cc-bac1-45ca-a250-8f44f8589162-dshm\") pod \"distinct-w2-5c446c-kserve-69d97d6968-gtvjv\" (UID: \"2d1185cc-bac1-45ca-a250-8f44f8589162\") " pod="e2e-models-e2e-worker-w2-5c446c/distinct-w2-5c446c-kserve-69d97d6968-gtvjv"
Oct 31 17:39:43.172865 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:43.172835    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2d1185cc-bac1-45ca-a250-8f44f8589162-model-cache\") pod \"distinct-w2-5c446c-kserve-69d97d6968-gtvjv\" (UID: \"2d1185cc-bac1-45ca-a250-8f44f8589162\") " pod="e2e-models-e2e-worker-w2-5c446c/distinct-w2-5c446c-kserve-69d97d6968-gtvjv"
Oct 31 17:39:43.173050 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:43.173026    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2d1185cc-bac1-45ca-a250-8f44f8589162-tmp-dir\") pod \"distinct-w2-5c446c-kserve-69d97d6968-gtvjv\" (UID: \"2d1185cc-bac1-45ca-a250-8f44f8589162\") " pod="e2e-models-e2e-worker-w2-5c446c/distinct-w2-5c446c-kserve-69d97d6968-gtvjv"
Oct 31 17:39:43.173165 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:43.173141    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2d1185cc-bac1-45ca-a250-8f44f8589162-tls-certs\") pod \"distinct-w2-5c446c-kserve-69d97d6968-gtvjv\" (UID: \"2d1185cc-bac1-45ca-a250-8f44f8589162\") " pod="e2e-models-e2e-worker-w2-5c446c/distinct-w2-5c446c-kserve-69d97d6968-gtvjv"
Oct 31 17:39:43.274078 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:43.274028    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2d1185cc-bac1-45ca-a250-8f44f8589162-home\") pod \"distinct-w2-5c446c-kserve-69d97d6968-gtvjv\" (UID: \"2d1185cc-bac1-45ca-a250-8f44f8589162\") " pod="e2e-models-e2e-worker-w2-5c446c/distinct-w2-5c446c-kserve-69d97d6968-gtvjv"
Oct 31 17:39:43.274078 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:43.274075    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sl555\" (UniqueName: \"kubernetes.io/projected/2d1185cc-bac1-45ca-a250-8f44f8589162-kube-api-access-sl555\") pod \"distinct-w2-5c446c-kserve-69d97d6968-gtvjv\" (UID: \"2d1185cc-bac1-45ca-a250-8f44f8589162\") " pod="e2e-models-e2e-worker-w2-5c446c/distinct-w2-5c446c-kserve-69d97d6968-gtvjv"
Oct 31 17:39:43.274352 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:43.274189    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2d1185cc-bac1-45ca-a250-8f44f8589162-dshm\") pod \"distinct-w2-5c446c-kserve-69d97d6968-gtvjv\" (UID: \"2d1185cc-bac1-45ca-a250-8f44f8589162\") " pod="e2e-models-e2e-worker-w2-5c446c/distinct-w2-5c446c-kserve-69d97d6968-gtvjv"
Oct 31 17:39:43.274352 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:43.274291    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2d1185cc-bac1-45ca-a250-8f44f8589162-model-cache\") pod \"distinct-w2-5c446c-kserve-69d97d6968-gtvjv\" (UID: \"2d1185cc-bac1-45ca-a250-8f44f8589162\") " pod="e2e-models-e2e-worker-w2-5c446c/distinct-w2-5c446c-kserve-69d97d6968-gtvjv"
Oct 31 17:39:43.274504 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:43.274368    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2d1185cc-bac1-45ca-a250-8f44f8589162-tmp-dir\") pod \"distinct-w2-5c446c-kserve-69d97d6968-gtvjv\" (UID: \"2d1185cc-bac1-45ca-a250-8f44f8589162\") " pod="e2e-models-e2e-worker-w2-5c446c/distinct-w2-5c446c-kserve-69d97d6968-gtvjv"
Oct 31 17:39:43.274504 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:43.274418    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2d1185cc-bac1-45ca-a250-8f44f8589162-tls-certs\") pod \"distinct-w2-5c446c-kserve-69d97d6968-gtvjv\" (UID: \"2d1185cc-bac1-45ca-a250-8f44f8589162\") " pod="e2e-models-e2e-worker-w2-5c446c/distinct-w2-5c446c-kserve-69d97d6968-gtvjv"
Oct 31 17:39:43.274504 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:43.274482    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2d1185cc-bac1-45ca-a250-8f44f8589162-home\") pod \"distinct-w2-5c446c-kserve-69d97d6968-gtvjv\" (UID: \"2d1185cc-bac1-45ca-a250-8f44f8589162\") " pod="e2e-models-e2e-worker-w2-5c446c/distinct-w2-5c446c-kserve-69d97d6968-gtvjv"
Oct 31 17:39:43.274777 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:43.274755    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2d1185cc-bac1-45ca-a250-8f44f8589162-model-cache\") pod \"distinct-w2-5c446c-kserve-69d97d6968-gtvjv\" (UID: \"2d1185cc-bac1-45ca-a250-8f44f8589162\") " pod="e2e-models-e2e-worker-w2-5c446c/distinct-w2-5c446c-kserve-69d97d6968-gtvjv"
Oct 31 17:39:43.274844 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:43.274775    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2d1185cc-bac1-45ca-a250-8f44f8589162-tmp-dir\") pod \"distinct-w2-5c446c-kserve-69d97d6968-gtvjv\" (UID: \"2d1185cc-bac1-45ca-a250-8f44f8589162\") " pod="e2e-models-e2e-worker-w2-5c446c/distinct-w2-5c446c-kserve-69d97d6968-gtvjv"
Oct 31 17:39:43.276550 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:43.276517    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2d1185cc-bac1-45ca-a250-8f44f8589162-dshm\") pod \"distinct-w2-5c446c-kserve-69d97d6968-gtvjv\" (UID: \"2d1185cc-bac1-45ca-a250-8f44f8589162\") " pod="e2e-models-e2e-worker-w2-5c446c/distinct-w2-5c446c-kserve-69d97d6968-gtvjv"
Oct 31 17:39:43.276816 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:43.276797    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2d1185cc-bac1-45ca-a250-8f44f8589162-tls-certs\") pod \"distinct-w2-5c446c-kserve-69d97d6968-gtvjv\" (UID: \"2d1185cc-bac1-45ca-a250-8f44f8589162\") " pod="e2e-models-e2e-worker-w2-5c446c/distinct-w2-5c446c-kserve-69d97d6968-gtvjv"
Oct 31 17:39:43.294198 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:43.294172    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sl555\" (UniqueName: \"kubernetes.io/projected/2d1185cc-bac1-45ca-a250-8f44f8589162-kube-api-access-sl555\") pod \"distinct-w2-5c446c-kserve-69d97d6968-gtvjv\" (UID: \"2d1185cc-bac1-45ca-a250-8f44f8589162\") " pod="e2e-models-e2e-worker-w2-5c446c/distinct-w2-5c446c-kserve-69d97d6968-gtvjv"
Oct 31 17:39:43.354871 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:43.353001    4276 patch_prober.go:29] interesting pod/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-fgvsp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body=
Oct 31 17:39:43.354871 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:43.353060    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-fgvsp" podUID="d46713a5-7bba-4dfb-b77f-8fae0b39271e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused"
Oct 31 17:39:43.418755 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:43.418707    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-5c446c/distinct-w2-5c446c-kserve-69d97d6968-gtvjv"
Oct 31 17:39:43.782516 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:43.781975    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-5c446c/distinct-w2-5c446c-kserve-69d97d6968-gtvjv"]
Oct 31 17:39:43.806298 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:39:43.806254    4276 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2d1185cc_bac1_45ca_a250_8f44f8589162.slice/crio-805e0d3ccc93d0325525a13ebc031326d1c8e5a9465a55ed4038a06399ec6c60 WatchSource:0}: Error finding container 805e0d3ccc93d0325525a13ebc031326d1c8e5a9465a55ed4038a06399ec6c60: Status 404 returned error can't find the container with id 805e0d3ccc93d0325525a13ebc031326d1c8e5a9465a55ed4038a06399ec6c60
Oct 31 17:39:44.000973 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:44.000937    4276 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-rfwx5"
Oct 31 17:39:44.002151 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:44.002122    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-rfwx5"
Oct 31 17:39:44.003821 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:44.003800    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-rfwx5"
Oct 31 17:39:44.129634 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:44.129593    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-5c446c/distinct-w2-5c446c-kserve-69d97d6968-gtvjv" event={"ID":"2d1185cc-bac1-45ca-a250-8f44f8589162","Type":"ContainerStarted","Data":"b9f950b5763a04a6e7f73de21e40008851382e1b9e75364b413def2e4f1204fc"}
Oct 31 17:39:44.129804 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:44.129643    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-5c446c/distinct-w2-5c446c-kserve-69d97d6968-gtvjv" event={"ID":"2d1185cc-bac1-45ca-a250-8f44f8589162","Type":"ContainerStarted","Data":"805e0d3ccc93d0325525a13ebc031326d1c8e5a9465a55ed4038a06399ec6c60"}
Oct 31 17:39:44.172039 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:44.171990    4276 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w2-5c446c/distinct-w2-5c446c-kserve-69d97d6968-gtvjv" podStartSLOduration=1.171977124 podStartE2EDuration="1.171977124s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 17:39:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 17:39:44.170468219 +0000 UTC m=+746.968213164" watchObservedRunningTime="2026-10-31 17:39:44.171977124 +0000 UTC m=+746.969722082"
Oct 31 17:39:44.278631 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:44.278588    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:39:44.360156 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:44.360121    4276 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-fgvsp"
Oct 31 17:39:44.360888 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:44.360777    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-fgvsp"
Oct 31 17:39:44.362185 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:44.362167    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-fgvsp"
Oct 31 17:39:44.414745 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:44.414670    4276 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w6-d80080/distinct-w6-d80080-kserve-664576c8f5-6jc4d"
Oct 31 17:39:44.424929 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:44.424907    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-5c446c/distinct-w2-5c446c-kserve-69d97d6968-gtvjv"
Oct 31 17:39:44.426145 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:44.426123    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-d80080/distinct-w6-d80080-kserve-664576c8f5-6jc4d"
Oct 31 17:39:46.167534 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:46.167500    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:39:46.485963 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:46.485879    4276 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-qlhn8"]
Oct 31 17:39:46.490146 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:46.490124    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-qlhn8"
Oct 31 17:39:46.525301 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:46.525269    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-qlhn8"]
Oct 31 17:39:46.536183 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:46.536160    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c3c40ebe-5cac-4981-913d-18f12d0bf439-workload-certs\") pod \"e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-qlhn8\" (UID: \"c3c40ebe-5cac-4981-913d-18f12d0bf439\") " pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-qlhn8"
Oct 31 17:39:46.536365 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:46.536197    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c3c40ebe-5cac-4981-913d-18f12d0bf439-istio-token\") pod \"e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-qlhn8\" (UID: \"c3c40ebe-5cac-4981-913d-18f12d0bf439\") " pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-qlhn8"
Oct 31 17:39:46.536365 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:46.536276    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3c40ebe-5cac-4981-913d-18f12d0bf439-credential-socket\") pod \"e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-qlhn8\" (UID: \"c3c40ebe-5cac-4981-913d-18f12d0bf439\") " pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-qlhn8"
Oct 31 17:39:46.536365 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:46.536292    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c3c40ebe-5cac-4981-913d-18f12d0bf439-istio-envoy\") pod \"e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-qlhn8\" (UID: \"c3c40ebe-5cac-4981-913d-18f12d0bf439\") " pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-qlhn8"
Oct 31 17:39:46.536559 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:46.536481    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w7v7k\" (UniqueName: \"kubernetes.io/projected/c3c40ebe-5cac-4981-913d-18f12d0bf439-kube-api-access-w7v7k\") pod \"e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-qlhn8\" (UID: \"c3c40ebe-5cac-4981-913d-18f12d0bf439\") " pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-qlhn8"
Oct 31 17:39:46.536559 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:46.536536    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c3c40ebe-5cac-4981-913d-18f12d0bf439-istiod-ca-cert\") pod \"e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-qlhn8\" (UID: \"c3c40ebe-5cac-4981-913d-18f12d0bf439\") " pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-qlhn8"
Oct 31 17:39:46.536681 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:46.536663    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c3c40ebe-5cac-4981-913d-18f12d0bf439-istio-podinfo\") pod \"e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-qlhn8\" (UID: \"c3c40ebe-5cac-4981-913d-18f12d0bf439\") " pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-qlhn8"
Oct 31 17:39:46.536733 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:46.536720    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3c40ebe-5cac-4981-913d-18f12d0bf439-workload-socket\") pod \"e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-qlhn8\" (UID: \"c3c40ebe-5cac-4981-913d-18f12d0bf439\") " pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-qlhn8"
Oct 31 17:39:46.536785 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:46.536756    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c3c40ebe-5cac-4981-913d-18f12d0bf439-istio-data\") pod \"e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-qlhn8\" (UID: \"c3c40ebe-5cac-4981-913d-18f12d0bf439\") " pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-qlhn8"
Oct 31 17:39:46.638277 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:46.638242    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c3c40ebe-5cac-4981-913d-18f12d0bf439-istio-podinfo\") pod \"e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-qlhn8\" (UID: \"c3c40ebe-5cac-4981-913d-18f12d0bf439\") " pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-qlhn8"
Oct 31 17:39:46.638492 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:46.638296    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3c40ebe-5cac-4981-913d-18f12d0bf439-workload-socket\") pod \"e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-qlhn8\" (UID: \"c3c40ebe-5cac-4981-913d-18f12d0bf439\") " pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-qlhn8"
Oct 31 17:39:46.638492 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:46.638312    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c3c40ebe-5cac-4981-913d-18f12d0bf439-istio-data\") pod \"e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-qlhn8\" (UID: \"c3c40ebe-5cac-4981-913d-18f12d0bf439\") " pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-qlhn8"
Oct 31 17:39:46.638492 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:46.638335    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c3c40ebe-5cac-4981-913d-18f12d0bf439-workload-certs\") pod \"e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-qlhn8\" (UID: \"c3c40ebe-5cac-4981-913d-18f12d0bf439\") " pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-qlhn8"
Oct 31 17:39:46.638492 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:46.638360    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c3c40ebe-5cac-4981-913d-18f12d0bf439-istio-token\") pod \"e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-qlhn8\" (UID: \"c3c40ebe-5cac-4981-913d-18f12d0bf439\") " pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-qlhn8"
Oct 31 17:39:46.638492 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:46.638398    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3c40ebe-5cac-4981-913d-18f12d0bf439-credential-socket\") pod \"e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-qlhn8\" (UID: \"c3c40ebe-5cac-4981-913d-18f12d0bf439\") " pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-qlhn8"
Oct 31 17:39:46.638492 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:46.638412    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c3c40ebe-5cac-4981-913d-18f12d0bf439-istio-envoy\") pod \"e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-qlhn8\" (UID: \"c3c40ebe-5cac-4981-913d-18f12d0bf439\") " pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-qlhn8"
Oct 31 17:39:46.638827 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:46.638529    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w7v7k\" (UniqueName: \"kubernetes.io/projected/c3c40ebe-5cac-4981-913d-18f12d0bf439-kube-api-access-w7v7k\") pod \"e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-qlhn8\" (UID: \"c3c40ebe-5cac-4981-913d-18f12d0bf439\") " pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-qlhn8"
Oct 31 17:39:46.638827 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:46.638564    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c3c40ebe-5cac-4981-913d-18f12d0bf439-istiod-ca-cert\") pod \"e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-qlhn8\" (UID: \"c3c40ebe-5cac-4981-913d-18f12d0bf439\") " pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-qlhn8"
Oct 31 17:39:46.638827 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:46.638710    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c3c40ebe-5cac-4981-913d-18f12d0bf439-istio-data\") pod \"e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-qlhn8\" (UID: \"c3c40ebe-5cac-4981-913d-18f12d0bf439\") " pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-qlhn8"
Oct 31 17:39:46.638827 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:46.638820    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3c40ebe-5cac-4981-913d-18f12d0bf439-credential-socket\") pod \"e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-qlhn8\" (UID: \"c3c40ebe-5cac-4981-913d-18f12d0bf439\") " pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-qlhn8"
Oct 31 17:39:46.639013 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:46.638857    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c3c40ebe-5cac-4981-913d-18f12d0bf439-workload-certs\") pod \"e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-qlhn8\" (UID: \"c3c40ebe-5cac-4981-913d-18f12d0bf439\") " pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-qlhn8"
Oct 31 17:39:46.639065 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:46.639019    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3c40ebe-5cac-4981-913d-18f12d0bf439-workload-socket\") pod \"e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-qlhn8\" (UID: \"c3c40ebe-5cac-4981-913d-18f12d0bf439\") " pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-qlhn8"
Oct 31 17:39:46.639391 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:46.639366    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c3c40ebe-5cac-4981-913d-18f12d0bf439-istiod-ca-cert\") pod \"e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-qlhn8\" (UID: \"c3c40ebe-5cac-4981-913d-18f12d0bf439\") " pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-qlhn8"
Oct 31 17:39:46.640730 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:46.640709    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c3c40ebe-5cac-4981-913d-18f12d0bf439-istio-envoy\") pod \"e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-qlhn8\" (UID: \"c3c40ebe-5cac-4981-913d-18f12d0bf439\") " pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-qlhn8"
Oct 31 17:39:46.641146 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:46.641124    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c3c40ebe-5cac-4981-913d-18f12d0bf439-istio-podinfo\") pod \"e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-qlhn8\" (UID: \"c3c40ebe-5cac-4981-913d-18f12d0bf439\") " pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-qlhn8"
Oct 31 17:39:46.662667 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:46.662648    4276 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-cfvtd"]
Oct 31 17:39:46.663908 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:46.663881    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c3c40ebe-5cac-4981-913d-18f12d0bf439-istio-token\") pod \"e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-qlhn8\" (UID: \"c3c40ebe-5cac-4981-913d-18f12d0bf439\") " pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-qlhn8"
Oct 31 17:39:46.665182 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:46.665164    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w7v7k\" (UniqueName: \"kubernetes.io/projected/c3c40ebe-5cac-4981-913d-18f12d0bf439-kube-api-access-w7v7k\") pod \"e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-qlhn8\" (UID: \"c3c40ebe-5cac-4981-913d-18f12d0bf439\") " pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-qlhn8"
Oct 31 17:39:46.667771 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:46.667757    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-cfvtd"
Oct 31 17:39:46.686954 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:46.686934    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-cfvtd"]
Oct 31 17:39:46.739735 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:46.739670    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bc5595c7-f428-4092-ac29-a30af2e01065-workload-certs\") pod \"e2e-worker-w6-d80080-openshift-default-956cddf57-cfvtd\" (UID: \"bc5595c7-f428-4092-ac29-a30af2e01065\") " pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-cfvtd"
Oct 31 17:39:46.739735 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:46.739708    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6pxdr\" (UniqueName: \"kubernetes.io/projected/bc5595c7-f428-4092-ac29-a30af2e01065-kube-api-access-6pxdr\") pod \"e2e-worker-w6-d80080-openshift-default-956cddf57-cfvtd\" (UID: \"bc5595c7-f428-4092-ac29-a30af2e01065\") " pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-cfvtd"
Oct 31 17:39:46.739735 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:46.739735    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bc5595c7-f428-4092-ac29-a30af2e01065-istiod-ca-cert\") pod \"e2e-worker-w6-d80080-openshift-default-956cddf57-cfvtd\" (UID: \"bc5595c7-f428-4092-ac29-a30af2e01065\") " pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-cfvtd"
Oct 31 17:39:46.739968 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:46.739854    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bc5595c7-f428-4092-ac29-a30af2e01065-istio-podinfo\") pod \"e2e-worker-w6-d80080-openshift-default-956cddf57-cfvtd\" (UID: \"bc5595c7-f428-4092-ac29-a30af2e01065\") " pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-cfvtd"
Oct 31 17:39:46.739968 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:46.739883    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bc5595c7-f428-4092-ac29-a30af2e01065-workload-socket\") pod \"e2e-worker-w6-d80080-openshift-default-956cddf57-cfvtd\" (UID: \"bc5595c7-f428-4092-ac29-a30af2e01065\") " pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-cfvtd"
Oct 31 17:39:46.739968 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:46.739949    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bc5595c7-f428-4092-ac29-a30af2e01065-istio-envoy\") pod \"e2e-worker-w6-d80080-openshift-default-956cddf57-cfvtd\" (UID: \"bc5595c7-f428-4092-ac29-a30af2e01065\") " pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-cfvtd"
Oct 31 17:39:46.740082 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:46.740023    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bc5595c7-f428-4092-ac29-a30af2e01065-credential-socket\") pod \"e2e-worker-w6-d80080-openshift-default-956cddf57-cfvtd\" (UID: \"bc5595c7-f428-4092-ac29-a30af2e01065\") " pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-cfvtd"
Oct 31 17:39:46.740082 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:46.740041    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bc5595c7-f428-4092-ac29-a30af2e01065-istio-data\") pod \"e2e-worker-w6-d80080-openshift-default-956cddf57-cfvtd\" (UID: \"bc5595c7-f428-4092-ac29-a30af2e01065\") " pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-cfvtd"
Oct 31 17:39:46.740082 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:46.740074    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bc5595c7-f428-4092-ac29-a30af2e01065-istio-token\") pod \"e2e-worker-w6-d80080-openshift-default-956cddf57-cfvtd\" (UID: \"bc5595c7-f428-4092-ac29-a30af2e01065\") " pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-cfvtd"
Oct 31 17:39:46.804246 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:46.804218    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-qlhn8"
Oct 31 17:39:46.841537 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:46.841503    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bc5595c7-f428-4092-ac29-a30af2e01065-workload-certs\") pod \"e2e-worker-w6-d80080-openshift-default-956cddf57-cfvtd\" (UID: \"bc5595c7-f428-4092-ac29-a30af2e01065\") " pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-cfvtd"
Oct 31 17:39:46.841662 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:46.841546    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6pxdr\" (UniqueName: \"kubernetes.io/projected/bc5595c7-f428-4092-ac29-a30af2e01065-kube-api-access-6pxdr\") pod \"e2e-worker-w6-d80080-openshift-default-956cddf57-cfvtd\" (UID: \"bc5595c7-f428-4092-ac29-a30af2e01065\") " pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-cfvtd"
Oct 31 17:39:46.841662 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:46.841568    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bc5595c7-f428-4092-ac29-a30af2e01065-istiod-ca-cert\") pod \"e2e-worker-w6-d80080-openshift-default-956cddf57-cfvtd\" (UID: \"bc5595c7-f428-4092-ac29-a30af2e01065\") " pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-cfvtd"
Oct 31 17:39:46.841662 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:46.841608    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bc5595c7-f428-4092-ac29-a30af2e01065-istio-podinfo\") pod \"e2e-worker-w6-d80080-openshift-default-956cddf57-cfvtd\" (UID: \"bc5595c7-f428-4092-ac29-a30af2e01065\") " pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-cfvtd"
Oct 31 17:39:46.841662 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:46.841640    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bc5595c7-f428-4092-ac29-a30af2e01065-workload-socket\") pod \"e2e-worker-w6-d80080-openshift-default-956cddf57-cfvtd\" (UID: \"bc5595c7-f428-4092-ac29-a30af2e01065\") " pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-cfvtd"
Oct 31 17:39:46.841662 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:46.841659    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bc5595c7-f428-4092-ac29-a30af2e01065-istio-envoy\") pod \"e2e-worker-w6-d80080-openshift-default-956cddf57-cfvtd\" (UID: \"bc5595c7-f428-4092-ac29-a30af2e01065\") " pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-cfvtd"
Oct 31 17:39:46.841926 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:46.841761    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bc5595c7-f428-4092-ac29-a30af2e01065-credential-socket\") pod \"e2e-worker-w6-d80080-openshift-default-956cddf57-cfvtd\" (UID: \"bc5595c7-f428-4092-ac29-a30af2e01065\") " pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-cfvtd"
Oct 31 17:39:46.841926 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:46.841785    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bc5595c7-f428-4092-ac29-a30af2e01065-istio-data\") pod \"e2e-worker-w6-d80080-openshift-default-956cddf57-cfvtd\" (UID: \"bc5595c7-f428-4092-ac29-a30af2e01065\") " pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-cfvtd"
Oct 31 17:39:46.841926 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:46.841833    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bc5595c7-f428-4092-ac29-a30af2e01065-istio-token\") pod \"e2e-worker-w6-d80080-openshift-default-956cddf57-cfvtd\" (UID: \"bc5595c7-f428-4092-ac29-a30af2e01065\") " pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-cfvtd"
Oct 31 17:39:46.842080 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:46.841963    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bc5595c7-f428-4092-ac29-a30af2e01065-workload-certs\") pod \"e2e-worker-w6-d80080-openshift-default-956cddf57-cfvtd\" (UID: \"bc5595c7-f428-4092-ac29-a30af2e01065\") " pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-cfvtd"
Oct 31 17:39:46.842133 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:46.842105    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bc5595c7-f428-4092-ac29-a30af2e01065-workload-socket\") pod \"e2e-worker-w6-d80080-openshift-default-956cddf57-cfvtd\" (UID: \"bc5595c7-f428-4092-ac29-a30af2e01065\") " pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-cfvtd"
Oct 31 17:39:46.842276 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:46.842251    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bc5595c7-f428-4092-ac29-a30af2e01065-istio-data\") pod \"e2e-worker-w6-d80080-openshift-default-956cddf57-cfvtd\" (UID: \"bc5595c7-f428-4092-ac29-a30af2e01065\") " pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-cfvtd"
Oct 31 17:39:46.842374 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:46.842356    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bc5595c7-f428-4092-ac29-a30af2e01065-credential-socket\") pod \"e2e-worker-w6-d80080-openshift-default-956cddf57-cfvtd\" (UID: \"bc5595c7-f428-4092-ac29-a30af2e01065\") " pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-cfvtd"
Oct 31 17:39:46.842530 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:46.842510    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bc5595c7-f428-4092-ac29-a30af2e01065-istiod-ca-cert\") pod \"e2e-worker-w6-d80080-openshift-default-956cddf57-cfvtd\" (UID: \"bc5595c7-f428-4092-ac29-a30af2e01065\") " pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-cfvtd"
Oct 31 17:39:46.844615 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:46.844593    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bc5595c7-f428-4092-ac29-a30af2e01065-istio-envoy\") pod \"e2e-worker-w6-d80080-openshift-default-956cddf57-cfvtd\" (UID: \"bc5595c7-f428-4092-ac29-a30af2e01065\") " pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-cfvtd"
Oct 31 17:39:46.844757 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:46.844740    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bc5595c7-f428-4092-ac29-a30af2e01065-istio-podinfo\") pod \"e2e-worker-w6-d80080-openshift-default-956cddf57-cfvtd\" (UID: \"bc5595c7-f428-4092-ac29-a30af2e01065\") " pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-cfvtd"
Oct 31 17:39:46.864195 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:46.863191    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6pxdr\" (UniqueName: \"kubernetes.io/projected/bc5595c7-f428-4092-ac29-a30af2e01065-kube-api-access-6pxdr\") pod \"e2e-worker-w6-d80080-openshift-default-956cddf57-cfvtd\" (UID: \"bc5595c7-f428-4092-ac29-a30af2e01065\") " pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-cfvtd"
Oct 31 17:39:46.864195 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:46.864109    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bc5595c7-f428-4092-ac29-a30af2e01065-istio-token\") pod \"e2e-worker-w6-d80080-openshift-default-956cddf57-cfvtd\" (UID: \"bc5595c7-f428-4092-ac29-a30af2e01065\") " pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-cfvtd"
Oct 31 17:39:46.996840 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:46.996737    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-cfvtd"
Oct 31 17:39:47.141862 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:47.141838    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-qlhn8"]
Oct 31 17:39:47.142507 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:39:47.142470    4276 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3c40ebe_5cac_4981_913d_18f12d0bf439.slice/crio-2e3de26ce2cb6849e35588910fe921c455392f80c54b6040b7926b943fa54d0b WatchSource:0}: Error finding container 2e3de26ce2cb6849e35588910fe921c455392f80c54b6040b7926b943fa54d0b: Status 404 returned error can't find the container with id 2e3de26ce2cb6849e35588910fe921c455392f80c54b6040b7926b943fa54d0b
Oct 31 17:39:47.145047 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:47.145009    4276 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 17:39:47.145129 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:47.145091    4276 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 17:39:47.145171 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:47.145132    4276 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 17:39:47.302808 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:47.302782    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-cfvtd"]
Oct 31 17:39:47.303310 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:39:47.303284    4276 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc5595c7_f428_4092_ac29_a30af2e01065.slice/crio-f63c2f4768c0c357bd9e08a6a9f96e198508803704ee7a6f08e71db8e2c5c7b3 WatchSource:0}: Error finding container f63c2f4768c0c357bd9e08a6a9f96e198508803704ee7a6f08e71db8e2c5c7b3: Status 404 returned error can't find the container with id f63c2f4768c0c357bd9e08a6a9f96e198508803704ee7a6f08e71db8e2c5c7b3
Oct 31 17:39:47.305919 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:47.305885    4276 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 17:39:47.306016 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:47.305952    4276 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 17:39:47.306016 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:47.305982    4276 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 17:39:47.870681 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:47.870633    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:39:48.204528 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:48.201970    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-qlhn8" event={"ID":"c3c40ebe-5cac-4981-913d-18f12d0bf439","Type":"ContainerStarted","Data":"395370f5dee6ae088f35afe2eeb8f02c629d745ac4ddf5cacb9129b95407042a"}
Oct 31 17:39:48.204528 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:48.202039    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-qlhn8" event={"ID":"c3c40ebe-5cac-4981-913d-18f12d0bf439","Type":"ContainerStarted","Data":"2e3de26ce2cb6849e35588910fe921c455392f80c54b6040b7926b943fa54d0b"}
Oct 31 17:39:48.204994 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:48.204962    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-cfvtd" event={"ID":"bc5595c7-f428-4092-ac29-a30af2e01065","Type":"ContainerStarted","Data":"94875b822b89638cdb029172564865300c6eabb87124433d1493ab36ff0356d6"}
Oct 31 17:39:48.205104 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:48.205031    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-cfvtd" event={"ID":"bc5595c7-f428-4092-ac29-a30af2e01065","Type":"ContainerStarted","Data":"f63c2f4768c0c357bd9e08a6a9f96e198508803704ee7a6f08e71db8e2c5c7b3"}
Oct 31 17:39:48.288466 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:48.284303    4276 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-qlhn8" podStartSLOduration=2.284283569 podStartE2EDuration="2.284283569s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 17:39:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 17:39:48.262919726 +0000 UTC m=+751.060664661" watchObservedRunningTime="2026-10-31 17:39:48.284283569 +0000 UTC m=+751.082028498"
Oct 31 17:39:48.317560 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:48.317509    4276 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-cfvtd" podStartSLOduration=2.317494514 podStartE2EDuration="2.317494514s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 17:39:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 17:39:48.316725199 +0000 UTC m=+751.114470127" watchObservedRunningTime="2026-10-31 17:39:48.317494514 +0000 UTC m=+751.115239441"
Oct 31 17:39:48.807189 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:48.807159    4276 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-qlhn8"
Oct 31 17:39:48.808489 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:48.808465    4276 patch_prober.go:29] interesting pod/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-qlhn8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body=
Oct 31 17:39:48.808602 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:48.808513    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-qlhn8" podUID="c3c40ebe-5cac-4981-913d-18f12d0bf439" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused"
Oct 31 17:39:49.014831 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:49.014789    4276 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-cfvtd"
Oct 31 17:39:49.015358 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:49.015334    4276 patch_prober.go:29] interesting pod/e2e-worker-w6-d80080-openshift-default-956cddf57-cfvtd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body=
Oct 31 17:39:49.015511 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:49.015379    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-cfvtd" podUID="bc5595c7-f428-4092-ac29-a30af2e01065" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused"
Oct 31 17:39:49.038357 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:49.038330    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:39:49.581465 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:49.580863    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:39:49.807053 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:49.806870    4276 patch_prober.go:29] interesting pod/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-qlhn8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body=
Oct 31 17:39:49.807053 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:49.806937    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-qlhn8" podUID="c3c40ebe-5cac-4981-913d-18f12d0bf439" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused"
Oct 31 17:39:50.001580 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:49.999805    4276 patch_prober.go:29] interesting pod/e2e-worker-w6-d80080-openshift-default-956cddf57-cfvtd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body=
Oct 31 17:39:50.001580 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:49.999863    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-cfvtd" podUID="bc5595c7-f428-4092-ac29-a30af2e01065" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused"
Oct 31 17:39:50.805678 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:50.805645    4276 patch_prober.go:29] interesting pod/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-qlhn8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body=
Oct 31 17:39:50.806280 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:50.806247    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-qlhn8" podUID="c3c40ebe-5cac-4981-913d-18f12d0bf439" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused"
Oct 31 17:39:50.999059 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:50.999021    4276 patch_prober.go:29] interesting pod/e2e-worker-w6-d80080-openshift-default-956cddf57-cfvtd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body=
Oct 31 17:39:50.999317 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:50.999290    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-cfvtd" podUID="bc5595c7-f428-4092-ac29-a30af2e01065" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused"
Oct 31 17:39:51.089523 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:51.085424    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:39:51.806470 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:51.804849    4276 patch_prober.go:29] interesting pod/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-qlhn8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body=
Oct 31 17:39:51.806470 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:51.804937    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-qlhn8" podUID="c3c40ebe-5cac-4981-913d-18f12d0bf439" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused"
Oct 31 17:39:51.998467 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:51.997837    4276 patch_prober.go:29] interesting pod/e2e-worker-w6-d80080-openshift-default-956cddf57-cfvtd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body=
Oct 31 17:39:51.998467 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:51.997912    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-cfvtd" podUID="bc5595c7-f428-4092-ac29-a30af2e01065" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused"
Oct 31 17:39:52.691464 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:52.685104    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:39:52.811469 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:52.807868    4276 patch_prober.go:29] interesting pod/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-qlhn8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body=
Oct 31 17:39:52.811469 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:52.807927    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-qlhn8" podUID="c3c40ebe-5cac-4981-913d-18f12d0bf439" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused"
Oct 31 17:39:52.999182 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:52.997588    4276 patch_prober.go:29] interesting pod/e2e-worker-w6-d80080-openshift-default-956cddf57-cfvtd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body=
Oct 31 17:39:52.999182 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:52.997643    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-cfvtd" podUID="bc5595c7-f428-4092-ac29-a30af2e01065" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused"
Oct 31 17:39:53.428764 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:53.426874    4276 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w2-5c446c/distinct-w2-5c446c-kserve-69d97d6968-gtvjv"
Oct 31 17:39:53.515500 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:53.514297    4276 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w2-5c446c/distinct-w2-5c446c-kserve-69d97d6968-gtvjv"
Oct 31 17:39:53.815517 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:53.808774    4276 patch_prober.go:29] interesting pod/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-qlhn8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body=
Oct 31 17:39:53.815517 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:53.808835    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-qlhn8" podUID="c3c40ebe-5cac-4981-913d-18f12d0bf439" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused"
Oct 31 17:39:54.006556 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:53.998798    4276 patch_prober.go:29] interesting pod/e2e-worker-w6-d80080-openshift-default-956cddf57-cfvtd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body=
Oct 31 17:39:54.006556 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:53.998870    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-cfvtd" podUID="bc5595c7-f428-4092-ac29-a30af2e01065" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused"
Oct 31 17:39:54.379962 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:54.379694    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-5c446c/distinct-w2-5c446c-kserve-69d97d6968-gtvjv"
Oct 31 17:39:54.743603 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:54.732506    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:39:54.969007 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:54.968965    4276 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-qlhn8"
Oct 31 17:39:54.973327 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:54.972905    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-qlhn8"
Oct 31 17:39:54.974649 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:54.974624    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-qlhn8"
Oct 31 17:39:55.169357 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:55.169323    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:39:55.455140 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:55.455048    4276 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-cfvtd"
Oct 31 17:39:55.456486 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:55.456462    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-cfvtd"
Oct 31 17:39:55.458945 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:55.458925    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-cfvtd"
Oct 31 17:39:56.345688 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:56.345654    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:39:57.972252 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:57.972217    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:39:59.358370 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:39:59.358323    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:40:00.686222 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:00.686147    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:40:00.874958 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:00.874919    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:40:02.510381 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:02.510352    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:40:02.661165 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:02.661132    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:40:04.056269 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:04.056235    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:40:05.795220 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:05.795184    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:40:07.269845 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:07.269810    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:40:07.807769 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:07.807731    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:40:08.761648 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:08.761616    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:40:10.362221 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:10.362172    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:40:11.869237 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:11.869201    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:40:12.235332 ip-10-0-129-127 kubenswrapper[4276]: E1031 17:40:12.235245    4276 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tg5qk" containerName="extractor"
Oct 31 17:40:13.360647 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:13.360569    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:40:14.860931 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:14.860898    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:40:16.351186 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:16.351157    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:40:16.841046 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:16.841015    4276 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-nqt2x"]
Oct 31 17:40:16.845989 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:16.845969    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-nqt2x"
Oct 31 17:40:16.854928 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:16.854722    4276 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-hl287"]
Oct 31 17:40:16.855889 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:16.855863    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f2899dc7-04e8-40b9-ac3f-6cf6cb019020-istio-podinfo\") pod \"e2e-worker-w2-5c446c-openshift-default-797c6dc855-nqt2x\" (UID: \"f2899dc7-04e8-40b9-ac3f-6cf6cb019020\") " pod="openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-nqt2x"
Oct 31 17:40:16.855979 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:16.855944    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f2899dc7-04e8-40b9-ac3f-6cf6cb019020-credential-socket\") pod \"e2e-worker-w2-5c446c-openshift-default-797c6dc855-nqt2x\" (UID: \"f2899dc7-04e8-40b9-ac3f-6cf6cb019020\") " pod="openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-nqt2x"
Oct 31 17:40:16.856025 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:16.855977    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f2899dc7-04e8-40b9-ac3f-6cf6cb019020-istio-token\") pod \"e2e-worker-w2-5c446c-openshift-default-797c6dc855-nqt2x\" (UID: \"f2899dc7-04e8-40b9-ac3f-6cf6cb019020\") " pod="openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-nqt2x"
Oct 31 17:40:16.856070 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:16.856049    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f2899dc7-04e8-40b9-ac3f-6cf6cb019020-istiod-ca-cert\") pod \"e2e-worker-w2-5c446c-openshift-default-797c6dc855-nqt2x\" (UID: \"f2899dc7-04e8-40b9-ac3f-6cf6cb019020\") " pod="openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-nqt2x"
Oct 31 17:40:16.856106 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:16.856082    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gbw6m\" (UniqueName: \"kubernetes.io/projected/f2899dc7-04e8-40b9-ac3f-6cf6cb019020-kube-api-access-gbw6m\") pod \"e2e-worker-w2-5c446c-openshift-default-797c6dc855-nqt2x\" (UID: \"f2899dc7-04e8-40b9-ac3f-6cf6cb019020\") " pod="openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-nqt2x"
Oct 31 17:40:16.856204 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:16.856188    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f2899dc7-04e8-40b9-ac3f-6cf6cb019020-istio-data\") pod \"e2e-worker-w2-5c446c-openshift-default-797c6dc855-nqt2x\" (UID: \"f2899dc7-04e8-40b9-ac3f-6cf6cb019020\") " pod="openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-nqt2x"
Oct 31 17:40:16.856263 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:16.856248    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f2899dc7-04e8-40b9-ac3f-6cf6cb019020-workload-certs\") pod \"e2e-worker-w2-5c446c-openshift-default-797c6dc855-nqt2x\" (UID: \"f2899dc7-04e8-40b9-ac3f-6cf6cb019020\") " pod="openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-nqt2x"
Oct 31 17:40:16.856327 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:16.856311    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f2899dc7-04e8-40b9-ac3f-6cf6cb019020-workload-socket\") pod \"e2e-worker-w2-5c446c-openshift-default-797c6dc855-nqt2x\" (UID: \"f2899dc7-04e8-40b9-ac3f-6cf6cb019020\") " pod="openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-nqt2x"
Oct 31 17:40:16.856377 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:16.856344    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f2899dc7-04e8-40b9-ac3f-6cf6cb019020-istio-envoy\") pod \"e2e-worker-w2-5c446c-openshift-default-797c6dc855-nqt2x\" (UID: \"f2899dc7-04e8-40b9-ac3f-6cf6cb019020\") " pod="openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-nqt2x"
Oct 31 17:40:16.858581 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:16.858564    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-hl287"
Oct 31 17:40:16.871091 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:16.871063    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-nqt2x"]
Oct 31 17:40:16.884124 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:16.884097    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-hl287"]
Oct 31 17:40:16.957990 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:16.957947    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f2899dc7-04e8-40b9-ac3f-6cf6cb019020-workload-socket\") pod \"e2e-worker-w2-5c446c-openshift-default-797c6dc855-nqt2x\" (UID: \"f2899dc7-04e8-40b9-ac3f-6cf6cb019020\") " pod="openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-nqt2x"
Oct 31 17:40:16.957990 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:16.957990    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f2899dc7-04e8-40b9-ac3f-6cf6cb019020-istio-envoy\") pod \"e2e-worker-w2-5c446c-openshift-default-797c6dc855-nqt2x\" (UID: \"f2899dc7-04e8-40b9-ac3f-6cf6cb019020\") " pod="openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-nqt2x"
Oct 31 17:40:16.958289 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:16.958053    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/daf53be5-f832-4696-96b7-310c33b09f03-istiod-ca-cert\") pod \"e2e-worker-w2-5c446c-openshift-default-797c6dc855-hl287\" (UID: \"daf53be5-f832-4696-96b7-310c33b09f03\") " pod="openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-hl287"
Oct 31 17:40:16.958289 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:16.958091    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f2899dc7-04e8-40b9-ac3f-6cf6cb019020-istio-podinfo\") pod \"e2e-worker-w2-5c446c-openshift-default-797c6dc855-nqt2x\" (UID: \"f2899dc7-04e8-40b9-ac3f-6cf6cb019020\") " pod="openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-nqt2x"
Oct 31 17:40:16.958289 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:16.958114    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/daf53be5-f832-4696-96b7-310c33b09f03-workload-certs\") pod \"e2e-worker-w2-5c446c-openshift-default-797c6dc855-hl287\" (UID: \"daf53be5-f832-4696-96b7-310c33b09f03\") " pod="openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-hl287"
Oct 31 17:40:16.958478 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:16.958367    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/daf53be5-f832-4696-96b7-310c33b09f03-istio-data\") pod \"e2e-worker-w2-5c446c-openshift-default-797c6dc855-hl287\" (UID: \"daf53be5-f832-4696-96b7-310c33b09f03\") " pod="openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-hl287"
Oct 31 17:40:16.958478 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:16.958415    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f2899dc7-04e8-40b9-ac3f-6cf6cb019020-workload-socket\") pod \"e2e-worker-w2-5c446c-openshift-default-797c6dc855-nqt2x\" (UID: \"f2899dc7-04e8-40b9-ac3f-6cf6cb019020\") " pod="openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-nqt2x"
Oct 31 17:40:16.958478 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:16.958435    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f2899dc7-04e8-40b9-ac3f-6cf6cb019020-credential-socket\") pod \"e2e-worker-w2-5c446c-openshift-default-797c6dc855-nqt2x\" (UID: \"f2899dc7-04e8-40b9-ac3f-6cf6cb019020\") " pod="openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-nqt2x"
Oct 31 17:40:16.958648 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:16.958494    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f2899dc7-04e8-40b9-ac3f-6cf6cb019020-istio-token\") pod \"e2e-worker-w2-5c446c-openshift-default-797c6dc855-nqt2x\" (UID: \"f2899dc7-04e8-40b9-ac3f-6cf6cb019020\") " pod="openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-nqt2x"
Oct 31 17:40:16.958648 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:16.958551    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f2899dc7-04e8-40b9-ac3f-6cf6cb019020-istiod-ca-cert\") pod \"e2e-worker-w2-5c446c-openshift-default-797c6dc855-nqt2x\" (UID: \"f2899dc7-04e8-40b9-ac3f-6cf6cb019020\") " pod="openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-nqt2x"
Oct 31 17:40:16.958648 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:16.958578    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gbw6m\" (UniqueName: \"kubernetes.io/projected/f2899dc7-04e8-40b9-ac3f-6cf6cb019020-kube-api-access-gbw6m\") pod \"e2e-worker-w2-5c446c-openshift-default-797c6dc855-nqt2x\" (UID: \"f2899dc7-04e8-40b9-ac3f-6cf6cb019020\") " pod="openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-nqt2x"
Oct 31 17:40:16.958802 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:16.958650    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/daf53be5-f832-4696-96b7-310c33b09f03-istio-token\") pod \"e2e-worker-w2-5c446c-openshift-default-797c6dc855-hl287\" (UID: \"daf53be5-f832-4696-96b7-310c33b09f03\") " pod="openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-hl287"
Oct 31 17:40:16.958802 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:16.958704    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f2899dc7-04e8-40b9-ac3f-6cf6cb019020-istio-data\") pod \"e2e-worker-w2-5c446c-openshift-default-797c6dc855-nqt2x\" (UID: \"f2899dc7-04e8-40b9-ac3f-6cf6cb019020\") " pod="openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-nqt2x"
Oct 31 17:40:16.958802 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:16.958729    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/daf53be5-f832-4696-96b7-310c33b09f03-istio-envoy\") pod \"e2e-worker-w2-5c446c-openshift-default-797c6dc855-hl287\" (UID: \"daf53be5-f832-4696-96b7-310c33b09f03\") " pod="openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-hl287"
Oct 31 17:40:16.958802 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:16.958757    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q4hzg\" (UniqueName: \"kubernetes.io/projected/daf53be5-f832-4696-96b7-310c33b09f03-kube-api-access-q4hzg\") pod \"e2e-worker-w2-5c446c-openshift-default-797c6dc855-hl287\" (UID: \"daf53be5-f832-4696-96b7-310c33b09f03\") " pod="openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-hl287"
Oct 31 17:40:16.958802 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:16.958758    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f2899dc7-04e8-40b9-ac3f-6cf6cb019020-credential-socket\") pod \"e2e-worker-w2-5c446c-openshift-default-797c6dc855-nqt2x\" (UID: \"f2899dc7-04e8-40b9-ac3f-6cf6cb019020\") " pod="openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-nqt2x"
Oct 31 17:40:16.958802 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:16.958763    4276 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-kfkwb"]
Oct 31 17:40:16.959101 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:16.958934    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/daf53be5-f832-4696-96b7-310c33b09f03-workload-socket\") pod \"e2e-worker-w2-5c446c-openshift-default-797c6dc855-hl287\" (UID: \"daf53be5-f832-4696-96b7-310c33b09f03\") " pod="openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-hl287"
Oct 31 17:40:16.959101 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:16.958983    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f2899dc7-04e8-40b9-ac3f-6cf6cb019020-workload-certs\") pod \"e2e-worker-w2-5c446c-openshift-default-797c6dc855-nqt2x\" (UID: \"f2899dc7-04e8-40b9-ac3f-6cf6cb019020\") " pod="openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-nqt2x"
Oct 31 17:40:16.959101 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:16.959012    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/daf53be5-f832-4696-96b7-310c33b09f03-credential-socket\") pod \"e2e-worker-w2-5c446c-openshift-default-797c6dc855-hl287\" (UID: \"daf53be5-f832-4696-96b7-310c33b09f03\") " pod="openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-hl287"
Oct 31 17:40:16.959101 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:16.959049    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/daf53be5-f832-4696-96b7-310c33b09f03-istio-podinfo\") pod \"e2e-worker-w2-5c446c-openshift-default-797c6dc855-hl287\" (UID: \"daf53be5-f832-4696-96b7-310c33b09f03\") " pod="openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-hl287"
Oct 31 17:40:16.959101 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:16.959055    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f2899dc7-04e8-40b9-ac3f-6cf6cb019020-istio-data\") pod \"e2e-worker-w2-5c446c-openshift-default-797c6dc855-nqt2x\" (UID: \"f2899dc7-04e8-40b9-ac3f-6cf6cb019020\") " pod="openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-nqt2x"
Oct 31 17:40:16.959273 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:16.959245    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f2899dc7-04e8-40b9-ac3f-6cf6cb019020-istiod-ca-cert\") pod \"e2e-worker-w2-5c446c-openshift-default-797c6dc855-nqt2x\" (UID: \"f2899dc7-04e8-40b9-ac3f-6cf6cb019020\") " pod="openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-nqt2x"
Oct 31 17:40:16.959317 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:16.959277    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f2899dc7-04e8-40b9-ac3f-6cf6cb019020-workload-certs\") pod \"e2e-worker-w2-5c446c-openshift-default-797c6dc855-nqt2x\" (UID: \"f2899dc7-04e8-40b9-ac3f-6cf6cb019020\") " pod="openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-nqt2x"
Oct 31 17:40:16.960912 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:16.960893    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f2899dc7-04e8-40b9-ac3f-6cf6cb019020-istio-envoy\") pod \"e2e-worker-w2-5c446c-openshift-default-797c6dc855-nqt2x\" (UID: \"f2899dc7-04e8-40b9-ac3f-6cf6cb019020\") " pod="openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-nqt2x"
Oct 31 17:40:16.961089 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:16.961074    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f2899dc7-04e8-40b9-ac3f-6cf6cb019020-istio-podinfo\") pod \"e2e-worker-w2-5c446c-openshift-default-797c6dc855-nqt2x\" (UID: \"f2899dc7-04e8-40b9-ac3f-6cf6cb019020\") " pod="openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-nqt2x"
Oct 31 17:40:16.967956 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:16.967938    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-kfkwb"
Oct 31 17:40:16.980575 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:16.980526    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f2899dc7-04e8-40b9-ac3f-6cf6cb019020-istio-token\") pod \"e2e-worker-w2-5c446c-openshift-default-797c6dc855-nqt2x\" (UID: \"f2899dc7-04e8-40b9-ac3f-6cf6cb019020\") " pod="openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-nqt2x"
Oct 31 17:40:16.980681 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:16.980589    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gbw6m\" (UniqueName: \"kubernetes.io/projected/f2899dc7-04e8-40b9-ac3f-6cf6cb019020-kube-api-access-gbw6m\") pod \"e2e-worker-w2-5c446c-openshift-default-797c6dc855-nqt2x\" (UID: \"f2899dc7-04e8-40b9-ac3f-6cf6cb019020\") " pod="openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-nqt2x"
Oct 31 17:40:16.984065 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:16.984042    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-kfkwb"]
Oct 31 17:40:17.060628 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:17.060588    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b04b1b0-d2cc-4304-92e6-90e48105f442-workload-socket\") pod \"e2e-worker-w0-ee816e-openshift-default-c4db85b5c-kfkwb\" (UID: \"4b04b1b0-d2cc-4304-92e6-90e48105f442\") " pod="openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-kfkwb"
Oct 31 17:40:17.060831 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:17.060638    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vv9w2\" (UniqueName: \"kubernetes.io/projected/4b04b1b0-d2cc-4304-92e6-90e48105f442-kube-api-access-vv9w2\") pod \"e2e-worker-w0-ee816e-openshift-default-c4db85b5c-kfkwb\" (UID: \"4b04b1b0-d2cc-4304-92e6-90e48105f442\") " pod="openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-kfkwb"
Oct 31 17:40:17.060831 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:17.060725    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4b04b1b0-d2cc-4304-92e6-90e48105f442-istio-data\") pod \"e2e-worker-w0-ee816e-openshift-default-c4db85b5c-kfkwb\" (UID: \"4b04b1b0-d2cc-4304-92e6-90e48105f442\") " pod="openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-kfkwb"
Oct 31 17:40:17.060831 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:17.060764    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4b04b1b0-d2cc-4304-92e6-90e48105f442-istio-podinfo\") pod \"e2e-worker-w0-ee816e-openshift-default-c4db85b5c-kfkwb\" (UID: \"4b04b1b0-d2cc-4304-92e6-90e48105f442\") " pod="openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-kfkwb"
Oct 31 17:40:17.060965 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:17.060831    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4b04b1b0-d2cc-4304-92e6-90e48105f442-istio-token\") pod \"e2e-worker-w0-ee816e-openshift-default-c4db85b5c-kfkwb\" (UID: \"4b04b1b0-d2cc-4304-92e6-90e48105f442\") " pod="openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-kfkwb"
Oct 31 17:40:17.060965 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:17.060866    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/daf53be5-f832-4696-96b7-310c33b09f03-istiod-ca-cert\") pod \"e2e-worker-w2-5c446c-openshift-default-797c6dc855-hl287\" (UID: \"daf53be5-f832-4696-96b7-310c33b09f03\") " pod="openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-hl287"
Oct 31 17:40:17.060965 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:17.060891    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/daf53be5-f832-4696-96b7-310c33b09f03-workload-certs\") pod \"e2e-worker-w2-5c446c-openshift-default-797c6dc855-hl287\" (UID: \"daf53be5-f832-4696-96b7-310c33b09f03\") " pod="openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-hl287"
Oct 31 17:40:17.060965 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:17.060912    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4b04b1b0-d2cc-4304-92e6-90e48105f442-istiod-ca-cert\") pod \"e2e-worker-w0-ee816e-openshift-default-c4db85b5c-kfkwb\" (UID: \"4b04b1b0-d2cc-4304-92e6-90e48105f442\") " pod="openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-kfkwb"
Oct 31 17:40:17.060965 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:17.060961    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/daf53be5-f832-4696-96b7-310c33b09f03-istio-data\") pod \"e2e-worker-w2-5c446c-openshift-default-797c6dc855-hl287\" (UID: \"daf53be5-f832-4696-96b7-310c33b09f03\") " pod="openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-hl287"
Oct 31 17:40:17.061209 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:17.060981    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4b04b1b0-d2cc-4304-92e6-90e48105f442-istio-envoy\") pod \"e2e-worker-w0-ee816e-openshift-default-c4db85b5c-kfkwb\" (UID: \"4b04b1b0-d2cc-4304-92e6-90e48105f442\") " pod="openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-kfkwb"
Oct 31 17:40:17.061209 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:17.061154    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/daf53be5-f832-4696-96b7-310c33b09f03-istio-token\") pod \"e2e-worker-w2-5c446c-openshift-default-797c6dc855-hl287\" (UID: \"daf53be5-f832-4696-96b7-310c33b09f03\") " pod="openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-hl287"
Oct 31 17:40:17.061209 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:17.061199    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/daf53be5-f832-4696-96b7-310c33b09f03-istio-envoy\") pod \"e2e-worker-w2-5c446c-openshift-default-797c6dc855-hl287\" (UID: \"daf53be5-f832-4696-96b7-310c33b09f03\") " pod="openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-hl287"
Oct 31 17:40:17.061367 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:17.061287    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q4hzg\" (UniqueName: \"kubernetes.io/projected/daf53be5-f832-4696-96b7-310c33b09f03-kube-api-access-q4hzg\") pod \"e2e-worker-w2-5c446c-openshift-default-797c6dc855-hl287\" (UID: \"daf53be5-f832-4696-96b7-310c33b09f03\") " pod="openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-hl287"
Oct 31 17:40:17.061367 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:17.061325    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/daf53be5-f832-4696-96b7-310c33b09f03-workload-socket\") pod \"e2e-worker-w2-5c446c-openshift-default-797c6dc855-hl287\" (UID: \"daf53be5-f832-4696-96b7-310c33b09f03\") " pod="openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-hl287"
Oct 31 17:40:17.061671 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:17.061370    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/daf53be5-f832-4696-96b7-310c33b09f03-credential-socket\") pod \"e2e-worker-w2-5c446c-openshift-default-797c6dc855-hl287\" (UID: \"daf53be5-f832-4696-96b7-310c33b09f03\") " pod="openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-hl287"
Oct 31 17:40:17.061671 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:17.061395    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/daf53be5-f832-4696-96b7-310c33b09f03-istio-podinfo\") pod \"e2e-worker-w2-5c446c-openshift-default-797c6dc855-hl287\" (UID: \"daf53be5-f832-4696-96b7-310c33b09f03\") " pod="openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-hl287"
Oct 31 17:40:17.061671 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:17.061423    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b04b1b0-d2cc-4304-92e6-90e48105f442-credential-socket\") pod \"e2e-worker-w0-ee816e-openshift-default-c4db85b5c-kfkwb\" (UID: \"4b04b1b0-d2cc-4304-92e6-90e48105f442\") " pod="openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-kfkwb"
Oct 31 17:40:17.061671 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:17.061493    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4b04b1b0-d2cc-4304-92e6-90e48105f442-workload-certs\") pod \"e2e-worker-w0-ee816e-openshift-default-c4db85b5c-kfkwb\" (UID: \"4b04b1b0-d2cc-4304-92e6-90e48105f442\") " pod="openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-kfkwb"
Oct 31 17:40:17.061671 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:17.061491    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/daf53be5-f832-4696-96b7-310c33b09f03-workload-certs\") pod \"e2e-worker-w2-5c446c-openshift-default-797c6dc855-hl287\" (UID: \"daf53be5-f832-4696-96b7-310c33b09f03\") " pod="openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-hl287"
Oct 31 17:40:17.061671 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:17.061572    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/daf53be5-f832-4696-96b7-310c33b09f03-istio-data\") pod \"e2e-worker-w2-5c446c-openshift-default-797c6dc855-hl287\" (UID: \"daf53be5-f832-4696-96b7-310c33b09f03\") " pod="openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-hl287"
Oct 31 17:40:17.061929 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:17.061707    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/daf53be5-f832-4696-96b7-310c33b09f03-istiod-ca-cert\") pod \"e2e-worker-w2-5c446c-openshift-default-797c6dc855-hl287\" (UID: \"daf53be5-f832-4696-96b7-310c33b09f03\") " pod="openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-hl287"
Oct 31 17:40:17.061929 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:17.061899    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/daf53be5-f832-4696-96b7-310c33b09f03-workload-socket\") pod \"e2e-worker-w2-5c446c-openshift-default-797c6dc855-hl287\" (UID: \"daf53be5-f832-4696-96b7-310c33b09f03\") " pod="openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-hl287"
Oct 31 17:40:17.062004 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:17.061978    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/daf53be5-f832-4696-96b7-310c33b09f03-credential-socket\") pod \"e2e-worker-w2-5c446c-openshift-default-797c6dc855-hl287\" (UID: \"daf53be5-f832-4696-96b7-310c33b09f03\") " pod="openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-hl287"
Oct 31 17:40:17.063531 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:17.063513    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/daf53be5-f832-4696-96b7-310c33b09f03-istio-envoy\") pod \"e2e-worker-w2-5c446c-openshift-default-797c6dc855-hl287\" (UID: \"daf53be5-f832-4696-96b7-310c33b09f03\") " pod="openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-hl287"
Oct 31 17:40:17.064060 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:17.064038    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/daf53be5-f832-4696-96b7-310c33b09f03-istio-podinfo\") pod \"e2e-worker-w2-5c446c-openshift-default-797c6dc855-hl287\" (UID: \"daf53be5-f832-4696-96b7-310c33b09f03\") " pod="openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-hl287"
Oct 31 17:40:17.081617 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:17.081590    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/daf53be5-f832-4696-96b7-310c33b09f03-istio-token\") pod \"e2e-worker-w2-5c446c-openshift-default-797c6dc855-hl287\" (UID: \"daf53be5-f832-4696-96b7-310c33b09f03\") " pod="openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-hl287"
Oct 31 17:40:17.081739 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:17.081705    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q4hzg\" (UniqueName: \"kubernetes.io/projected/daf53be5-f832-4696-96b7-310c33b09f03-kube-api-access-q4hzg\") pod \"e2e-worker-w2-5c446c-openshift-default-797c6dc855-hl287\" (UID: \"daf53be5-f832-4696-96b7-310c33b09f03\") " pod="openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-hl287"
Oct 31 17:40:17.162143 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:17.162051    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-nqt2x"
Oct 31 17:40:17.162512 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:17.162490    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b04b1b0-d2cc-4304-92e6-90e48105f442-workload-socket\") pod \"e2e-worker-w0-ee816e-openshift-default-c4db85b5c-kfkwb\" (UID: \"4b04b1b0-d2cc-4304-92e6-90e48105f442\") " pod="openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-kfkwb"
Oct 31 17:40:17.162577 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:17.162535    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vv9w2\" (UniqueName: \"kubernetes.io/projected/4b04b1b0-d2cc-4304-92e6-90e48105f442-kube-api-access-vv9w2\") pod \"e2e-worker-w0-ee816e-openshift-default-c4db85b5c-kfkwb\" (UID: \"4b04b1b0-d2cc-4304-92e6-90e48105f442\") " pod="openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-kfkwb"
Oct 31 17:40:17.162577 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:17.162558    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4b04b1b0-d2cc-4304-92e6-90e48105f442-istio-data\") pod \"e2e-worker-w0-ee816e-openshift-default-c4db85b5c-kfkwb\" (UID: \"4b04b1b0-d2cc-4304-92e6-90e48105f442\") " pod="openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-kfkwb"
Oct 31 17:40:17.162664 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:17.162581    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4b04b1b0-d2cc-4304-92e6-90e48105f442-istio-podinfo\") pod \"e2e-worker-w0-ee816e-openshift-default-c4db85b5c-kfkwb\" (UID: \"4b04b1b0-d2cc-4304-92e6-90e48105f442\") " pod="openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-kfkwb"
Oct 31 17:40:17.162664 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:17.162599    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4b04b1b0-d2cc-4304-92e6-90e48105f442-istio-token\") pod \"e2e-worker-w0-ee816e-openshift-default-c4db85b5c-kfkwb\" (UID: \"4b04b1b0-d2cc-4304-92e6-90e48105f442\") " pod="openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-kfkwb"
Oct 31 17:40:17.162664 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:17.162632    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4b04b1b0-d2cc-4304-92e6-90e48105f442-istiod-ca-cert\") pod \"e2e-worker-w0-ee816e-openshift-default-c4db85b5c-kfkwb\" (UID: \"4b04b1b0-d2cc-4304-92e6-90e48105f442\") " pod="openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-kfkwb"
Oct 31 17:40:17.162810 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:17.162669    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4b04b1b0-d2cc-4304-92e6-90e48105f442-istio-envoy\") pod \"e2e-worker-w0-ee816e-openshift-default-c4db85b5c-kfkwb\" (UID: \"4b04b1b0-d2cc-4304-92e6-90e48105f442\") " pod="openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-kfkwb"
Oct 31 17:40:17.162924 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:17.162896    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b04b1b0-d2cc-4304-92e6-90e48105f442-credential-socket\") pod \"e2e-worker-w0-ee816e-openshift-default-c4db85b5c-kfkwb\" (UID: \"4b04b1b0-d2cc-4304-92e6-90e48105f442\") " pod="openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-kfkwb"
Oct 31 17:40:17.162989 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:17.162936    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b04b1b0-d2cc-4304-92e6-90e48105f442-workload-socket\") pod \"e2e-worker-w0-ee816e-openshift-default-c4db85b5c-kfkwb\" (UID: \"4b04b1b0-d2cc-4304-92e6-90e48105f442\") " pod="openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-kfkwb"
Oct 31 17:40:17.162989 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:17.162948    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4b04b1b0-d2cc-4304-92e6-90e48105f442-workload-certs\") pod \"e2e-worker-w0-ee816e-openshift-default-c4db85b5c-kfkwb\" (UID: \"4b04b1b0-d2cc-4304-92e6-90e48105f442\") " pod="openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-kfkwb"
Oct 31 17:40:17.162989 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:17.162976    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4b04b1b0-d2cc-4304-92e6-90e48105f442-istio-data\") pod \"e2e-worker-w0-ee816e-openshift-default-c4db85b5c-kfkwb\" (UID: \"4b04b1b0-d2cc-4304-92e6-90e48105f442\") " pod="openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-kfkwb"
Oct 31 17:40:17.163271 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:17.163245    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4b04b1b0-d2cc-4304-92e6-90e48105f442-workload-certs\") pod \"e2e-worker-w0-ee816e-openshift-default-c4db85b5c-kfkwb\" (UID: \"4b04b1b0-d2cc-4304-92e6-90e48105f442\") " pod="openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-kfkwb"
Oct 31 17:40:17.163354 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:17.163275    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b04b1b0-d2cc-4304-92e6-90e48105f442-credential-socket\") pod \"e2e-worker-w0-ee816e-openshift-default-c4db85b5c-kfkwb\" (UID: \"4b04b1b0-d2cc-4304-92e6-90e48105f442\") " pod="openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-kfkwb"
Oct 31 17:40:17.163413 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:17.163388    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4b04b1b0-d2cc-4304-92e6-90e48105f442-istiod-ca-cert\") pod \"e2e-worker-w0-ee816e-openshift-default-c4db85b5c-kfkwb\" (UID: \"4b04b1b0-d2cc-4304-92e6-90e48105f442\") " pod="openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-kfkwb"
Oct 31 17:40:17.164937 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:17.164911    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4b04b1b0-d2cc-4304-92e6-90e48105f442-istio-envoy\") pod \"e2e-worker-w0-ee816e-openshift-default-c4db85b5c-kfkwb\" (UID: \"4b04b1b0-d2cc-4304-92e6-90e48105f442\") " pod="openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-kfkwb"
Oct 31 17:40:17.165353 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:17.165336    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4b04b1b0-d2cc-4304-92e6-90e48105f442-istio-podinfo\") pod \"e2e-worker-w0-ee816e-openshift-default-c4db85b5c-kfkwb\" (UID: \"4b04b1b0-d2cc-4304-92e6-90e48105f442\") " pod="openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-kfkwb"
Oct 31 17:40:17.180641 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:17.180621    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-hl287"
Oct 31 17:40:17.183046 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:17.183027    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4b04b1b0-d2cc-4304-92e6-90e48105f442-istio-token\") pod \"e2e-worker-w0-ee816e-openshift-default-c4db85b5c-kfkwb\" (UID: \"4b04b1b0-d2cc-4304-92e6-90e48105f442\") " pod="openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-kfkwb"
Oct 31 17:40:17.183295 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:17.183276    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vv9w2\" (UniqueName: \"kubernetes.io/projected/4b04b1b0-d2cc-4304-92e6-90e48105f442-kube-api-access-vv9w2\") pod \"e2e-worker-w0-ee816e-openshift-default-c4db85b5c-kfkwb\" (UID: \"4b04b1b0-d2cc-4304-92e6-90e48105f442\") " pod="openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-kfkwb"
Oct 31 17:40:17.310566 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:17.310538    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-kfkwb"
Oct 31 17:40:17.484022 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:40:17.483991    4276 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2899dc7_04e8_40b9_ac3f_6cf6cb019020.slice/crio-96a5a35e488f47f60fecf7c894fcefac02d49d50013248f66e8d4844fec4aa9f WatchSource:0}: Error finding container 96a5a35e488f47f60fecf7c894fcefac02d49d50013248f66e8d4844fec4aa9f: Status 404 returned error can't find the container with id 96a5a35e488f47f60fecf7c894fcefac02d49d50013248f66e8d4844fec4aa9f
Oct 31 17:40:17.484387 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:17.484181    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-nqt2x"]
Oct 31 17:40:17.486412 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:17.486380    4276 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 17:40:17.486534 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:17.486472    4276 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 17:40:17.486534 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:17.486516    4276 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 17:40:17.525653 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:17.525633    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-hl287"]
Oct 31 17:40:17.525827 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:40:17.525802    4276 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddaf53be5_f832_4696_96b7_310c33b09f03.slice/crio-50bdd6ba867785971a90e7baeb48a1885059eb1f77f71d5873016c03ddb9e397 WatchSource:0}: Error finding container 50bdd6ba867785971a90e7baeb48a1885059eb1f77f71d5873016c03ddb9e397: Status 404 returned error can't find the container with id 50bdd6ba867785971a90e7baeb48a1885059eb1f77f71d5873016c03ddb9e397
Oct 31 17:40:17.528515 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:17.528488    4276 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 17:40:17.528592 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:17.528546    4276 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 17:40:17.528592 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:17.528576    4276 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 17:40:17.615382 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:17.615279    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-hl287" event={"ID":"daf53be5-f832-4696-96b7-310c33b09f03","Type":"ContainerStarted","Data":"247fc7436ec961ce182dd395a926a2c8ef25f7cc3d398880c0503bd8774291ce"}
Oct 31 17:40:17.615382 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:17.615347    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-hl287" event={"ID":"daf53be5-f832-4696-96b7-310c33b09f03","Type":"ContainerStarted","Data":"50bdd6ba867785971a90e7baeb48a1885059eb1f77f71d5873016c03ddb9e397"}
Oct 31 17:40:17.619643 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:17.619601    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-nqt2x" event={"ID":"f2899dc7-04e8-40b9-ac3f-6cf6cb019020","Type":"ContainerStarted","Data":"7ecff18217515087184c4171450b64676aee451bb8da3d527e4059ecb26fcfaf"}
Oct 31 17:40:17.619840 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:17.619662    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-nqt2x" event={"ID":"f2899dc7-04e8-40b9-ac3f-6cf6cb019020","Type":"ContainerStarted","Data":"96a5a35e488f47f60fecf7c894fcefac02d49d50013248f66e8d4844fec4aa9f"}
Oct 31 17:40:17.620512 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:17.620455    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-kfkwb"]
Oct 31 17:40:17.625939 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:40:17.625904    4276 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b04b1b0_d2cc_4304_92e6_90e48105f442.slice/crio-0ad611b8b13fdeb9160e042202344ebfba288506288598aea295199c24338850 WatchSource:0}: Error finding container 0ad611b8b13fdeb9160e042202344ebfba288506288598aea295199c24338850: Status 404 returned error can't find the container with id 0ad611b8b13fdeb9160e042202344ebfba288506288598aea295199c24338850
Oct 31 17:40:17.629054 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:17.629016    4276 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 17:40:17.629421 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:17.629318    4276 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 17:40:17.629421 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:17.629393    4276 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 17:40:17.661473 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:17.661389    4276 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-hl287" podStartSLOduration=1.661371369 podStartE2EDuration="1.661371369s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 17:40:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 17:40:17.657968145 +0000 UTC m=+780.455713074" watchObservedRunningTime="2026-10-31 17:40:17.661371369 +0000 UTC m=+780.459116300"
Oct 31 17:40:17.707481 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:17.705915    4276 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-nqt2x" podStartSLOduration=1.7058975429999998 podStartE2EDuration="1.705897543s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 17:40:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 17:40:17.703397087 +0000 UTC m=+780.501142018" watchObservedRunningTime="2026-10-31 17:40:17.705897543 +0000 UTC m=+780.503642472"
Oct 31 17:40:17.867430 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:17.866694    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:40:17.994535 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:17.992864    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:40:18.162941 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:18.162898    4276 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-nqt2x"
Oct 31 17:40:18.164318 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:18.164294    4276 patch_prober.go:29] interesting pod/e2e-worker-w2-5c446c-openshift-default-797c6dc855-nqt2x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body=
Oct 31 17:40:18.164388 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:18.164346    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-nqt2x" podUID="f2899dc7-04e8-40b9-ac3f-6cf6cb019020" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused"
Oct 31 17:40:18.181814 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:18.181789    4276 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-hl287"
Oct 31 17:40:18.183036 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:18.183010    4276 patch_prober.go:29] interesting pod/e2e-worker-w2-5c446c-openshift-default-797c6dc855-hl287 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Oct 31 17:40:18.183150 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:18.183068    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-hl287" podUID="daf53be5-f832-4696-96b7-310c33b09f03" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Oct 31 17:40:18.626894 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:18.626855    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-kfkwb" event={"ID":"4b04b1b0-d2cc-4304-92e6-90e48105f442","Type":"ContainerStarted","Data":"a819b54f19dd2fb68538d2c781d08d3ab2856ca18c49b5f01808e5bc0c355a28"}
Oct 31 17:40:18.627323 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:18.626913    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-kfkwb" event={"ID":"4b04b1b0-d2cc-4304-92e6-90e48105f442","Type":"ContainerStarted","Data":"0ad611b8b13fdeb9160e042202344ebfba288506288598aea295199c24338850"}
Oct 31 17:40:18.691466 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:18.690735    4276 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-kfkwb" podStartSLOduration=2.690715184 podStartE2EDuration="2.690715184s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 17:40:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 17:40:18.67073591 +0000 UTC m=+781.468480839" watchObservedRunningTime="2026-10-31 17:40:18.690715184 +0000 UTC m=+781.488460115"
Oct 31 17:40:19.023524 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:19.023092    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:40:19.172463 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:19.169507    4276 patch_prober.go:29] interesting pod/e2e-worker-w2-5c446c-openshift-default-797c6dc855-nqt2x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body=
Oct 31 17:40:19.172463 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:19.169661    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-nqt2x" podUID="f2899dc7-04e8-40b9-ac3f-6cf6cb019020" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused"
Oct 31 17:40:19.181142 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:19.181107    4276 patch_prober.go:29] interesting pod/e2e-worker-w2-5c446c-openshift-default-797c6dc855-hl287 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Oct 31 17:40:19.181624 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:19.181379    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-hl287" podUID="daf53be5-f832-4696-96b7-310c33b09f03" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Oct 31 17:40:19.320648 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:19.320620    4276 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-kfkwb"
Oct 31 17:40:19.321040 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:19.321019    4276 patch_prober.go:29] interesting pod/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-kfkwb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body=
Oct 31 17:40:19.321248 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:19.321224    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-kfkwb" podUID="4b04b1b0-d2cc-4304-92e6-90e48105f442" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused"
Oct 31 17:40:19.721906 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:19.721818    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:40:20.172076 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:20.172042    4276 patch_prober.go:29] interesting pod/e2e-worker-w2-5c446c-openshift-default-797c6dc855-nqt2x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body=
Oct 31 17:40:20.172346 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:20.172321    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-nqt2x" podUID="f2899dc7-04e8-40b9-ac3f-6cf6cb019020" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused"
Oct 31 17:40:20.188710 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:20.188679    4276 patch_prober.go:29] interesting pod/e2e-worker-w2-5c446c-openshift-default-797c6dc855-hl287 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Oct 31 17:40:20.188940 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:20.188913    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-hl287" podUID="daf53be5-f832-4696-96b7-310c33b09f03" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Oct 31 17:40:20.319468 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:20.318425    4276 patch_prober.go:29] interesting pod/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-kfkwb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body=
Oct 31 17:40:20.319468 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:20.318503    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-kfkwb" podUID="4b04b1b0-d2cc-4304-92e6-90e48105f442" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused"
Oct 31 17:40:21.170877 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:21.170841    4276 patch_prober.go:29] interesting pod/e2e-worker-w2-5c446c-openshift-default-797c6dc855-nqt2x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body=
Oct 31 17:40:21.171656 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:21.171619    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-nqt2x" podUID="f2899dc7-04e8-40b9-ac3f-6cf6cb019020" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused"
Oct 31 17:40:21.189847 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:21.189809    4276 patch_prober.go:29] interesting pod/e2e-worker-w2-5c446c-openshift-default-797c6dc855-hl287 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Oct 31 17:40:21.190104 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:21.190074    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-hl287" podUID="daf53be5-f832-4696-96b7-310c33b09f03" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Oct 31 17:40:21.278467 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:21.278330    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:40:21.315467 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:21.314836    4276 patch_prober.go:29] interesting pod/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-kfkwb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body=
Oct 31 17:40:21.315467 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:21.314896    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-kfkwb" podUID="4b04b1b0-d2cc-4304-92e6-90e48105f442" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused"
Oct 31 17:40:22.170464 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:22.162853    4276 patch_prober.go:29] interesting pod/e2e-worker-w2-5c446c-openshift-default-797c6dc855-nqt2x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body=
Oct 31 17:40:22.170464 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:22.162923    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-nqt2x" podUID="f2899dc7-04e8-40b9-ac3f-6cf6cb019020" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused"
Oct 31 17:40:22.189466 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:22.184669    4276 patch_prober.go:29] interesting pod/e2e-worker-w2-5c446c-openshift-default-797c6dc855-hl287 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Oct 31 17:40:22.189466 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:22.184744    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-hl287" podUID="daf53be5-f832-4696-96b7-310c33b09f03" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Oct 31 17:40:22.313466 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:22.312961    4276 patch_prober.go:29] interesting pod/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-kfkwb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body=
Oct 31 17:40:22.313466 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:22.313019    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-kfkwb" podUID="4b04b1b0-d2cc-4304-92e6-90e48105f442" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused"
Oct 31 17:40:22.975262 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:22.975220    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:40:23.176494 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:23.173276    4276 patch_prober.go:29] interesting pod/e2e-worker-w2-5c446c-openshift-default-797c6dc855-nqt2x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body=
Oct 31 17:40:23.176494 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:23.173335    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-nqt2x" podUID="f2899dc7-04e8-40b9-ac3f-6cf6cb019020" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused"
Oct 31 17:40:23.188554 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:23.187879    4276 patch_prober.go:29] interesting pod/e2e-worker-w2-5c446c-openshift-default-797c6dc855-hl287 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Oct 31 17:40:23.188554 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:23.187940    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-hl287" podUID="daf53be5-f832-4696-96b7-310c33b09f03" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Oct 31 17:40:23.313987 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:23.313947    4276 patch_prober.go:29] interesting pod/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-kfkwb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body=
Oct 31 17:40:23.314649 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:23.314602    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-kfkwb" podUID="4b04b1b0-d2cc-4304-92e6-90e48105f442" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused"
Oct 31 17:40:24.170481 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:24.169655    4276 patch_prober.go:29] interesting pod/e2e-worker-w2-5c446c-openshift-default-797c6dc855-nqt2x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body=
Oct 31 17:40:24.170481 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:24.169717    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-nqt2x" podUID="f2899dc7-04e8-40b9-ac3f-6cf6cb019020" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused"
Oct 31 17:40:24.187466 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:24.187173    4276 patch_prober.go:29] interesting pod/e2e-worker-w2-5c446c-openshift-default-797c6dc855-hl287 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Oct 31 17:40:24.187466 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:24.187230    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-hl287" podUID="daf53be5-f832-4696-96b7-310c33b09f03" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Oct 31 17:40:24.314906 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:24.314867    4276 patch_prober.go:29] interesting pod/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-kfkwb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body=
Oct 31 17:40:24.315567 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:24.315518    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-kfkwb" podUID="4b04b1b0-d2cc-4304-92e6-90e48105f442" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused"
Oct 31 17:40:24.383588 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:24.383523    4276 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-ee816e/premium-w0-ee816e-kserve-65c86cd6db-lnt9p" podUID="e536c902-52a9-4b36-a2ad-eb5500781a5e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.49:8000/health\": context deadline exceeded"
Oct 31 17:40:24.423645 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:24.423537    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-d80080/distinct-w6-d80080-kserve-664576c8f5-6jc4d" podUID="b9c2f445-a191-4b00-802a-1b0ded068411" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": context deadline exceeded"
Oct 31 17:40:24.428559 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:24.428518    4276 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-5c446c/distinct-w2-5c446c-kserve-69d97d6968-gtvjv" podUID="2d1185cc-bac1-45ca-a250-8f44f8589162" containerName="main" probeResult="failure" output="Get \"https://10.133.0.58:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 17:40:24.506584 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:24.506530    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ee816e/premium-w0-ee816e-kserve-65c86cd6db-lnt9p" podUID="e536c902-52a9-4b36-a2ad-eb5500781a5e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.49:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 17:40:24.867941 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:24.867904    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:40:25.163024 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:25.162912    4276 patch_prober.go:29] interesting pod/e2e-worker-w2-5c446c-openshift-default-797c6dc855-nqt2x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body=
Oct 31 17:40:25.163300 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:25.163250    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-nqt2x" podUID="f2899dc7-04e8-40b9-ac3f-6cf6cb019020" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused"
Oct 31 17:40:25.186638 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:25.186514    4276 patch_prober.go:29] interesting pod/e2e-worker-w2-5c446c-openshift-default-797c6dc855-hl287 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Oct 31 17:40:25.187027 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:25.186898    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-hl287" podUID="daf53be5-f832-4696-96b7-310c33b09f03" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Oct 31 17:40:25.343481 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:25.314872    4276 patch_prober.go:29] interesting pod/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-kfkwb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body=
Oct 31 17:40:25.343481 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:25.314940    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-kfkwb" podUID="4b04b1b0-d2cc-4304-92e6-90e48105f442" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused"
Oct 31 17:40:25.377602 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:25.377552    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-5c446c/distinct-w2-5c446c-kserve-69d97d6968-gtvjv" podUID="2d1185cc-bac1-45ca-a250-8f44f8589162" containerName="main" probeResult="failure" output="Get \"https://10.133.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 17:40:25.449531 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:25.445156    4276 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w6-d80080/distinct-w6-d80080-kserve-664576c8f5-6jc4d" podUID="b9c2f445-a191-4b00-802a-1b0ded068411" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/health\": context deadline exceeded"
Oct 31 17:40:25.449863 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:25.449827    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-d80080/distinct-w6-d80080-kserve-664576c8f5-6jc4d" podUID="b9c2f445-a191-4b00-802a-1b0ded068411" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 17:40:25.524893 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:25.507514    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ee816e/premium-w0-ee816e-kserve-65c86cd6db-lnt9p" podUID="e536c902-52a9-4b36-a2ad-eb5500781a5e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.49:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 17:40:25.524893 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:25.508330    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-0e59ca/premium-w1-0e59ca-kserve-58765f9746-fqwq7" podUID="8cf1e52e-70f8-42e2-bdf3-a315e17f10f5" containerName="main" probeResult="failure" output="Get \"https://10.133.0.48:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 17:40:25.528079 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:25.526241    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-d80080/distinct-w6-d80080-kserve-664576c8f5-6jc4d"
Oct 31 17:40:25.528079 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:25.526311    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-ee816e/premium-w0-ee816e-kserve-65c86cd6db-lnt9p"
Oct 31 17:40:25.538089 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:25.538057    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-ee816e/premium-w0-ee816e-kserve-65c86cd6db-lnt9p"
Oct 31 17:40:25.564677 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:25.564642    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-d80080/distinct-w6-d80080-kserve-664576c8f5-6jc4d"
Oct 31 17:40:26.163594 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:26.163539    4276 patch_prober.go:29] interesting pod/e2e-worker-w2-5c446c-openshift-default-797c6dc855-nqt2x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body=
Oct 31 17:40:26.163852 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:26.163614    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-nqt2x" podUID="f2899dc7-04e8-40b9-ac3f-6cf6cb019020" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused"
Oct 31 17:40:26.182869 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:26.182832    4276 patch_prober.go:29] interesting pod/e2e-worker-w2-5c446c-openshift-default-797c6dc855-hl287 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Oct 31 17:40:26.183068 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:26.182893    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-hl287" podUID="daf53be5-f832-4696-96b7-310c33b09f03" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Oct 31 17:40:26.281125 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:26.281067    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:40:26.312413 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:26.312368    4276 patch_prober.go:29] interesting pod/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-kfkwb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body=
Oct 31 17:40:26.312650 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:26.312464    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-kfkwb" podUID="4b04b1b0-d2cc-4304-92e6-90e48105f442" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused"
Oct 31 17:40:26.420265 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:26.420162    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:40:27.169578 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:27.169540    4276 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-nqt2x"
Oct 31 17:40:27.170021 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:27.169999    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-nqt2x"
Oct 31 17:40:27.170874 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:27.170853    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-nqt2x"
Oct 31 17:40:27.186538 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:27.186511    4276 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-hl287"
Oct 31 17:40:27.186937 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:27.186906    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-hl287"
Oct 31 17:40:27.187491 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:27.187473    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-hl287"
Oct 31 17:40:27.317175 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:27.317145    4276 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-kfkwb"
Oct 31 17:40:27.317791 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:27.317763    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-kfkwb"
Oct 31 17:40:27.318728 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:27.318708    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-kfkwb"
Oct 31 17:40:28.007550 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:28.007507    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:40:29.475203 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:29.475164    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:40:31.014859 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:31.014823    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:40:32.153926 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:32.153895    4276 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-9lts8"]
Oct 31 17:40:32.159487 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:32.159466    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-9lts8"
Oct 31 17:40:32.179575 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:32.179545    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-9lts8"]
Oct 31 17:40:32.200122 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:32.200092    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/79a9c2b2-8458-4a74-afde-ee00374beb5b-istio-data\") pod \"e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-9lts8\" (UID: \"79a9c2b2-8458-4a74-afde-ee00374beb5b\") " pod="openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-9lts8"
Oct 31 17:40:32.200309 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:32.200135    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/79a9c2b2-8458-4a74-afde-ee00374beb5b-workload-socket\") pod \"e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-9lts8\" (UID: \"79a9c2b2-8458-4a74-afde-ee00374beb5b\") " pod="openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-9lts8"
Oct 31 17:40:32.200309 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:32.200245    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/79a9c2b2-8458-4a74-afde-ee00374beb5b-credential-socket\") pod \"e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-9lts8\" (UID: \"79a9c2b2-8458-4a74-afde-ee00374beb5b\") " pod="openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-9lts8"
Oct 31 17:40:32.200309 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:32.200271    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/79a9c2b2-8458-4a74-afde-ee00374beb5b-istiod-ca-cert\") pod \"e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-9lts8\" (UID: \"79a9c2b2-8458-4a74-afde-ee00374beb5b\") " pod="openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-9lts8"
Oct 31 17:40:32.200309 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:32.200306    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/79a9c2b2-8458-4a74-afde-ee00374beb5b-istio-podinfo\") pod \"e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-9lts8\" (UID: \"79a9c2b2-8458-4a74-afde-ee00374beb5b\") " pod="openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-9lts8"
Oct 31 17:40:32.200578 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:32.200400    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/79a9c2b2-8458-4a74-afde-ee00374beb5b-istio-token\") pod \"e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-9lts8\" (UID: \"79a9c2b2-8458-4a74-afde-ee00374beb5b\") " pod="openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-9lts8"
Oct 31 17:40:32.200578 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:32.200423    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/79a9c2b2-8458-4a74-afde-ee00374beb5b-istio-envoy\") pod \"e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-9lts8\" (UID: \"79a9c2b2-8458-4a74-afde-ee00374beb5b\") " pod="openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-9lts8"
Oct 31 17:40:32.200578 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:32.200505    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5572l\" (UniqueName: \"kubernetes.io/projected/79a9c2b2-8458-4a74-afde-ee00374beb5b-kube-api-access-5572l\") pod \"e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-9lts8\" (UID: \"79a9c2b2-8458-4a74-afde-ee00374beb5b\") " pod="openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-9lts8"
Oct 31 17:40:32.200578 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:32.200571    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/79a9c2b2-8458-4a74-afde-ee00374beb5b-workload-certs\") pod \"e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-9lts8\" (UID: \"79a9c2b2-8458-4a74-afde-ee00374beb5b\") " pod="openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-9lts8"
Oct 31 17:40:32.301844 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:32.301811    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/79a9c2b2-8458-4a74-afde-ee00374beb5b-credential-socket\") pod \"e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-9lts8\" (UID: \"79a9c2b2-8458-4a74-afde-ee00374beb5b\") " pod="openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-9lts8"
Oct 31 17:40:32.301844 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:32.301843    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/79a9c2b2-8458-4a74-afde-ee00374beb5b-istiod-ca-cert\") pod \"e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-9lts8\" (UID: \"79a9c2b2-8458-4a74-afde-ee00374beb5b\") " pod="openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-9lts8"
Oct 31 17:40:32.302068 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:32.301867    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/79a9c2b2-8458-4a74-afde-ee00374beb5b-istio-podinfo\") pod \"e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-9lts8\" (UID: \"79a9c2b2-8458-4a74-afde-ee00374beb5b\") " pod="openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-9lts8"
Oct 31 17:40:32.302068 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:32.301895    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/79a9c2b2-8458-4a74-afde-ee00374beb5b-istio-token\") pod \"e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-9lts8\" (UID: \"79a9c2b2-8458-4a74-afde-ee00374beb5b\") " pod="openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-9lts8"
Oct 31 17:40:32.302068 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:32.301922    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/79a9c2b2-8458-4a74-afde-ee00374beb5b-istio-envoy\") pod \"e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-9lts8\" (UID: \"79a9c2b2-8458-4a74-afde-ee00374beb5b\") " pod="openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-9lts8"
Oct 31 17:40:32.302068 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:32.301946    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5572l\" (UniqueName: \"kubernetes.io/projected/79a9c2b2-8458-4a74-afde-ee00374beb5b-kube-api-access-5572l\") pod \"e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-9lts8\" (UID: \"79a9c2b2-8458-4a74-afde-ee00374beb5b\") " pod="openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-9lts8"
Oct 31 17:40:32.302068 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:32.301995    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/79a9c2b2-8458-4a74-afde-ee00374beb5b-workload-certs\") pod \"e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-9lts8\" (UID: \"79a9c2b2-8458-4a74-afde-ee00374beb5b\") " pod="openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-9lts8"
Oct 31 17:40:32.302326 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:32.302154    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/79a9c2b2-8458-4a74-afde-ee00374beb5b-istio-data\") pod \"e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-9lts8\" (UID: \"79a9c2b2-8458-4a74-afde-ee00374beb5b\") " pod="openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-9lts8"
Oct 31 17:40:32.302326 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:32.302205    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/79a9c2b2-8458-4a74-afde-ee00374beb5b-workload-socket\") pod \"e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-9lts8\" (UID: \"79a9c2b2-8458-4a74-afde-ee00374beb5b\") " pod="openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-9lts8"
Oct 31 17:40:32.302522 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:32.302497    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/79a9c2b2-8458-4a74-afde-ee00374beb5b-credential-socket\") pod \"e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-9lts8\" (UID: \"79a9c2b2-8458-4a74-afde-ee00374beb5b\") " pod="openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-9lts8"
Oct 31 17:40:32.302646 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:32.302623    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/79a9c2b2-8458-4a74-afde-ee00374beb5b-workload-socket\") pod \"e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-9lts8\" (UID: \"79a9c2b2-8458-4a74-afde-ee00374beb5b\") " pod="openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-9lts8"
Oct 31 17:40:32.302722 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:32.302647    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/79a9c2b2-8458-4a74-afde-ee00374beb5b-workload-certs\") pod \"e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-9lts8\" (UID: \"79a9c2b2-8458-4a74-afde-ee00374beb5b\") " pod="openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-9lts8"
Oct 31 17:40:32.302722 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:32.302694    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/79a9c2b2-8458-4a74-afde-ee00374beb5b-istio-data\") pod \"e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-9lts8\" (UID: \"79a9c2b2-8458-4a74-afde-ee00374beb5b\") " pod="openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-9lts8"
Oct 31 17:40:32.302836 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:32.302694    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/79a9c2b2-8458-4a74-afde-ee00374beb5b-istiod-ca-cert\") pod \"e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-9lts8\" (UID: \"79a9c2b2-8458-4a74-afde-ee00374beb5b\") " pod="openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-9lts8"
Oct 31 17:40:32.304295 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:32.304268    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/79a9c2b2-8458-4a74-afde-ee00374beb5b-istio-envoy\") pod \"e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-9lts8\" (UID: \"79a9c2b2-8458-4a74-afde-ee00374beb5b\") " pod="openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-9lts8"
Oct 31 17:40:32.304500 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:32.304481    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/79a9c2b2-8458-4a74-afde-ee00374beb5b-istio-podinfo\") pod \"e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-9lts8\" (UID: \"79a9c2b2-8458-4a74-afde-ee00374beb5b\") " pod="openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-9lts8"
Oct 31 17:40:32.321614 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:32.321593    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/79a9c2b2-8458-4a74-afde-ee00374beb5b-istio-token\") pod \"e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-9lts8\" (UID: \"79a9c2b2-8458-4a74-afde-ee00374beb5b\") " pod="openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-9lts8"
Oct 31 17:40:32.321696 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:32.321676    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5572l\" (UniqueName: \"kubernetes.io/projected/79a9c2b2-8458-4a74-afde-ee00374beb5b-kube-api-access-5572l\") pod \"e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-9lts8\" (UID: \"79a9c2b2-8458-4a74-afde-ee00374beb5b\") " pod="openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-9lts8"
Oct 31 17:40:32.447286 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:32.447206    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:40:32.497340 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:32.497305    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-9lts8"
Oct 31 17:40:32.877834 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:32.877808    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-9lts8"]
Oct 31 17:40:32.878730 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:40:32.878706    4276 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79a9c2b2_8458_4a74_afde_ee00374beb5b.slice/crio-2025f3ff521fd539691fccf5d10485bb5ef879f003b99cf51f6a111b6505f81e WatchSource:0}: Error finding container 2025f3ff521fd539691fccf5d10485bb5ef879f003b99cf51f6a111b6505f81e: Status 404 returned error can't find the container with id 2025f3ff521fd539691fccf5d10485bb5ef879f003b99cf51f6a111b6505f81e
Oct 31 17:40:32.881093 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:32.881062    4276 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 17:40:32.881165 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:32.881127    4276 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 17:40:32.881165 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:32.881157    4276 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 17:40:33.828525 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:33.828489    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-9lts8" event={"ID":"79a9c2b2-8458-4a74-afde-ee00374beb5b","Type":"ContainerStarted","Data":"086d73b8f16757218462e0350e934c0a704569110746210e0031445ea87e9d90"}
Oct 31 17:40:33.828913 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:33.828550    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-9lts8" event={"ID":"79a9c2b2-8458-4a74-afde-ee00374beb5b","Type":"ContainerStarted","Data":"2025f3ff521fd539691fccf5d10485bb5ef879f003b99cf51f6a111b6505f81e"}
Oct 31 17:40:33.887994 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:33.887940    4276 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-9lts8" podStartSLOduration=1.8879270259999998 podStartE2EDuration="1.887927026s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 17:40:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 17:40:33.885525833 +0000 UTC m=+796.683270761" watchObservedRunningTime="2026-10-31 17:40:33.887927026 +0000 UTC m=+796.685671954"
Oct 31 17:40:34.085438 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:34.085215    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:40:34.087220 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:34.087178    4276 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w2-5c446c/distinct-2-w2-5c446c-kserve-5d9cd79c47-p4j5l"]
Oct 31 17:40:34.092318 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:34.092294    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-5c446c/distinct-2-w2-5c446c-kserve-5d9cd79c47-p4j5l"
Oct 31 17:40:34.105673 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:34.105650    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-5c446c/distinct-2-w2-5c446c-kserve-5d9cd79c47-p4j5l"]
Oct 31 17:40:34.109185 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:34.109158    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-5c446c\"/\"distinct-2-w2-5c446c-kserve-self-signed-certs\""
Oct 31 17:40:34.128811 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:34.128786    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4c288908-92e3-4c45-8ba5-2f86c9f38b3d-tls-certs\") pod \"distinct-2-w2-5c446c-kserve-5d9cd79c47-p4j5l\" (UID: \"4c288908-92e3-4c45-8ba5-2f86c9f38b3d\") " pod="e2e-models-e2e-worker-w2-5c446c/distinct-2-w2-5c446c-kserve-5d9cd79c47-p4j5l"
Oct 31 17:40:34.128976 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:34.128951    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4c288908-92e3-4c45-8ba5-2f86c9f38b3d-dshm\") pod \"distinct-2-w2-5c446c-kserve-5d9cd79c47-p4j5l\" (UID: \"4c288908-92e3-4c45-8ba5-2f86c9f38b3d\") " pod="e2e-models-e2e-worker-w2-5c446c/distinct-2-w2-5c446c-kserve-5d9cd79c47-p4j5l"
Oct 31 17:40:34.129052 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:34.129027    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-phc42\" (UniqueName: \"kubernetes.io/projected/4c288908-92e3-4c45-8ba5-2f86c9f38b3d-kube-api-access-phc42\") pod \"distinct-2-w2-5c446c-kserve-5d9cd79c47-p4j5l\" (UID: \"4c288908-92e3-4c45-8ba5-2f86c9f38b3d\") " pod="e2e-models-e2e-worker-w2-5c446c/distinct-2-w2-5c446c-kserve-5d9cd79c47-p4j5l"
Oct 31 17:40:34.129164 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:34.129146    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4c288908-92e3-4c45-8ba5-2f86c9f38b3d-home\") pod \"distinct-2-w2-5c446c-kserve-5d9cd79c47-p4j5l\" (UID: \"4c288908-92e3-4c45-8ba5-2f86c9f38b3d\") " pod="e2e-models-e2e-worker-w2-5c446c/distinct-2-w2-5c446c-kserve-5d9cd79c47-p4j5l"
Oct 31 17:40:34.129233 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:34.129205    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4c288908-92e3-4c45-8ba5-2f86c9f38b3d-model-cache\") pod \"distinct-2-w2-5c446c-kserve-5d9cd79c47-p4j5l\" (UID: \"4c288908-92e3-4c45-8ba5-2f86c9f38b3d\") " pod="e2e-models-e2e-worker-w2-5c446c/distinct-2-w2-5c446c-kserve-5d9cd79c47-p4j5l"
Oct 31 17:40:34.129338 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:34.129320    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4c288908-92e3-4c45-8ba5-2f86c9f38b3d-tmp-dir\") pod \"distinct-2-w2-5c446c-kserve-5d9cd79c47-p4j5l\" (UID: \"4c288908-92e3-4c45-8ba5-2f86c9f38b3d\") " pod="e2e-models-e2e-worker-w2-5c446c/distinct-2-w2-5c446c-kserve-5d9cd79c47-p4j5l"
Oct 31 17:40:34.197968 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:34.197939    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:40:34.230368 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:34.230339    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4c288908-92e3-4c45-8ba5-2f86c9f38b3d-home\") pod \"distinct-2-w2-5c446c-kserve-5d9cd79c47-p4j5l\" (UID: \"4c288908-92e3-4c45-8ba5-2f86c9f38b3d\") " pod="e2e-models-e2e-worker-w2-5c446c/distinct-2-w2-5c446c-kserve-5d9cd79c47-p4j5l"
Oct 31 17:40:34.230553 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:34.230385    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4c288908-92e3-4c45-8ba5-2f86c9f38b3d-model-cache\") pod \"distinct-2-w2-5c446c-kserve-5d9cd79c47-p4j5l\" (UID: \"4c288908-92e3-4c45-8ba5-2f86c9f38b3d\") " pod="e2e-models-e2e-worker-w2-5c446c/distinct-2-w2-5c446c-kserve-5d9cd79c47-p4j5l"
Oct 31 17:40:34.230553 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:34.230434    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4c288908-92e3-4c45-8ba5-2f86c9f38b3d-tmp-dir\") pod \"distinct-2-w2-5c446c-kserve-5d9cd79c47-p4j5l\" (UID: \"4c288908-92e3-4c45-8ba5-2f86c9f38b3d\") " pod="e2e-models-e2e-worker-w2-5c446c/distinct-2-w2-5c446c-kserve-5d9cd79c47-p4j5l"
Oct 31 17:40:34.230553 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:34.230489    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4c288908-92e3-4c45-8ba5-2f86c9f38b3d-tls-certs\") pod \"distinct-2-w2-5c446c-kserve-5d9cd79c47-p4j5l\" (UID: \"4c288908-92e3-4c45-8ba5-2f86c9f38b3d\") " pod="e2e-models-e2e-worker-w2-5c446c/distinct-2-w2-5c446c-kserve-5d9cd79c47-p4j5l"
Oct 31 17:40:34.230724 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:34.230567    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4c288908-92e3-4c45-8ba5-2f86c9f38b3d-dshm\") pod \"distinct-2-w2-5c446c-kserve-5d9cd79c47-p4j5l\" (UID: \"4c288908-92e3-4c45-8ba5-2f86c9f38b3d\") " pod="e2e-models-e2e-worker-w2-5c446c/distinct-2-w2-5c446c-kserve-5d9cd79c47-p4j5l"
Oct 31 17:40:34.230724 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:34.230610    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-phc42\" (UniqueName: \"kubernetes.io/projected/4c288908-92e3-4c45-8ba5-2f86c9f38b3d-kube-api-access-phc42\") pod \"distinct-2-w2-5c446c-kserve-5d9cd79c47-p4j5l\" (UID: \"4c288908-92e3-4c45-8ba5-2f86c9f38b3d\") " pod="e2e-models-e2e-worker-w2-5c446c/distinct-2-w2-5c446c-kserve-5d9cd79c47-p4j5l"
Oct 31 17:40:34.230827 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:34.230812    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4c288908-92e3-4c45-8ba5-2f86c9f38b3d-home\") pod \"distinct-2-w2-5c446c-kserve-5d9cd79c47-p4j5l\" (UID: \"4c288908-92e3-4c45-8ba5-2f86c9f38b3d\") " pod="e2e-models-e2e-worker-w2-5c446c/distinct-2-w2-5c446c-kserve-5d9cd79c47-p4j5l"
Oct 31 17:40:34.230885 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:34.230843    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4c288908-92e3-4c45-8ba5-2f86c9f38b3d-model-cache\") pod \"distinct-2-w2-5c446c-kserve-5d9cd79c47-p4j5l\" (UID: \"4c288908-92e3-4c45-8ba5-2f86c9f38b3d\") " pod="e2e-models-e2e-worker-w2-5c446c/distinct-2-w2-5c446c-kserve-5d9cd79c47-p4j5l"
Oct 31 17:40:34.230954 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:34.230921    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4c288908-92e3-4c45-8ba5-2f86c9f38b3d-tmp-dir\") pod \"distinct-2-w2-5c446c-kserve-5d9cd79c47-p4j5l\" (UID: \"4c288908-92e3-4c45-8ba5-2f86c9f38b3d\") " pod="e2e-models-e2e-worker-w2-5c446c/distinct-2-w2-5c446c-kserve-5d9cd79c47-p4j5l"
Oct 31 17:40:34.232774 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:34.232748    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4c288908-92e3-4c45-8ba5-2f86c9f38b3d-dshm\") pod \"distinct-2-w2-5c446c-kserve-5d9cd79c47-p4j5l\" (UID: \"4c288908-92e3-4c45-8ba5-2f86c9f38b3d\") " pod="e2e-models-e2e-worker-w2-5c446c/distinct-2-w2-5c446c-kserve-5d9cd79c47-p4j5l"
Oct 31 17:40:34.233002 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:34.232986    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4c288908-92e3-4c45-8ba5-2f86c9f38b3d-tls-certs\") pod \"distinct-2-w2-5c446c-kserve-5d9cd79c47-p4j5l\" (UID: \"4c288908-92e3-4c45-8ba5-2f86c9f38b3d\") " pod="e2e-models-e2e-worker-w2-5c446c/distinct-2-w2-5c446c-kserve-5d9cd79c47-p4j5l"
Oct 31 17:40:34.270212 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:34.270141    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-phc42\" (UniqueName: \"kubernetes.io/projected/4c288908-92e3-4c45-8ba5-2f86c9f38b3d-kube-api-access-phc42\") pod \"distinct-2-w2-5c446c-kserve-5d9cd79c47-p4j5l\" (UID: \"4c288908-92e3-4c45-8ba5-2f86c9f38b3d\") " pod="e2e-models-e2e-worker-w2-5c446c/distinct-2-w2-5c446c-kserve-5d9cd79c47-p4j5l"
Oct 31 17:40:34.417518 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:34.416931    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-5c446c/distinct-2-w2-5c446c-kserve-5d9cd79c47-p4j5l"
Oct 31 17:40:34.499627 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:34.499598    4276 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-9lts8"
Oct 31 17:40:34.513611 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:34.513575    4276 patch_prober.go:29] interesting pod/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-9lts8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body=
Oct 31 17:40:34.513780 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:34.513668    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-9lts8" podUID="79a9c2b2-8458-4a74-afde-ee00374beb5b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused"
Oct 31 17:40:34.836204 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:34.836173    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-5c446c/distinct-2-w2-5c446c-kserve-5d9cd79c47-p4j5l"]
Oct 31 17:40:34.837786 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:40:34.837761    4276 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4c288908_92e3_4c45_8ba5_2f86c9f38b3d.slice/crio-55b50a582324c4fbc081dff75ede695e54805bbddf4985ce6fd31c2b7e0f2e82 WatchSource:0}: Error finding container 55b50a582324c4fbc081dff75ede695e54805bbddf4985ce6fd31c2b7e0f2e82: Status 404 returned error can't find the container with id 55b50a582324c4fbc081dff75ede695e54805bbddf4985ce6fd31c2b7e0f2e82
Oct 31 17:40:35.498722 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:35.498685    4276 patch_prober.go:29] interesting pod/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-9lts8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body=
Oct 31 17:40:35.498904 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:35.498736    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-9lts8" podUID="79a9c2b2-8458-4a74-afde-ee00374beb5b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused"
Oct 31 17:40:35.811311 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:35.811277    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:40:35.902462 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:35.894369    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-5c446c/distinct-2-w2-5c446c-kserve-5d9cd79c47-p4j5l" event={"ID":"4c288908-92e3-4c45-8ba5-2f86c9f38b3d","Type":"ContainerStarted","Data":"6274d0f6e3ab3ef86dc4be5543d1748a8d2c5cfbd4c6bb1c80547c0702dcdd5e"}
Oct 31 17:40:35.902462 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:35.894421    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-5c446c/distinct-2-w2-5c446c-kserve-5d9cd79c47-p4j5l" event={"ID":"4c288908-92e3-4c45-8ba5-2f86c9f38b3d","Type":"ContainerStarted","Data":"55b50a582324c4fbc081dff75ede695e54805bbddf4985ce6fd31c2b7e0f2e82"}
Oct 31 17:40:35.943468 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:35.943349    4276 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w2-5c446c/distinct-2-w2-5c446c-kserve-5d9cd79c47-p4j5l" podStartSLOduration=1.9433319089999999 podStartE2EDuration="1.943331909s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 17:40:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 17:40:35.94284155 +0000 UTC m=+798.740586478" watchObservedRunningTime="2026-10-31 17:40:35.943331909 +0000 UTC m=+798.741076839"
Oct 31 17:40:36.417904 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:36.417876    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-5c446c/distinct-2-w2-5c446c-kserve-5d9cd79c47-p4j5l"
Oct 31 17:40:36.498603 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:36.498572    4276 patch_prober.go:29] interesting pod/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-9lts8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body=
Oct 31 17:40:36.498782 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:36.498621    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-9lts8" podUID="79a9c2b2-8458-4a74-afde-ee00374beb5b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused"
Oct 31 17:40:37.511468 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:37.502141    4276 patch_prober.go:29] interesting pod/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-9lts8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body=
Oct 31 17:40:37.511468 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:37.502203    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-9lts8" podUID="79a9c2b2-8458-4a74-afde-ee00374beb5b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused"
Oct 31 17:40:37.574740 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:37.574702    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:40:38.376468 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:38.374529    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-5c446c/distinct-w2-5c446c-kserve-69d97d6968-gtvjv" podUID="2d1185cc-bac1-45ca-a250-8f44f8589162" containerName="main" probeResult="failure" output="Get \"https://10.133.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 17:40:38.504468 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:38.503852    4276 patch_prober.go:29] interesting pod/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-9lts8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body=
Oct 31 17:40:38.504468 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:38.503912    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-9lts8" podUID="79a9c2b2-8458-4a74-afde-ee00374beb5b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused"
Oct 31 17:40:38.539584 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:38.539537    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ee816e/premium-w0-ee816e-kserve-65c86cd6db-lnt9p" podUID="e536c902-52a9-4b36-a2ad-eb5500781a5e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.49:8000/ready\": context deadline exceeded"
Oct 31 17:40:38.577597 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:38.577545    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-d80080/distinct-w6-d80080-kserve-664576c8f5-6jc4d" podUID="b9c2f445-a191-4b00-802a-1b0ded068411" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 17:40:39.375468 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:39.374974    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-5c446c/distinct-w2-5c446c-kserve-69d97d6968-gtvjv" podUID="2d1185cc-bac1-45ca-a250-8f44f8589162" containerName="main" probeResult="failure" output="Get \"https://10.133.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 17:40:39.407498 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:39.404069    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-5c446c/distinct-w2-5c446c-kserve-69d97d6968-gtvjv"
Oct 31 17:40:39.407498 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:39.405300    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-0e59ca/premium-w1-0e59ca-kserve-58765f9746-fqwq7" podUID="8cf1e52e-70f8-42e2-bdf3-a315e17f10f5" containerName="main" probeResult="failure" output="Get \"https://10.133.0.48:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 17:40:39.698529 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:39.697519    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ee816e/premium-w0-ee816e-kserve-65c86cd6db-lnt9p" podUID="e536c902-52a9-4b36-a2ad-eb5500781a5e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.49:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 17:40:39.706476 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:39.702998    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-ee816e/premium-w0-ee816e-kserve-65c86cd6db-lnt9p"
Oct 31 17:40:39.709549 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:39.709521    4276 patch_prober.go:29] interesting pod/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-9lts8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body=
Oct 31 17:40:39.710681 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:39.710654    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-9lts8" podUID="79a9c2b2-8458-4a74-afde-ee00374beb5b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused"
Oct 31 17:40:39.812468 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:39.809048    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:40:40.293997 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:40.293966    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-ee816e/premium-w0-ee816e-kserve-65c86cd6db-lnt9p"
Oct 31 17:40:40.325970 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:40.324886    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-5c446c/distinct-w2-5c446c-kserve-69d97d6968-gtvjv"
Oct 31 17:40:40.334738 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:40.334693    4276 patch_prober.go:29] interesting pod/e2e-worker-w6-d80080-openshift-default-956cddf57-25kdb container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 17:40:40.335025 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:40.334980    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-25kdb" podUID="6ec6accb-4b2a-45a8-bab2-fe65f2f5df2d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 17:40:40.506468 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:40.499835    4276 patch_prober.go:29] interesting pod/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-9lts8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body=
Oct 31 17:40:40.506468 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:40.499915    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-9lts8" podUID="79a9c2b2-8458-4a74-afde-ee00374beb5b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused"
Oct 31 17:40:41.299339 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:41.299302    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:40:41.795959 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:41.795875    4276 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-9lts8"
Oct 31 17:40:41.796318 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:41.796299    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-9lts8"
Oct 31 17:40:41.797189 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:41.797169    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-9lts8"
Oct 31 17:40:42.237347 ip-10-0-129-127 kubenswrapper[4276]: E1031 17:40:42.237306    4276 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tg5qk" containerName="extractor"
Oct 31 17:40:42.767233 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:42.767191    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:40:44.362318 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:44.362286    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:40:44.418261 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:44.418227    4276 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w2-5c446c/distinct-2-w2-5c446c-kserve-5d9cd79c47-p4j5l"
Oct 31 17:40:44.433060 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:44.433027    4276 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w2-5c446c/distinct-2-w2-5c446c-kserve-5d9cd79c47-p4j5l"
Oct 31 17:40:44.509039 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:44.508970    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:40:45.128831 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:45.128802    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-5c446c/distinct-2-w2-5c446c-kserve-5d9cd79c47-p4j5l"
Oct 31 17:40:45.954389 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:45.954359    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:40:47.465412 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:47.465385    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:40:49.177967 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:49.177931    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:40:51.005022 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:51.004987    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:40:52.937915 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:52.937881    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:40:54.921400 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:54.921360    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:40:55.522652 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:55.522623    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:40:56.551061 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:56.551032    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:40:58.001693 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:58.001655    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:40:59.453933 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:40:59.453903    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:41:01.008437 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:01.008399    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:41:02.129550 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:02.129514    4276 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-m99sg"]
Oct 31 17:41:02.180468 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:02.169261    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-m99sg"
Oct 31 17:41:02.190627 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:02.190593    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-m99sg"]
Oct 31 17:41:02.550531 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:02.535890    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/56c4ff3e-b2f2-4dce-938d-48672451d1fe-istio-token\") pod \"e2e-worker-w0-ee816e-openshift-default-c4db85b5c-m99sg\" (UID: \"56c4ff3e-b2f2-4dce-938d-48672451d1fe\") " pod="openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-m99sg"
Oct 31 17:41:02.550531 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:02.535961    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/56c4ff3e-b2f2-4dce-938d-48672451d1fe-istio-data\") pod \"e2e-worker-w0-ee816e-openshift-default-c4db85b5c-m99sg\" (UID: \"56c4ff3e-b2f2-4dce-938d-48672451d1fe\") " pod="openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-m99sg"
Oct 31 17:41:02.550531 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:02.536004    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gk7gj\" (UniqueName: \"kubernetes.io/projected/56c4ff3e-b2f2-4dce-938d-48672451d1fe-kube-api-access-gk7gj\") pod \"e2e-worker-w0-ee816e-openshift-default-c4db85b5c-m99sg\" (UID: \"56c4ff3e-b2f2-4dce-938d-48672451d1fe\") " pod="openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-m99sg"
Oct 31 17:41:02.550531 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:02.536056    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/56c4ff3e-b2f2-4dce-938d-48672451d1fe-workload-certs\") pod \"e2e-worker-w0-ee816e-openshift-default-c4db85b5c-m99sg\" (UID: \"56c4ff3e-b2f2-4dce-938d-48672451d1fe\") " pod="openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-m99sg"
Oct 31 17:41:02.550531 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:02.536203    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/56c4ff3e-b2f2-4dce-938d-48672451d1fe-istio-podinfo\") pod \"e2e-worker-w0-ee816e-openshift-default-c4db85b5c-m99sg\" (UID: \"56c4ff3e-b2f2-4dce-938d-48672451d1fe\") " pod="openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-m99sg"
Oct 31 17:41:02.550531 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:02.536248    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/56c4ff3e-b2f2-4dce-938d-48672451d1fe-workload-socket\") pod \"e2e-worker-w0-ee816e-openshift-default-c4db85b5c-m99sg\" (UID: \"56c4ff3e-b2f2-4dce-938d-48672451d1fe\") " pod="openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-m99sg"
Oct 31 17:41:02.550531 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:02.536368    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/56c4ff3e-b2f2-4dce-938d-48672451d1fe-credential-socket\") pod \"e2e-worker-w0-ee816e-openshift-default-c4db85b5c-m99sg\" (UID: \"56c4ff3e-b2f2-4dce-938d-48672451d1fe\") " pod="openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-m99sg"
Oct 31 17:41:02.550531 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:02.536410    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/56c4ff3e-b2f2-4dce-938d-48672451d1fe-istio-envoy\") pod \"e2e-worker-w0-ee816e-openshift-default-c4db85b5c-m99sg\" (UID: \"56c4ff3e-b2f2-4dce-938d-48672451d1fe\") " pod="openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-m99sg"
Oct 31 17:41:02.550531 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:02.536436    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/56c4ff3e-b2f2-4dce-938d-48672451d1fe-istiod-ca-cert\") pod \"e2e-worker-w0-ee816e-openshift-default-c4db85b5c-m99sg\" (UID: \"56c4ff3e-b2f2-4dce-938d-48672451d1fe\") " pod="openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-m99sg"
Oct 31 17:41:02.639136 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:02.639095    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/56c4ff3e-b2f2-4dce-938d-48672451d1fe-credential-socket\") pod \"e2e-worker-w0-ee816e-openshift-default-c4db85b5c-m99sg\" (UID: \"56c4ff3e-b2f2-4dce-938d-48672451d1fe\") " pod="openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-m99sg"
Oct 31 17:41:02.663996 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:02.663966    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/56c4ff3e-b2f2-4dce-938d-48672451d1fe-istio-envoy\") pod \"e2e-worker-w0-ee816e-openshift-default-c4db85b5c-m99sg\" (UID: \"56c4ff3e-b2f2-4dce-938d-48672451d1fe\") " pod="openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-m99sg"
Oct 31 17:41:02.664705 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:02.643250    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/56c4ff3e-b2f2-4dce-938d-48672451d1fe-credential-socket\") pod \"e2e-worker-w0-ee816e-openshift-default-c4db85b5c-m99sg\" (UID: \"56c4ff3e-b2f2-4dce-938d-48672451d1fe\") " pod="openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-m99sg"
Oct 31 17:41:02.665613 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:02.664670    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/56c4ff3e-b2f2-4dce-938d-48672451d1fe-istiod-ca-cert\") pod \"e2e-worker-w0-ee816e-openshift-default-c4db85b5c-m99sg\" (UID: \"56c4ff3e-b2f2-4dce-938d-48672451d1fe\") " pod="openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-m99sg"
Oct 31 17:41:02.665809 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:02.665578    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/56c4ff3e-b2f2-4dce-938d-48672451d1fe-istiod-ca-cert\") pod \"e2e-worker-w0-ee816e-openshift-default-c4db85b5c-m99sg\" (UID: \"56c4ff3e-b2f2-4dce-938d-48672451d1fe\") " pod="openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-m99sg"
Oct 31 17:41:02.666251 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:02.665936    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/56c4ff3e-b2f2-4dce-938d-48672451d1fe-istio-token\") pod \"e2e-worker-w0-ee816e-openshift-default-c4db85b5c-m99sg\" (UID: \"56c4ff3e-b2f2-4dce-938d-48672451d1fe\") " pod="openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-m99sg"
Oct 31 17:41:02.692514 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:02.689127    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/56c4ff3e-b2f2-4dce-938d-48672451d1fe-istio-data\") pod \"e2e-worker-w0-ee816e-openshift-default-c4db85b5c-m99sg\" (UID: \"56c4ff3e-b2f2-4dce-938d-48672451d1fe\") " pod="openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-m99sg"
Oct 31 17:41:02.692514 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:02.689334    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/56c4ff3e-b2f2-4dce-938d-48672451d1fe-istio-envoy\") pod \"e2e-worker-w0-ee816e-openshift-default-c4db85b5c-m99sg\" (UID: \"56c4ff3e-b2f2-4dce-938d-48672451d1fe\") " pod="openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-m99sg"
Oct 31 17:41:02.692514 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:02.666429    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/56c4ff3e-b2f2-4dce-938d-48672451d1fe-istio-data\") pod \"e2e-worker-w0-ee816e-openshift-default-c4db85b5c-m99sg\" (UID: \"56c4ff3e-b2f2-4dce-938d-48672451d1fe\") " pod="openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-m99sg"
Oct 31 17:41:02.692514 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:02.691958    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gk7gj\" (UniqueName: \"kubernetes.io/projected/56c4ff3e-b2f2-4dce-938d-48672451d1fe-kube-api-access-gk7gj\") pod \"e2e-worker-w0-ee816e-openshift-default-c4db85b5c-m99sg\" (UID: \"56c4ff3e-b2f2-4dce-938d-48672451d1fe\") " pod="openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-m99sg"
Oct 31 17:41:02.692514 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:02.692030    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/56c4ff3e-b2f2-4dce-938d-48672451d1fe-workload-certs\") pod \"e2e-worker-w0-ee816e-openshift-default-c4db85b5c-m99sg\" (UID: \"56c4ff3e-b2f2-4dce-938d-48672451d1fe\") " pod="openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-m99sg"
Oct 31 17:41:02.692514 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:02.692192    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/56c4ff3e-b2f2-4dce-938d-48672451d1fe-istio-podinfo\") pod \"e2e-worker-w0-ee816e-openshift-default-c4db85b5c-m99sg\" (UID: \"56c4ff3e-b2f2-4dce-938d-48672451d1fe\") " pod="openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-m99sg"
Oct 31 17:41:02.692514 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:02.692231    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/56c4ff3e-b2f2-4dce-938d-48672451d1fe-workload-socket\") pod \"e2e-worker-w0-ee816e-openshift-default-c4db85b5c-m99sg\" (UID: \"56c4ff3e-b2f2-4dce-938d-48672451d1fe\") " pod="openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-m99sg"
Oct 31 17:41:02.693030 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:02.692657    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/56c4ff3e-b2f2-4dce-938d-48672451d1fe-workload-socket\") pod \"e2e-worker-w0-ee816e-openshift-default-c4db85b5c-m99sg\" (UID: \"56c4ff3e-b2f2-4dce-938d-48672451d1fe\") " pod="openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-m99sg"
Oct 31 17:41:02.701485 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:02.693569    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/56c4ff3e-b2f2-4dce-938d-48672451d1fe-workload-certs\") pod \"e2e-worker-w0-ee816e-openshift-default-c4db85b5c-m99sg\" (UID: \"56c4ff3e-b2f2-4dce-938d-48672451d1fe\") " pod="openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-m99sg"
Oct 31 17:41:02.705480 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:02.702974    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/56c4ff3e-b2f2-4dce-938d-48672451d1fe-istio-token\") pod \"e2e-worker-w0-ee816e-openshift-default-c4db85b5c-m99sg\" (UID: \"56c4ff3e-b2f2-4dce-938d-48672451d1fe\") " pod="openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-m99sg"
Oct 31 17:41:02.738479 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:02.734778    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/56c4ff3e-b2f2-4dce-938d-48672451d1fe-istio-podinfo\") pod \"e2e-worker-w0-ee816e-openshift-default-c4db85b5c-m99sg\" (UID: \"56c4ff3e-b2f2-4dce-938d-48672451d1fe\") " pod="openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-m99sg"
Oct 31 17:41:02.752581 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:02.750523    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gk7gj\" (UniqueName: \"kubernetes.io/projected/56c4ff3e-b2f2-4dce-938d-48672451d1fe-kube-api-access-gk7gj\") pod \"e2e-worker-w0-ee816e-openshift-default-c4db85b5c-m99sg\" (UID: \"56c4ff3e-b2f2-4dce-938d-48672451d1fe\") " pod="openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-m99sg"
Oct 31 17:41:03.005849 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:03.005758    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-m99sg"
Oct 31 17:41:03.086680 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:03.086643    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:41:03.125191 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:03.124678    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-5c446c/distinct-2-w2-5c446c-kserve-5d9cd79c47-p4j5l" podUID="4c288908-92e3-4c45-8ba5-2f86c9f38b3d" containerName="main" probeResult="failure" output="Get \"https://10.133.0.65:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 17:41:03.334466 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:03.333527    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-5c446c/distinct-w2-5c446c-kserve-69d97d6968-gtvjv" podUID="2d1185cc-bac1-45ca-a250-8f44f8589162" containerName="main" probeResult="failure" output="Get \"https://10.133.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 17:41:03.595543 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:03.587531    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-d80080/distinct-w6-d80080-kserve-664576c8f5-6jc4d" podUID="b9c2f445-a191-4b00-802a-1b0ded068411" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 17:41:04.071124 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:04.057892    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-m99sg"]
Oct 31 17:41:04.213488 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:04.157820    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-5c446c/distinct-2-w2-5c446c-kserve-5d9cd79c47-p4j5l" podUID="4c288908-92e3-4c45-8ba5-2f86c9f38b3d" containerName="main" probeResult="failure" output="Get \"https://10.133.0.65:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 17:41:04.213488 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:04.158328    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-5c446c/distinct-2-w2-5c446c-kserve-5d9cd79c47-p4j5l"
Oct 31 17:41:04.213488 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:04.201961    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:41:04.285727 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:04.285678    4276 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 17:41:04.289569 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:04.289532    4276 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 17:41:04.289785 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:04.289764    4276 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 17:41:04.359306 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:04.321782    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-5c446c/distinct-2-w2-5c446c-kserve-5d9cd79c47-p4j5l"
Oct 31 17:41:04.522083 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:04.522043    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-m99sg" event={"ID":"56c4ff3e-b2f2-4dce-938d-48672451d1fe","Type":"ContainerStarted","Data":"66c58ec7b7bdcbc58d60926d1895953f0a02d105f87cac6d376071f1b2490915"}
Oct 31 17:41:05.529037 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:05.528998    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-m99sg" event={"ID":"56c4ff3e-b2f2-4dce-938d-48672451d1fe","Type":"ContainerStarted","Data":"4f62c3ff869ee3e1962a35435aba4b5f419648311c604b0634911e8968b4c93f"}
Oct 31 17:41:05.599549 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:05.599494    4276 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-m99sg" podStartSLOduration=3.599481024 podStartE2EDuration="3.599481024s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 17:41:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 17:41:05.597863181 +0000 UTC m=+828.395608110" watchObservedRunningTime="2026-10-31 17:41:05.599481024 +0000 UTC m=+828.397225951"
Oct 31 17:41:06.008149 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:06.008111    4276 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-m99sg"
Oct 31 17:41:06.009214 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:06.009190    4276 patch_prober.go:29] interesting pod/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-m99sg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body=
Oct 31 17:41:06.009335 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:06.009239    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-m99sg" podUID="56c4ff3e-b2f2-4dce-938d-48672451d1fe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused"
Oct 31 17:41:06.316966 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:06.316933    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:41:07.007965 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:07.007933    4276 patch_prober.go:29] interesting pod/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-m99sg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body=
Oct 31 17:41:07.008353 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:07.008013    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-m99sg" podUID="56c4ff3e-b2f2-4dce-938d-48672451d1fe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused"
Oct 31 17:41:07.293629 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:07.293552    4276 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-1c0bfd/distinct-2-w4-1c0bfd-kserve-847bcf6cf4-lt8ls"]
Oct 31 17:41:07.297820 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:07.297792    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-1c0bfd/distinct-2-w4-1c0bfd-kserve-847bcf6cf4-lt8ls"
Oct 31 17:41:07.314747 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:07.314723    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-1c0bfd/distinct-2-w4-1c0bfd-kserve-847bcf6cf4-lt8ls"]
Oct 31 17:41:07.325524 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:07.325492    4276 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-1c0bfd\"/\"kube-root-ca.crt\""
Oct 31 17:41:07.325657 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:07.325612    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-1c0bfd\"/\"default-dockercfg-x4sxr\""
Oct 31 17:41:07.325720 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:07.325687    4276 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-1c0bfd\"/\"openshift-service-ca.crt\""
Oct 31 17:41:07.328088 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:07.328056    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-1c0bfd\"/\"distinct-2-w4-1c0bfd-kserve-self-signed-certs\""
Oct 31 17:41:07.377688 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:07.377628    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:41:07.444475 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:07.444417    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f8a45b7c-e5ae-4321-ad81-fd6353b2495d-tmp-dir\") pod \"distinct-2-w4-1c0bfd-kserve-847bcf6cf4-lt8ls\" (UID: \"f8a45b7c-e5ae-4321-ad81-fd6353b2495d\") " pod="e2e-models-e2e-worker-w4-1c0bfd/distinct-2-w4-1c0bfd-kserve-847bcf6cf4-lt8ls"
Oct 31 17:41:07.444662 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:07.444536    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f8a45b7c-e5ae-4321-ad81-fd6353b2495d-dshm\") pod \"distinct-2-w4-1c0bfd-kserve-847bcf6cf4-lt8ls\" (UID: \"f8a45b7c-e5ae-4321-ad81-fd6353b2495d\") " pod="e2e-models-e2e-worker-w4-1c0bfd/distinct-2-w4-1c0bfd-kserve-847bcf6cf4-lt8ls"
Oct 31 17:41:07.444662 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:07.444566    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9l4fz\" (UniqueName: \"kubernetes.io/projected/f8a45b7c-e5ae-4321-ad81-fd6353b2495d-kube-api-access-9l4fz\") pod \"distinct-2-w4-1c0bfd-kserve-847bcf6cf4-lt8ls\" (UID: \"f8a45b7c-e5ae-4321-ad81-fd6353b2495d\") " pod="e2e-models-e2e-worker-w4-1c0bfd/distinct-2-w4-1c0bfd-kserve-847bcf6cf4-lt8ls"
Oct 31 17:41:07.444662 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:07.444641    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f8a45b7c-e5ae-4321-ad81-fd6353b2495d-tls-certs\") pod \"distinct-2-w4-1c0bfd-kserve-847bcf6cf4-lt8ls\" (UID: \"f8a45b7c-e5ae-4321-ad81-fd6353b2495d\") " pod="e2e-models-e2e-worker-w4-1c0bfd/distinct-2-w4-1c0bfd-kserve-847bcf6cf4-lt8ls"
Oct 31 17:41:07.444816 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:07.444795    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f8a45b7c-e5ae-4321-ad81-fd6353b2495d-home\") pod \"distinct-2-w4-1c0bfd-kserve-847bcf6cf4-lt8ls\" (UID: \"f8a45b7c-e5ae-4321-ad81-fd6353b2495d\") " pod="e2e-models-e2e-worker-w4-1c0bfd/distinct-2-w4-1c0bfd-kserve-847bcf6cf4-lt8ls"
Oct 31 17:41:07.444852 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:07.444824    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f8a45b7c-e5ae-4321-ad81-fd6353b2495d-model-cache\") pod \"distinct-2-w4-1c0bfd-kserve-847bcf6cf4-lt8ls\" (UID: \"f8a45b7c-e5ae-4321-ad81-fd6353b2495d\") " pod="e2e-models-e2e-worker-w4-1c0bfd/distinct-2-w4-1c0bfd-kserve-847bcf6cf4-lt8ls"
Oct 31 17:41:07.553392 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:07.553353    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f8a45b7c-e5ae-4321-ad81-fd6353b2495d-tls-certs\") pod \"distinct-2-w4-1c0bfd-kserve-847bcf6cf4-lt8ls\" (UID: \"f8a45b7c-e5ae-4321-ad81-fd6353b2495d\") " pod="e2e-models-e2e-worker-w4-1c0bfd/distinct-2-w4-1c0bfd-kserve-847bcf6cf4-lt8ls"
Oct 31 17:41:07.575473 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:07.571387    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f8a45b7c-e5ae-4321-ad81-fd6353b2495d-home\") pod \"distinct-2-w4-1c0bfd-kserve-847bcf6cf4-lt8ls\" (UID: \"f8a45b7c-e5ae-4321-ad81-fd6353b2495d\") " pod="e2e-models-e2e-worker-w4-1c0bfd/distinct-2-w4-1c0bfd-kserve-847bcf6cf4-lt8ls"
Oct 31 17:41:07.575473 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:07.571815    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f8a45b7c-e5ae-4321-ad81-fd6353b2495d-home\") pod \"distinct-2-w4-1c0bfd-kserve-847bcf6cf4-lt8ls\" (UID: \"f8a45b7c-e5ae-4321-ad81-fd6353b2495d\") " pod="e2e-models-e2e-worker-w4-1c0bfd/distinct-2-w4-1c0bfd-kserve-847bcf6cf4-lt8ls"
Oct 31 17:41:07.575473 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:07.571889    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f8a45b7c-e5ae-4321-ad81-fd6353b2495d-model-cache\") pod \"distinct-2-w4-1c0bfd-kserve-847bcf6cf4-lt8ls\" (UID: \"f8a45b7c-e5ae-4321-ad81-fd6353b2495d\") " pod="e2e-models-e2e-worker-w4-1c0bfd/distinct-2-w4-1c0bfd-kserve-847bcf6cf4-lt8ls"
Oct 31 17:41:07.575473 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:07.572075    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f8a45b7c-e5ae-4321-ad81-fd6353b2495d-tmp-dir\") pod \"distinct-2-w4-1c0bfd-kserve-847bcf6cf4-lt8ls\" (UID: \"f8a45b7c-e5ae-4321-ad81-fd6353b2495d\") " pod="e2e-models-e2e-worker-w4-1c0bfd/distinct-2-w4-1c0bfd-kserve-847bcf6cf4-lt8ls"
Oct 31 17:41:07.575473 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:07.572133    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f8a45b7c-e5ae-4321-ad81-fd6353b2495d-dshm\") pod \"distinct-2-w4-1c0bfd-kserve-847bcf6cf4-lt8ls\" (UID: \"f8a45b7c-e5ae-4321-ad81-fd6353b2495d\") " pod="e2e-models-e2e-worker-w4-1c0bfd/distinct-2-w4-1c0bfd-kserve-847bcf6cf4-lt8ls"
Oct 31 17:41:07.575473 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:07.572160    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9l4fz\" (UniqueName: \"kubernetes.io/projected/f8a45b7c-e5ae-4321-ad81-fd6353b2495d-kube-api-access-9l4fz\") pod \"distinct-2-w4-1c0bfd-kserve-847bcf6cf4-lt8ls\" (UID: \"f8a45b7c-e5ae-4321-ad81-fd6353b2495d\") " pod="e2e-models-e2e-worker-w4-1c0bfd/distinct-2-w4-1c0bfd-kserve-847bcf6cf4-lt8ls"
Oct 31 17:41:07.575473 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:07.572814    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f8a45b7c-e5ae-4321-ad81-fd6353b2495d-model-cache\") pod \"distinct-2-w4-1c0bfd-kserve-847bcf6cf4-lt8ls\" (UID: \"f8a45b7c-e5ae-4321-ad81-fd6353b2495d\") " pod="e2e-models-e2e-worker-w4-1c0bfd/distinct-2-w4-1c0bfd-kserve-847bcf6cf4-lt8ls"
Oct 31 17:41:07.575473 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:07.573060    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f8a45b7c-e5ae-4321-ad81-fd6353b2495d-tmp-dir\") pod \"distinct-2-w4-1c0bfd-kserve-847bcf6cf4-lt8ls\" (UID: \"f8a45b7c-e5ae-4321-ad81-fd6353b2495d\") " pod="e2e-models-e2e-worker-w4-1c0bfd/distinct-2-w4-1c0bfd-kserve-847bcf6cf4-lt8ls"
Oct 31 17:41:07.610745 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:07.610713    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f8a45b7c-e5ae-4321-ad81-fd6353b2495d-tls-certs\") pod \"distinct-2-w4-1c0bfd-kserve-847bcf6cf4-lt8ls\" (UID: \"f8a45b7c-e5ae-4321-ad81-fd6353b2495d\") " pod="e2e-models-e2e-worker-w4-1c0bfd/distinct-2-w4-1c0bfd-kserve-847bcf6cf4-lt8ls"
Oct 31 17:41:07.611265 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:07.611240    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f8a45b7c-e5ae-4321-ad81-fd6353b2495d-dshm\") pod \"distinct-2-w4-1c0bfd-kserve-847bcf6cf4-lt8ls\" (UID: \"f8a45b7c-e5ae-4321-ad81-fd6353b2495d\") " pod="e2e-models-e2e-worker-w4-1c0bfd/distinct-2-w4-1c0bfd-kserve-847bcf6cf4-lt8ls"
Oct 31 17:41:07.656496 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:07.649525    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9l4fz\" (UniqueName: \"kubernetes.io/projected/f8a45b7c-e5ae-4321-ad81-fd6353b2495d-kube-api-access-9l4fz\") pod \"distinct-2-w4-1c0bfd-kserve-847bcf6cf4-lt8ls\" (UID: \"f8a45b7c-e5ae-4321-ad81-fd6353b2495d\") " pod="e2e-models-e2e-worker-w4-1c0bfd/distinct-2-w4-1c0bfd-kserve-847bcf6cf4-lt8ls"
Oct 31 17:41:07.912729 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:07.912652    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-1c0bfd/distinct-2-w4-1c0bfd-kserve-847bcf6cf4-lt8ls"
Oct 31 17:41:08.008859 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:08.008825    4276 patch_prober.go:29] interesting pod/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-m99sg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body=
Oct 31 17:41:08.009182 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:08.008883    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-m99sg" podUID="56c4ff3e-b2f2-4dce-938d-48672451d1fe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused"
Oct 31 17:41:08.238773 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:08.238742    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-1c0bfd/distinct-2-w4-1c0bfd-kserve-847bcf6cf4-lt8ls"]
Oct 31 17:41:08.239144 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:41:08.239119    4276 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf8a45b7c_e5ae_4321_ad81_fd6353b2495d.slice/crio-fe1cf9d5110f8b3acca81bf656e42f5b6911df2bf00432993b110404c38d70b4 WatchSource:0}: Error finding container fe1cf9d5110f8b3acca81bf656e42f5b6911df2bf00432993b110404c38d70b4: Status 404 returned error can't find the container with id fe1cf9d5110f8b3acca81bf656e42f5b6911df2bf00432993b110404c38d70b4
Oct 31 17:41:08.597766 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:08.597727    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-1c0bfd/distinct-2-w4-1c0bfd-kserve-847bcf6cf4-lt8ls" event={"ID":"f8a45b7c-e5ae-4321-ad81-fd6353b2495d","Type":"ContainerStarted","Data":"14f49c1216072040ea6045823346d1158f41c921240b17b9e18bf79d9258fada"}
Oct 31 17:41:08.597958 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:08.597775    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-1c0bfd/distinct-2-w4-1c0bfd-kserve-847bcf6cf4-lt8ls" event={"ID":"f8a45b7c-e5ae-4321-ad81-fd6353b2495d","Type":"ContainerStarted","Data":"fe1cf9d5110f8b3acca81bf656e42f5b6911df2bf00432993b110404c38d70b4"}
Oct 31 17:41:08.641635 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:08.641588    4276 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-1c0bfd/distinct-2-w4-1c0bfd-kserve-847bcf6cf4-lt8ls" podStartSLOduration=1.641574967 podStartE2EDuration="1.641574967s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 17:41:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 17:41:08.640176712 +0000 UTC m=+831.437921641" watchObservedRunningTime="2026-10-31 17:41:08.641574967 +0000 UTC m=+831.439319912"
Oct 31 17:41:08.914688 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:08.914598    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-1c0bfd/distinct-2-w4-1c0bfd-kserve-847bcf6cf4-lt8ls"
Oct 31 17:41:09.008151 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:09.008116    4276 patch_prober.go:29] interesting pod/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-m99sg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body=
Oct 31 17:41:09.008325 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:09.008167    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-m99sg" podUID="56c4ff3e-b2f2-4dce-938d-48672451d1fe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused"
Oct 31 17:41:09.183319 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:09.183217    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:41:10.008665 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:10.008629    4276 patch_prober.go:29] interesting pod/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-m99sg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body=
Oct 31 17:41:10.008830 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:10.008695    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-m99sg" podUID="56c4ff3e-b2f2-4dce-938d-48672451d1fe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused"
Oct 31 17:41:11.007815 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:11.007781    4276 patch_prober.go:29] interesting pod/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-m99sg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body=
Oct 31 17:41:11.008189 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:11.007832    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-m99sg" podUID="56c4ff3e-b2f2-4dce-938d-48672451d1fe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused"
Oct 31 17:41:11.181407 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:11.181378    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:41:12.012087 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:12.012060    4276 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-m99sg"
Oct 31 17:41:12.012564 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:12.012365    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-m99sg"
Oct 31 17:41:12.013190 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:12.013174    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-m99sg"
Oct 31 17:41:12.243696 ip-10-0-129-127 kubenswrapper[4276]: E1031 17:41:12.243654    4276 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tg5qk" containerName="extractor"
Oct 31 17:41:12.942004 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:12.941971    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:41:14.549726 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:14.549691    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:41:16.053246 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:16.053210    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:41:16.645060 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:16.644980    4276 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-zsl5x"]
Oct 31 17:41:16.649566 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:16.649537    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-zsl5x"
Oct 31 17:41:16.671213 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:16.671187    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-zsl5x"]
Oct 31 17:41:16.693941 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:16.693906    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b2b7f15e-756a-40e6-90af-961551719a7d-istio-podinfo\") pod \"e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-zsl5x\" (UID: \"b2b7f15e-756a-40e6-90af-961551719a7d\") " pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-zsl5x"
Oct 31 17:41:16.694090 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:16.693947    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wxzcv\" (UniqueName: \"kubernetes.io/projected/b2b7f15e-756a-40e6-90af-961551719a7d-kube-api-access-wxzcv\") pod \"e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-zsl5x\" (UID: \"b2b7f15e-756a-40e6-90af-961551719a7d\") " pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-zsl5x"
Oct 31 17:41:16.694157 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:16.694111    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b2b7f15e-756a-40e6-90af-961551719a7d-istio-token\") pod \"e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-zsl5x\" (UID: \"b2b7f15e-756a-40e6-90af-961551719a7d\") " pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-zsl5x"
Oct 31 17:41:16.694240 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:16.694223    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b2b7f15e-756a-40e6-90af-961551719a7d-workload-certs\") pod \"e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-zsl5x\" (UID: \"b2b7f15e-756a-40e6-90af-961551719a7d\") " pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-zsl5x"
Oct 31 17:41:16.694297 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:16.694253    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b2b7f15e-756a-40e6-90af-961551719a7d-istio-data\") pod \"e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-zsl5x\" (UID: \"b2b7f15e-756a-40e6-90af-961551719a7d\") " pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-zsl5x"
Oct 31 17:41:16.694297 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:16.694287    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2b7f15e-756a-40e6-90af-961551719a7d-workload-socket\") pod \"e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-zsl5x\" (UID: \"b2b7f15e-756a-40e6-90af-961551719a7d\") " pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-zsl5x"
Oct 31 17:41:16.694472 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:16.694437    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b2b7f15e-756a-40e6-90af-961551719a7d-istiod-ca-cert\") pod \"e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-zsl5x\" (UID: \"b2b7f15e-756a-40e6-90af-961551719a7d\") " pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-zsl5x"
Oct 31 17:41:16.694631 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:16.694612    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b2b7f15e-756a-40e6-90af-961551719a7d-istio-envoy\") pod \"e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-zsl5x\" (UID: \"b2b7f15e-756a-40e6-90af-961551719a7d\") " pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-zsl5x"
Oct 31 17:41:16.694682 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:16.694650    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2b7f15e-756a-40e6-90af-961551719a7d-credential-socket\") pod \"e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-zsl5x\" (UID: \"b2b7f15e-756a-40e6-90af-961551719a7d\") " pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-zsl5x"
Oct 31 17:41:16.795361 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:16.795330    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b2b7f15e-756a-40e6-90af-961551719a7d-workload-certs\") pod \"e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-zsl5x\" (UID: \"b2b7f15e-756a-40e6-90af-961551719a7d\") " pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-zsl5x"
Oct 31 17:41:16.795361 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:16.795360    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b2b7f15e-756a-40e6-90af-961551719a7d-istio-data\") pod \"e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-zsl5x\" (UID: \"b2b7f15e-756a-40e6-90af-961551719a7d\") " pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-zsl5x"
Oct 31 17:41:16.795588 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:16.795379    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2b7f15e-756a-40e6-90af-961551719a7d-workload-socket\") pod \"e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-zsl5x\" (UID: \"b2b7f15e-756a-40e6-90af-961551719a7d\") " pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-zsl5x"
Oct 31 17:41:16.795588 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:16.795481    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b2b7f15e-756a-40e6-90af-961551719a7d-istiod-ca-cert\") pod \"e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-zsl5x\" (UID: \"b2b7f15e-756a-40e6-90af-961551719a7d\") " pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-zsl5x"
Oct 31 17:41:16.795588 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:16.795581    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b2b7f15e-756a-40e6-90af-961551719a7d-istio-envoy\") pod \"e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-zsl5x\" (UID: \"b2b7f15e-756a-40e6-90af-961551719a7d\") " pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-zsl5x"
Oct 31 17:41:16.795745 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:16.795602    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2b7f15e-756a-40e6-90af-961551719a7d-credential-socket\") pod \"e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-zsl5x\" (UID: \"b2b7f15e-756a-40e6-90af-961551719a7d\") " pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-zsl5x"
Oct 31 17:41:16.795745 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:16.795658    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b2b7f15e-756a-40e6-90af-961551719a7d-istio-podinfo\") pod \"e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-zsl5x\" (UID: \"b2b7f15e-756a-40e6-90af-961551719a7d\") " pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-zsl5x"
Oct 31 17:41:16.795745 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:16.795682    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wxzcv\" (UniqueName: \"kubernetes.io/projected/b2b7f15e-756a-40e6-90af-961551719a7d-kube-api-access-wxzcv\") pod \"e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-zsl5x\" (UID: \"b2b7f15e-756a-40e6-90af-961551719a7d\") " pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-zsl5x"
Oct 31 17:41:16.795902 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:16.795747    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b2b7f15e-756a-40e6-90af-961551719a7d-istio-token\") pod \"e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-zsl5x\" (UID: \"b2b7f15e-756a-40e6-90af-961551719a7d\") " pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-zsl5x"
Oct 31 17:41:16.795902 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:16.795755    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b2b7f15e-756a-40e6-90af-961551719a7d-workload-certs\") pod \"e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-zsl5x\" (UID: \"b2b7f15e-756a-40e6-90af-961551719a7d\") " pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-zsl5x"
Oct 31 17:41:16.796008 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:16.795935    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b2b7f15e-756a-40e6-90af-961551719a7d-istio-data\") pod \"e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-zsl5x\" (UID: \"b2b7f15e-756a-40e6-90af-961551719a7d\") " pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-zsl5x"
Oct 31 17:41:16.796068 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:16.796007    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2b7f15e-756a-40e6-90af-961551719a7d-credential-socket\") pod \"e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-zsl5x\" (UID: \"b2b7f15e-756a-40e6-90af-961551719a7d\") " pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-zsl5x"
Oct 31 17:41:16.796068 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:16.796043    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2b7f15e-756a-40e6-90af-961551719a7d-workload-socket\") pod \"e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-zsl5x\" (UID: \"b2b7f15e-756a-40e6-90af-961551719a7d\") " pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-zsl5x"
Oct 31 17:41:16.796235 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:16.796212    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b2b7f15e-756a-40e6-90af-961551719a7d-istiod-ca-cert\") pod \"e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-zsl5x\" (UID: \"b2b7f15e-756a-40e6-90af-961551719a7d\") " pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-zsl5x"
Oct 31 17:41:16.797811 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:16.797784    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b2b7f15e-756a-40e6-90af-961551719a7d-istio-envoy\") pod \"e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-zsl5x\" (UID: \"b2b7f15e-756a-40e6-90af-961551719a7d\") " pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-zsl5x"
Oct 31 17:41:16.798103 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:16.798084    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b2b7f15e-756a-40e6-90af-961551719a7d-istio-podinfo\") pod \"e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-zsl5x\" (UID: \"b2b7f15e-756a-40e6-90af-961551719a7d\") " pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-zsl5x"
Oct 31 17:41:16.815750 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:16.815728    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b2b7f15e-756a-40e6-90af-961551719a7d-istio-token\") pod \"e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-zsl5x\" (UID: \"b2b7f15e-756a-40e6-90af-961551719a7d\") " pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-zsl5x"
Oct 31 17:41:16.815896 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:16.815875    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wxzcv\" (UniqueName: \"kubernetes.io/projected/b2b7f15e-756a-40e6-90af-961551719a7d-kube-api-access-wxzcv\") pod \"e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-zsl5x\" (UID: \"b2b7f15e-756a-40e6-90af-961551719a7d\") " pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-zsl5x"
Oct 31 17:41:16.893230 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:16.893200    4276 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-g646p"]
Oct 31 17:41:16.898413 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:16.898354    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-g646p"
Oct 31 17:41:16.930335 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:16.930306    4276 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-hgtdx"]
Oct 31 17:41:16.940876 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:16.940822    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-g646p"]
Oct 31 17:41:16.941255 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:16.941236    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-hgtdx"
Oct 31 17:41:16.947404 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:16.947383    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-hgtdx"]
Oct 31 17:41:16.974404 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:16.974380    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-zsl5x"
Oct 31 17:41:16.998378 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:16.998352    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a59e3d47-2d57-4bf7-9a6c-5584d2d58d63-istio-token\") pod \"e2e-worker-w6-d80080-openshift-default-956cddf57-hgtdx\" (UID: \"a59e3d47-2d57-4bf7-9a6c-5584d2d58d63\") " pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-hgtdx"
Oct 31 17:41:16.998575 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:16.998384    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a59e3d47-2d57-4bf7-9a6c-5584d2d58d63-istio-podinfo\") pod \"e2e-worker-w6-d80080-openshift-default-956cddf57-hgtdx\" (UID: \"a59e3d47-2d57-4bf7-9a6c-5584d2d58d63\") " pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-hgtdx"
Oct 31 17:41:16.998575 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:16.998420    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a59e3d47-2d57-4bf7-9a6c-5584d2d58d63-workload-certs\") pod \"e2e-worker-w6-d80080-openshift-default-956cddf57-hgtdx\" (UID: \"a59e3d47-2d57-4bf7-9a6c-5584d2d58d63\") " pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-hgtdx"
Oct 31 17:41:16.998704 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:16.998584    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/14900971-460e-4126-9089-c4b110d539ab-istiod-ca-cert\") pod \"e2e-worker-w6-d80080-openshift-default-956cddf57-g646p\" (UID: \"14900971-460e-4126-9089-c4b110d539ab\") " pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-g646p"
Oct 31 17:41:16.998704 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:16.998604    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/14900971-460e-4126-9089-c4b110d539ab-istio-data\") pod \"e2e-worker-w6-d80080-openshift-default-956cddf57-g646p\" (UID: \"14900971-460e-4126-9089-c4b110d539ab\") " pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-g646p"
Oct 31 17:41:16.998704 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:16.998680    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a59e3d47-2d57-4bf7-9a6c-5584d2d58d63-istio-envoy\") pod \"e2e-worker-w6-d80080-openshift-default-956cddf57-hgtdx\" (UID: \"a59e3d47-2d57-4bf7-9a6c-5584d2d58d63\") " pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-hgtdx"
Oct 31 17:41:16.998858 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:16.998703    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a59e3d47-2d57-4bf7-9a6c-5584d2d58d63-credential-socket\") pod \"e2e-worker-w6-d80080-openshift-default-956cddf57-hgtdx\" (UID: \"a59e3d47-2d57-4bf7-9a6c-5584d2d58d63\") " pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-hgtdx"
Oct 31 17:41:16.998858 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:16.998776    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a59e3d47-2d57-4bf7-9a6c-5584d2d58d63-istiod-ca-cert\") pod \"e2e-worker-w6-d80080-openshift-default-956cddf57-hgtdx\" (UID: \"a59e3d47-2d57-4bf7-9a6c-5584d2d58d63\") " pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-hgtdx"
Oct 31 17:41:16.998858 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:16.998817    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p6jb8\" (UniqueName: \"kubernetes.io/projected/14900971-460e-4126-9089-c4b110d539ab-kube-api-access-p6jb8\") pod \"e2e-worker-w6-d80080-openshift-default-956cddf57-g646p\" (UID: \"14900971-460e-4126-9089-c4b110d539ab\") " pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-g646p"
Oct 31 17:41:16.999037 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:16.998856    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a59e3d47-2d57-4bf7-9a6c-5584d2d58d63-istio-data\") pod \"e2e-worker-w6-d80080-openshift-default-956cddf57-hgtdx\" (UID: \"a59e3d47-2d57-4bf7-9a6c-5584d2d58d63\") " pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-hgtdx"
Oct 31 17:41:16.999037 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:16.998939    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/14900971-460e-4126-9089-c4b110d539ab-workload-socket\") pod \"e2e-worker-w6-d80080-openshift-default-956cddf57-g646p\" (UID: \"14900971-460e-4126-9089-c4b110d539ab\") " pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-g646p"
Oct 31 17:41:16.999037 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:16.998974    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v6ng4\" (UniqueName: \"kubernetes.io/projected/a59e3d47-2d57-4bf7-9a6c-5584d2d58d63-kube-api-access-v6ng4\") pod \"e2e-worker-w6-d80080-openshift-default-956cddf57-hgtdx\" (UID: \"a59e3d47-2d57-4bf7-9a6c-5584d2d58d63\") " pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-hgtdx"
Oct 31 17:41:16.999037 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:16.998995    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/14900971-460e-4126-9089-c4b110d539ab-credential-socket\") pod \"e2e-worker-w6-d80080-openshift-default-956cddf57-g646p\" (UID: \"14900971-460e-4126-9089-c4b110d539ab\") " pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-g646p"
Oct 31 17:41:16.999234 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:16.999181    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/14900971-460e-4126-9089-c4b110d539ab-istio-envoy\") pod \"e2e-worker-w6-d80080-openshift-default-956cddf57-g646p\" (UID: \"14900971-460e-4126-9089-c4b110d539ab\") " pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-g646p"
Oct 31 17:41:16.999234 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:16.999203    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/14900971-460e-4126-9089-c4b110d539ab-istio-token\") pod \"e2e-worker-w6-d80080-openshift-default-956cddf57-g646p\" (UID: \"14900971-460e-4126-9089-c4b110d539ab\") " pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-g646p"
Oct 31 17:41:16.999342 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:16.999240    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a59e3d47-2d57-4bf7-9a6c-5584d2d58d63-workload-socket\") pod \"e2e-worker-w6-d80080-openshift-default-956cddf57-hgtdx\" (UID: \"a59e3d47-2d57-4bf7-9a6c-5584d2d58d63\") " pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-hgtdx"
Oct 31 17:41:16.999342 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:16.999319    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/14900971-460e-4126-9089-c4b110d539ab-istio-podinfo\") pod \"e2e-worker-w6-d80080-openshift-default-956cddf57-g646p\" (UID: \"14900971-460e-4126-9089-c4b110d539ab\") " pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-g646p"
Oct 31 17:41:16.999461 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:16.999360    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/14900971-460e-4126-9089-c4b110d539ab-workload-certs\") pod \"e2e-worker-w6-d80080-openshift-default-956cddf57-g646p\" (UID: \"14900971-460e-4126-9089-c4b110d539ab\") " pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-g646p"
Oct 31 17:41:17.100392 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:17.100355    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a59e3d47-2d57-4bf7-9a6c-5584d2d58d63-istio-envoy\") pod \"e2e-worker-w6-d80080-openshift-default-956cddf57-hgtdx\" (UID: \"a59e3d47-2d57-4bf7-9a6c-5584d2d58d63\") " pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-hgtdx"
Oct 31 17:41:17.100959 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:17.100420    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a59e3d47-2d57-4bf7-9a6c-5584d2d58d63-credential-socket\") pod \"e2e-worker-w6-d80080-openshift-default-956cddf57-hgtdx\" (UID: \"a59e3d47-2d57-4bf7-9a6c-5584d2d58d63\") " pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-hgtdx"
Oct 31 17:41:17.100959 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:17.100464    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a59e3d47-2d57-4bf7-9a6c-5584d2d58d63-istiod-ca-cert\") pod \"e2e-worker-w6-d80080-openshift-default-956cddf57-hgtdx\" (UID: \"a59e3d47-2d57-4bf7-9a6c-5584d2d58d63\") " pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-hgtdx"
Oct 31 17:41:17.100959 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:17.100492    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p6jb8\" (UniqueName: \"kubernetes.io/projected/14900971-460e-4126-9089-c4b110d539ab-kube-api-access-p6jb8\") pod \"e2e-worker-w6-d80080-openshift-default-956cddf57-g646p\" (UID: \"14900971-460e-4126-9089-c4b110d539ab\") " pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-g646p"
Oct 31 17:41:17.100959 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:17.100529    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a59e3d47-2d57-4bf7-9a6c-5584d2d58d63-istio-data\") pod \"e2e-worker-w6-d80080-openshift-default-956cddf57-hgtdx\" (UID: \"a59e3d47-2d57-4bf7-9a6c-5584d2d58d63\") " pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-hgtdx"
Oct 31 17:41:17.100959 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:17.100596    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/14900971-460e-4126-9089-c4b110d539ab-workload-socket\") pod \"e2e-worker-w6-d80080-openshift-default-956cddf57-g646p\" (UID: \"14900971-460e-4126-9089-c4b110d539ab\") " pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-g646p"
Oct 31 17:41:17.100959 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:17.100631    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v6ng4\" (UniqueName: \"kubernetes.io/projected/a59e3d47-2d57-4bf7-9a6c-5584d2d58d63-kube-api-access-v6ng4\") pod \"e2e-worker-w6-d80080-openshift-default-956cddf57-hgtdx\" (UID: \"a59e3d47-2d57-4bf7-9a6c-5584d2d58d63\") " pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-hgtdx"
Oct 31 17:41:17.100959 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:17.100661    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/14900971-460e-4126-9089-c4b110d539ab-credential-socket\") pod \"e2e-worker-w6-d80080-openshift-default-956cddf57-g646p\" (UID: \"14900971-460e-4126-9089-c4b110d539ab\") " pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-g646p"
Oct 31 17:41:17.100959 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:17.100743    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/14900971-460e-4126-9089-c4b110d539ab-istio-envoy\") pod \"e2e-worker-w6-d80080-openshift-default-956cddf57-g646p\" (UID: \"14900971-460e-4126-9089-c4b110d539ab\") " pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-g646p"
Oct 31 17:41:17.100959 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:17.100768    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/14900971-460e-4126-9089-c4b110d539ab-istio-token\") pod \"e2e-worker-w6-d80080-openshift-default-956cddf57-g646p\" (UID: \"14900971-460e-4126-9089-c4b110d539ab\") " pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-g646p"
Oct 31 17:41:17.100959 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:17.100833    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a59e3d47-2d57-4bf7-9a6c-5584d2d58d63-workload-socket\") pod \"e2e-worker-w6-d80080-openshift-default-956cddf57-hgtdx\" (UID: \"a59e3d47-2d57-4bf7-9a6c-5584d2d58d63\") " pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-hgtdx"
Oct 31 17:41:17.100959 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:17.100887    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a59e3d47-2d57-4bf7-9a6c-5584d2d58d63-credential-socket\") pod \"e2e-worker-w6-d80080-openshift-default-956cddf57-hgtdx\" (UID: \"a59e3d47-2d57-4bf7-9a6c-5584d2d58d63\") " pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-hgtdx"
Oct 31 17:41:17.101881 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:17.100876    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/14900971-460e-4126-9089-c4b110d539ab-istio-podinfo\") pod \"e2e-worker-w6-d80080-openshift-default-956cddf57-g646p\" (UID: \"14900971-460e-4126-9089-c4b110d539ab\") " pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-g646p"
Oct 31 17:41:17.101881 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:17.101135    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/14900971-460e-4126-9089-c4b110d539ab-workload-certs\") pod \"e2e-worker-w6-d80080-openshift-default-956cddf57-g646p\" (UID: \"14900971-460e-4126-9089-c4b110d539ab\") " pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-g646p"
Oct 31 17:41:17.101881 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:17.101145    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a59e3d47-2d57-4bf7-9a6c-5584d2d58d63-istiod-ca-cert\") pod \"e2e-worker-w6-d80080-openshift-default-956cddf57-hgtdx\" (UID: \"a59e3d47-2d57-4bf7-9a6c-5584d2d58d63\") " pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-hgtdx"
Oct 31 17:41:17.101881 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:17.101212    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a59e3d47-2d57-4bf7-9a6c-5584d2d58d63-istio-token\") pod \"e2e-worker-w6-d80080-openshift-default-956cddf57-hgtdx\" (UID: \"a59e3d47-2d57-4bf7-9a6c-5584d2d58d63\") " pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-hgtdx"
Oct 31 17:41:17.101881 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:17.101235    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a59e3d47-2d57-4bf7-9a6c-5584d2d58d63-istio-podinfo\") pod \"e2e-worker-w6-d80080-openshift-default-956cddf57-hgtdx\" (UID: \"a59e3d47-2d57-4bf7-9a6c-5584d2d58d63\") " pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-hgtdx"
Oct 31 17:41:17.101881 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:17.101331    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a59e3d47-2d57-4bf7-9a6c-5584d2d58d63-workload-certs\") pod \"e2e-worker-w6-d80080-openshift-default-956cddf57-hgtdx\" (UID: \"a59e3d47-2d57-4bf7-9a6c-5584d2d58d63\") " pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-hgtdx"
Oct 31 17:41:17.101881 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:17.101354    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a59e3d47-2d57-4bf7-9a6c-5584d2d58d63-istio-data\") pod \"e2e-worker-w6-d80080-openshift-default-956cddf57-hgtdx\" (UID: \"a59e3d47-2d57-4bf7-9a6c-5584d2d58d63\") " pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-hgtdx"
Oct 31 17:41:17.101881 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:17.101419    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/14900971-460e-4126-9089-c4b110d539ab-istiod-ca-cert\") pod \"e2e-worker-w6-d80080-openshift-default-956cddf57-g646p\" (UID: \"14900971-460e-4126-9089-c4b110d539ab\") " pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-g646p"
Oct 31 17:41:17.101881 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:17.101473    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/14900971-460e-4126-9089-c4b110d539ab-istio-data\") pod \"e2e-worker-w6-d80080-openshift-default-956cddf57-g646p\" (UID: \"14900971-460e-4126-9089-c4b110d539ab\") " pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-g646p"
Oct 31 17:41:17.101881 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:17.101527    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/14900971-460e-4126-9089-c4b110d539ab-workload-certs\") pod \"e2e-worker-w6-d80080-openshift-default-956cddf57-g646p\" (UID: \"14900971-460e-4126-9089-c4b110d539ab\") " pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-g646p"
Oct 31 17:41:17.101881 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:17.101678    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a59e3d47-2d57-4bf7-9a6c-5584d2d58d63-workload-certs\") pod \"e2e-worker-w6-d80080-openshift-default-956cddf57-hgtdx\" (UID: \"a59e3d47-2d57-4bf7-9a6c-5584d2d58d63\") " pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-hgtdx"
Oct 31 17:41:17.101881 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:17.101809    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/14900971-460e-4126-9089-c4b110d539ab-istio-data\") pod \"e2e-worker-w6-d80080-openshift-default-956cddf57-g646p\" (UID: \"14900971-460e-4126-9089-c4b110d539ab\") " pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-g646p"
Oct 31 17:41:17.102567 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:17.101986    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a59e3d47-2d57-4bf7-9a6c-5584d2d58d63-workload-socket\") pod \"e2e-worker-w6-d80080-openshift-default-956cddf57-hgtdx\" (UID: \"a59e3d47-2d57-4bf7-9a6c-5584d2d58d63\") " pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-hgtdx"
Oct 31 17:41:17.102567 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:17.102049    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/14900971-460e-4126-9089-c4b110d539ab-workload-socket\") pod \"e2e-worker-w6-d80080-openshift-default-956cddf57-g646p\" (UID: \"14900971-460e-4126-9089-c4b110d539ab\") " pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-g646p"
Oct 31 17:41:17.103074 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:17.102675    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/14900971-460e-4126-9089-c4b110d539ab-istiod-ca-cert\") pod \"e2e-worker-w6-d80080-openshift-default-956cddf57-g646p\" (UID: \"14900971-460e-4126-9089-c4b110d539ab\") " pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-g646p"
Oct 31 17:41:17.103074 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:17.102954    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/14900971-460e-4126-9089-c4b110d539ab-credential-socket\") pod \"e2e-worker-w6-d80080-openshift-default-956cddf57-g646p\" (UID: \"14900971-460e-4126-9089-c4b110d539ab\") " pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-g646p"
Oct 31 17:41:17.103394 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:17.103343    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/14900971-460e-4126-9089-c4b110d539ab-istio-envoy\") pod \"e2e-worker-w6-d80080-openshift-default-956cddf57-g646p\" (UID: \"14900971-460e-4126-9089-c4b110d539ab\") " pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-g646p"
Oct 31 17:41:17.104704 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:17.104686    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/14900971-460e-4126-9089-c4b110d539ab-istio-podinfo\") pod \"e2e-worker-w6-d80080-openshift-default-956cddf57-g646p\" (UID: \"14900971-460e-4126-9089-c4b110d539ab\") " pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-g646p"
Oct 31 17:41:17.104793 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:17.104704    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a59e3d47-2d57-4bf7-9a6c-5584d2d58d63-istio-podinfo\") pod \"e2e-worker-w6-d80080-openshift-default-956cddf57-hgtdx\" (UID: \"a59e3d47-2d57-4bf7-9a6c-5584d2d58d63\") " pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-hgtdx"
Oct 31 17:41:17.105317 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:17.105297    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a59e3d47-2d57-4bf7-9a6c-5584d2d58d63-istio-envoy\") pod \"e2e-worker-w6-d80080-openshift-default-956cddf57-hgtdx\" (UID: \"a59e3d47-2d57-4bf7-9a6c-5584d2d58d63\") " pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-hgtdx"
Oct 31 17:41:17.120673 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:17.120648    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v6ng4\" (UniqueName: \"kubernetes.io/projected/a59e3d47-2d57-4bf7-9a6c-5584d2d58d63-kube-api-access-v6ng4\") pod \"e2e-worker-w6-d80080-openshift-default-956cddf57-hgtdx\" (UID: \"a59e3d47-2d57-4bf7-9a6c-5584d2d58d63\") " pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-hgtdx"
Oct 31 17:41:17.120863 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:17.120846    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p6jb8\" (UniqueName: \"kubernetes.io/projected/14900971-460e-4126-9089-c4b110d539ab-kube-api-access-p6jb8\") pod \"e2e-worker-w6-d80080-openshift-default-956cddf57-g646p\" (UID: \"14900971-460e-4126-9089-c4b110d539ab\") " pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-g646p"
Oct 31 17:41:17.121311 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:17.121290    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/14900971-460e-4126-9089-c4b110d539ab-istio-token\") pod \"e2e-worker-w6-d80080-openshift-default-956cddf57-g646p\" (UID: \"14900971-460e-4126-9089-c4b110d539ab\") " pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-g646p"
Oct 31 17:41:17.121917 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:17.121898    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a59e3d47-2d57-4bf7-9a6c-5584d2d58d63-istio-token\") pod \"e2e-worker-w6-d80080-openshift-default-956cddf57-hgtdx\" (UID: \"a59e3d47-2d57-4bf7-9a6c-5584d2d58d63\") " pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-hgtdx"
Oct 31 17:41:17.212629 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:17.212577    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-g646p"
Oct 31 17:41:17.265963 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:17.265931    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-hgtdx"
Oct 31 17:41:17.304522 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:17.303122    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-zsl5x"]
Oct 31 17:41:17.306986 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:41:17.306956    4276 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb2b7f15e_756a_40e6_90af_961551719a7d.slice/crio-cb136a0a66bdd40f74a955e6da45153396d648385a1fd675d7132591b9698c44 WatchSource:0}: Error finding container cb136a0a66bdd40f74a955e6da45153396d648385a1fd675d7132591b9698c44: Status 404 returned error can't find the container with id cb136a0a66bdd40f74a955e6da45153396d648385a1fd675d7132591b9698c44
Oct 31 17:41:17.310069 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:17.310031    4276 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 17:41:17.310170 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:17.310124    4276 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 17:41:17.310231 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:17.310167    4276 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 17:41:17.516383 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:41:17.516354    4276 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod14900971_460e_4126_9089_c4b110d539ab.slice/crio-1b2bf4b873e4dbe5f87f6511810d902f3434a7072e5690bf22601c643ad47228 WatchSource:0}: Error finding container 1b2bf4b873e4dbe5f87f6511810d902f3434a7072e5690bf22601c643ad47228: Status 404 returned error can't find the container with id 1b2bf4b873e4dbe5f87f6511810d902f3434a7072e5690bf22601c643ad47228
Oct 31 17:41:17.517953 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:17.517932    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-g646p"]
Oct 31 17:41:17.519381 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:17.519347    4276 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 17:41:17.519523 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:17.519428    4276 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 17:41:17.519523 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:17.519491    4276 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 17:41:17.589936 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:17.589914    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-hgtdx"]
Oct 31 17:41:17.590876 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:41:17.590851    4276 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda59e3d47_2d57_4bf7_9a6c_5584d2d58d63.slice/crio-8509102b76b002a2836ee593e236405c1e58aef0590999ed688a837d48ea1f60 WatchSource:0}: Error finding container 8509102b76b002a2836ee593e236405c1e58aef0590999ed688a837d48ea1f60: Status 404 returned error can't find the container with id 8509102b76b002a2836ee593e236405c1e58aef0590999ed688a837d48ea1f60
Oct 31 17:41:17.593644 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:17.593612    4276 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 17:41:17.593725 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:17.593692    4276 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 17:41:17.593725 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:17.593719    4276 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 17:41:17.667170 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:17.667099    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-hgtdx" event={"ID":"a59e3d47-2d57-4bf7-9a6c-5584d2d58d63","Type":"ContainerStarted","Data":"8509102b76b002a2836ee593e236405c1e58aef0590999ed688a837d48ea1f60"}
Oct 31 17:41:17.669317 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:17.669294    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-zsl5x" event={"ID":"b2b7f15e-756a-40e6-90af-961551719a7d","Type":"ContainerStarted","Data":"d8d93d1ddbaec4597147921eac145f6223bd6f3d5d1e15e0f4f400c39bf468f1"}
Oct 31 17:41:17.669432 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:17.669334    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-zsl5x" event={"ID":"b2b7f15e-756a-40e6-90af-961551719a7d","Type":"ContainerStarted","Data":"cb136a0a66bdd40f74a955e6da45153396d648385a1fd675d7132591b9698c44"}
Oct 31 17:41:17.671212 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:17.671181    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-g646p" event={"ID":"14900971-460e-4126-9089-c4b110d539ab","Type":"ContainerStarted","Data":"661fccd383b68b0099aeec974f6cc1dd943864f8e2e0df4870d8bcd831a23a1a"}
Oct 31 17:41:17.671323 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:17.671230    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-g646p" event={"ID":"14900971-460e-4126-9089-c4b110d539ab","Type":"ContainerStarted","Data":"1b2bf4b873e4dbe5f87f6511810d902f3434a7072e5690bf22601c643ad47228"}
Oct 31 17:41:17.717928 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:17.717863    4276 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-zsl5x" podStartSLOduration=1.717846208 podStartE2EDuration="1.717846208s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 17:41:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 17:41:17.714406604 +0000 UTC m=+840.512151516" watchObservedRunningTime="2026-10-31 17:41:17.717846208 +0000 UTC m=+840.515591138"
Oct 31 17:41:17.761353 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:17.761282    4276 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-g646p" podStartSLOduration=1.761262579 podStartE2EDuration="1.761262579s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 17:41:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 17:41:17.756983682 +0000 UTC m=+840.554728610" watchObservedRunningTime="2026-10-31 17:41:17.761262579 +0000 UTC m=+840.559007508"
Oct 31 17:41:17.853925 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:17.853894    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:41:17.914874 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:17.913867    4276 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-1c0bfd/distinct-2-w4-1c0bfd-kserve-847bcf6cf4-lt8ls"
Oct 31 17:41:17.981464 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:17.980537    4276 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-zsl5x"
Oct 31 17:41:18.003745 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:18.002064    4276 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-1c0bfd/distinct-2-w4-1c0bfd-kserve-847bcf6cf4-lt8ls"
Oct 31 17:41:18.213495 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:18.213376    4276 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-g646p"
Oct 31 17:41:18.214959 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:18.214933    4276 patch_prober.go:29] interesting pod/e2e-worker-w6-d80080-openshift-default-956cddf57-g646p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body=
Oct 31 17:41:18.215101 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:18.214986    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-g646p" podUID="14900971-460e-4126-9089-c4b110d539ab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused"
Oct 31 17:41:18.714355 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:18.714314    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-hgtdx" event={"ID":"a59e3d47-2d57-4bf7-9a6c-5584d2d58d63","Type":"ContainerStarted","Data":"c11a9fcac7492f02a86e3e250a3a1b0c7b66682168c63268a347b1c9db01093e"}
Oct 31 17:41:18.745832 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:18.745799    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-1c0bfd/distinct-2-w4-1c0bfd-kserve-847bcf6cf4-lt8ls"
Oct 31 17:41:18.801933 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:18.801863    4276 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-hgtdx" podStartSLOduration=2.801830127 podStartE2EDuration="2.801830127s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 17:41:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 17:41:18.775038555 +0000 UTC m=+841.572783489" watchObservedRunningTime="2026-10-31 17:41:18.801830127 +0000 UTC m=+841.599575057"
Oct 31 17:41:18.983968 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:18.983859    4276 patch_prober.go:29] interesting pod/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-zsl5x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body=
Oct 31 17:41:18.984265 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:18.984216    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-zsl5x" podUID="b2b7f15e-756a-40e6-90af-961551719a7d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused"
Oct 31 17:41:19.215352 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:19.214876    4276 patch_prober.go:29] interesting pod/e2e-worker-w6-d80080-openshift-default-956cddf57-g646p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body=
Oct 31 17:41:19.215352 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:19.214957    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-g646p" podUID="14900971-460e-4126-9089-c4b110d539ab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused"
Oct 31 17:41:19.284523 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:19.279740    4276 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-hgtdx"
Oct 31 17:41:19.301467 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:19.291579    4276 patch_prober.go:29] interesting pod/e2e-worker-w6-d80080-openshift-default-956cddf57-hgtdx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body=
Oct 31 17:41:19.301467 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:19.291636    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-hgtdx" podUID="a59e3d47-2d57-4bf7-9a6c-5584d2d58d63" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused"
Oct 31 17:41:19.981467 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:19.976517    4276 patch_prober.go:29] interesting pod/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-zsl5x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body=
Oct 31 17:41:19.981467 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:19.976571    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-zsl5x" podUID="b2b7f15e-756a-40e6-90af-961551719a7d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused"
Oct 31 17:41:19.981467 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:19.977337    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:41:20.220418 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:20.220055    4276 patch_prober.go:29] interesting pod/e2e-worker-w6-d80080-openshift-default-956cddf57-g646p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body=
Oct 31 17:41:20.220418 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:20.220129    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-g646p" podUID="14900971-460e-4126-9089-c4b110d539ab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused"
Oct 31 17:41:20.291522 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:20.290474    4276 patch_prober.go:29] interesting pod/e2e-worker-w6-d80080-openshift-default-956cddf57-hgtdx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body=
Oct 31 17:41:20.291522 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:20.290530    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-hgtdx" podUID="a59e3d47-2d57-4bf7-9a6c-5584d2d58d63" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused"
Oct 31 17:41:20.983468 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:20.983274    4276 patch_prober.go:29] interesting pod/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-zsl5x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body=
Oct 31 17:41:20.983468 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:20.983335    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-zsl5x" podUID="b2b7f15e-756a-40e6-90af-961551719a7d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused"
Oct 31 17:41:21.344465 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:21.341917    4276 patch_prober.go:29] interesting pod/e2e-worker-w6-d80080-openshift-default-956cddf57-hgtdx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body=
Oct 31 17:41:21.344465 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:21.341975    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-hgtdx" podUID="a59e3d47-2d57-4bf7-9a6c-5584d2d58d63" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused"
Oct 31 17:41:21.348565 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:21.345123    4276 patch_prober.go:29] interesting pod/e2e-worker-w6-d80080-openshift-default-956cddf57-g646p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body=
Oct 31 17:41:21.348565 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:21.345201    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-g646p" podUID="14900971-460e-4126-9089-c4b110d539ab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused"
Oct 31 17:41:22.038489 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:22.038015    4276 patch_prober.go:29] interesting pod/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-zsl5x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body=
Oct 31 17:41:22.038489 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:22.038085    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-zsl5x" podUID="b2b7f15e-756a-40e6-90af-961551719a7d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused"
Oct 31 17:41:22.247489 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:22.247065    4276 patch_prober.go:29] interesting pod/e2e-worker-w6-d80080-openshift-default-956cddf57-g646p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body=
Oct 31 17:41:22.247489 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:22.247123    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-g646p" podUID="14900971-460e-4126-9089-c4b110d539ab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused"
Oct 31 17:41:22.338501 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:22.335674    4276 patch_prober.go:29] interesting pod/e2e-worker-w6-d80080-openshift-default-956cddf57-hgtdx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body=
Oct 31 17:41:22.338501 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:22.335737    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-hgtdx" podUID="a59e3d47-2d57-4bf7-9a6c-5584d2d58d63" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused"
Oct 31 17:41:22.338501 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:22.336848    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:41:22.994482 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:22.994424    4276 patch_prober.go:29] interesting pod/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-zsl5x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body=
Oct 31 17:41:22.995078 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:22.995045    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-zsl5x" podUID="b2b7f15e-756a-40e6-90af-961551719a7d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused"
Oct 31 17:41:23.224875 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:23.224840    4276 patch_prober.go:29] interesting pod/e2e-worker-w6-d80080-openshift-default-956cddf57-g646p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body=
Oct 31 17:41:23.225124 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:23.225094    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-g646p" podUID="14900971-460e-4126-9089-c4b110d539ab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused"
Oct 31 17:41:23.273912 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:23.273828    4276 patch_prober.go:29] interesting pod/e2e-worker-w6-d80080-openshift-default-956cddf57-hgtdx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body=
Oct 31 17:41:23.274160 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:23.274134    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-hgtdx" podUID="a59e3d47-2d57-4bf7-9a6c-5584d2d58d63" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused"
Oct 31 17:41:23.992311 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:23.988346    4276 patch_prober.go:29] interesting pod/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-zsl5x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body=
Oct 31 17:41:23.992311 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:23.988404    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-zsl5x" podUID="b2b7f15e-756a-40e6-90af-961551719a7d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused"
Oct 31 17:41:24.228470 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:24.217844    4276 patch_prober.go:29] interesting pod/e2e-worker-w6-d80080-openshift-default-956cddf57-g646p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body=
Oct 31 17:41:24.228470 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:24.217908    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-g646p" podUID="14900971-460e-4126-9089-c4b110d539ab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused"
Oct 31 17:41:24.273521 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:24.270813    4276 patch_prober.go:29] interesting pod/e2e-worker-w6-d80080-openshift-default-956cddf57-hgtdx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body=
Oct 31 17:41:24.273521 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:24.270875    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-hgtdx" podUID="a59e3d47-2d57-4bf7-9a6c-5584d2d58d63" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused"
Oct 31 17:41:24.982922 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:24.982879    4276 patch_prober.go:29] interesting pod/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-zsl5x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body=
Oct 31 17:41:24.983259 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:24.983214    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-zsl5x" podUID="b2b7f15e-756a-40e6-90af-961551719a7d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused"
Oct 31 17:41:25.161473 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:25.147346    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:41:25.284521 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:25.221817    4276 patch_prober.go:29] interesting pod/e2e-worker-w6-d80080-openshift-default-956cddf57-g646p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body=
Oct 31 17:41:25.284521 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:25.282235    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-g646p" podUID="14900971-460e-4126-9089-c4b110d539ab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused"
Oct 31 17:41:25.305466 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:25.299725    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ee816e/premium-w0-ee816e-kserve-65c86cd6db-lnt9p" podUID="e536c902-52a9-4b36-a2ad-eb5500781a5e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.49:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 17:41:25.427930 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:25.385576    4276 patch_prober.go:29] interesting pod/e2e-worker-w6-d80080-openshift-default-956cddf57-hgtdx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body=
Oct 31 17:41:25.429041 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:25.429004    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-hgtdx" podUID="a59e3d47-2d57-4bf7-9a6c-5584d2d58d63" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused"
Oct 31 17:41:25.978600 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:25.978085    4276 patch_prober.go:29] interesting pod/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-zsl5x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body=
Oct 31 17:41:25.978600 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:25.978161    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-zsl5x" podUID="b2b7f15e-756a-40e6-90af-961551719a7d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused"
Oct 31 17:41:26.265520 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:26.256112    4276 patch_prober.go:29] interesting pod/e2e-worker-w6-d80080-openshift-default-956cddf57-g646p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body=
Oct 31 17:41:26.265520 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:26.256169    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-g646p" podUID="14900971-460e-4126-9089-c4b110d539ab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused"
Oct 31 17:41:26.267091 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:26.267055    4276 patch_prober.go:29] interesting pod/e2e-worker-w6-d80080-openshift-default-956cddf57-hgtdx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body=
Oct 31 17:41:26.267219 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:26.267117    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-hgtdx" podUID="a59e3d47-2d57-4bf7-9a6c-5584d2d58d63" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused"
Oct 31 17:41:26.330333 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:26.330249    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ee816e/premium-w0-ee816e-kserve-65c86cd6db-lnt9p" podUID="e536c902-52a9-4b36-a2ad-eb5500781a5e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.49:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 17:41:26.331258 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:26.331237    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-ee816e/premium-w0-ee816e-kserve-65c86cd6db-lnt9p"
Oct 31 17:41:26.333687 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:26.333659    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-5c446c/distinct-w2-5c446c-kserve-69d97d6968-gtvjv" podUID="2d1185cc-bac1-45ca-a250-8f44f8589162" containerName="main" probeResult="failure" output="Get \"https://10.133.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 17:41:26.333888 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:26.333867    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-5c446c/distinct-2-w2-5c446c-kserve-5d9cd79c47-p4j5l" podUID="4c288908-92e3-4c45-8ba5-2f86c9f38b3d" containerName="main" probeResult="failure" output="Get \"https://10.133.0.65:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 17:41:26.440198 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:26.438026    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-0e59ca/premium-w1-0e59ca-kserve-58765f9746-fqwq7" podUID="8cf1e52e-70f8-42e2-bdf3-a315e17f10f5" containerName="main" probeResult="failure" output="Get \"https://10.133.0.48:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 17:41:26.592479 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:26.585123    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-d80080/distinct-w6-d80080-kserve-664576c8f5-6jc4d" podUID="b9c2f445-a191-4b00-802a-1b0ded068411" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 17:41:26.835761 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:26.835711    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-1c0bfd/distinct-2-w4-1c0bfd-kserve-847bcf6cf4-lt8ls" podUID="f8a45b7c-e5ae-4321-ad81-fd6353b2495d" containerName="main" probeResult="failure" output="Get \"https://10.133.0.67:8000/ready\": context deadline exceeded"
Oct 31 17:41:27.293467 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:27.254528    4276 patch_prober.go:29] interesting pod/e2e-worker-w6-d80080-openshift-default-956cddf57-g646p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body=
Oct 31 17:41:27.293467 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:27.291572    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-g646p" podUID="14900971-460e-4126-9089-c4b110d539ab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused"
Oct 31 17:41:27.427497 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:27.389254    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-5c446c/distinct-w2-5c446c-kserve-69d97d6968-gtvjv" podUID="2d1185cc-bac1-45ca-a250-8f44f8589162" containerName="main" probeResult="failure" output="Get \"https://10.133.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 17:41:27.427497 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:27.412159    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-5c446c/distinct-w2-5c446c-kserve-69d97d6968-gtvjv"
Oct 31 17:41:27.471484 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:27.455001    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-0e59ca/premium-w1-0e59ca-kserve-58765f9746-fqwq7" podUID="8cf1e52e-70f8-42e2-bdf3-a315e17f10f5" containerName="main" probeResult="failure" output="Get \"https://10.133.0.48:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 17:41:27.471484 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:27.456073    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-0e59ca/premium-w1-0e59ca-kserve-58765f9746-fqwq7"
Oct 31 17:41:27.516577 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:27.503879    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-5c446c/distinct-2-w2-5c446c-kserve-5d9cd79c47-p4j5l" podUID="4c288908-92e3-4c45-8ba5-2f86c9f38b3d" containerName="main" probeResult="failure" output="Get \"https://10.133.0.65:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 17:41:27.563282 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:27.532581    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ee816e/premium-w0-ee816e-kserve-65c86cd6db-lnt9p" podUID="e536c902-52a9-4b36-a2ad-eb5500781a5e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.49:8000/ready\": context deadline exceeded"
Oct 31 17:41:27.565805 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:27.565780    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-5c446c/distinct-2-w2-5c446c-kserve-5d9cd79c47-p4j5l"
Oct 31 17:41:27.804742 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:27.619389    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-d80080/distinct-w6-d80080-kserve-664576c8f5-6jc4d" podUID="b9c2f445-a191-4b00-802a-1b0ded068411" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": context deadline exceeded"
Oct 31 17:41:27.910870 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:27.856542    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-1c0bfd/distinct-2-w4-1c0bfd-kserve-847bcf6cf4-lt8ls" podUID="f8a45b7c-e5ae-4321-ad81-fd6353b2495d" containerName="main" probeResult="failure" output="Get \"https://10.133.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 17:41:28.044588 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:28.044545    4276 patch_prober.go:29] interesting pod/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-zsl5x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 17:41:28.044863 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:28.044831    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-zsl5x" podUID="b2b7f15e-756a-40e6-90af-961551719a7d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 17:41:28.135473 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:28.132953    4276 patch_prober.go:29] interesting pod/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-9lts8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 17:41:28.135473 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:28.133043    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-9lts8" podUID="79a9c2b2-8458-4a74-afde-ee00374beb5b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 17:41:28.237298 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:28.237205    4276 patch_prober.go:29] interesting pod/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-m99sg container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 17:41:28.237590 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:28.237543    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-m99sg" podUID="56c4ff3e-b2f2-4dce-938d-48672451d1fe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 17:41:28.496891 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:28.496537    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-0e59ca/premium-w1-0e59ca-kserve-58765f9746-fqwq7" podUID="8cf1e52e-70f8-42e2-bdf3-a315e17f10f5" containerName="main" probeResult="failure" output="Get \"https://10.133.0.48:8000/ready\": context deadline exceeded"
Oct 31 17:41:28.498678 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:28.498633    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-5c446c/distinct-w2-5c446c-kserve-69d97d6968-gtvjv" podUID="2d1185cc-bac1-45ca-a250-8f44f8589162" containerName="main" probeResult="failure" output="Get \"https://10.133.0.58:8000/ready\": context deadline exceeded"
Oct 31 17:41:28.499160 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:28.499124    4276 patch_prober.go:29] interesting pod/e2e-worker-w6-d80080-openshift-default-956cddf57-hgtdx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 17:41:28.499359 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:28.499336    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-hgtdx" podUID="a59e3d47-2d57-4bf7-9a6c-5584d2d58d63" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 17:41:28.540707 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:28.540562    4276 patch_prober.go:29] interesting pod/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-kfkwb container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 17:41:28.540707 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:28.540629    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-kfkwb" podUID="4b04b1b0-d2cc-4304-92e6-90e48105f442" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 17:41:28.612470 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:28.584761    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ee816e/premium-w0-ee816e-kserve-65c86cd6db-lnt9p" podUID="e536c902-52a9-4b36-a2ad-eb5500781a5e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.49:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 17:41:28.612470 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:28.585103    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-5c446c/distinct-2-w2-5c446c-kserve-5d9cd79c47-p4j5l" podUID="4c288908-92e3-4c45-8ba5-2f86c9f38b3d" containerName="main" probeResult="failure" output="Get \"https://10.133.0.65:8000/ready\": context deadline exceeded"
Oct 31 17:41:28.707567 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:28.707518    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-d80080/distinct-w6-d80080-kserve-664576c8f5-6jc4d"
Oct 31 17:41:28.755868 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:28.755774    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-1c0bfd/distinct-2-w4-1c0bfd-kserve-847bcf6cf4-lt8ls"
Oct 31 17:41:28.846483 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:28.843779    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-d80080/distinct-w6-d80080-kserve-664576c8f5-6jc4d" podUID="b9c2f445-a191-4b00-802a-1b0ded068411" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 17:41:29.203497 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:29.164527    4276 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-1c0bfd/distinct-2-w4-1c0bfd-kserve-847bcf6cf4-lt8ls" podUID="f8a45b7c-e5ae-4321-ad81-fd6353b2495d" containerName="main" probeResult="failure" output="Get \"https://10.133.0.67:8000/health\": context deadline exceeded"
Oct 31 17:41:29.216481 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:29.214169    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-1c0bfd/distinct-2-w4-1c0bfd-kserve-847bcf6cf4-lt8ls" podUID="f8a45b7c-e5ae-4321-ad81-fd6353b2495d" containerName="main" probeResult="failure" output="Get \"https://10.133.0.67:8000/ready\": context deadline exceeded"
Oct 31 17:41:29.322469 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:29.312094    4276 patch_prober.go:29] interesting pod/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-zsl5x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 17:41:29.322469 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:29.312165    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-zsl5x" podUID="b2b7f15e-756a-40e6-90af-961551719a7d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 17:41:29.324547 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:29.324513    4276 patch_prober.go:29] interesting pod/e2e-worker-w6-d80080-openshift-default-956cddf57-g646p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 17:41:29.324753 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:29.324726    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-g646p" podUID="14900971-460e-4126-9089-c4b110d539ab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 17:41:29.520519 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:29.510602    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-0e59ca/premium-w1-0e59ca-kserve-58765f9746-fqwq7" podUID="8cf1e52e-70f8-42e2-bdf3-a315e17f10f5" containerName="main" probeResult="failure" output="Get \"https://10.133.0.48:8000/ready\": context deadline exceeded"
Oct 31 17:41:29.520519 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:29.512172    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-5c446c/distinct-w2-5c446c-kserve-69d97d6968-gtvjv" podUID="2d1185cc-bac1-45ca-a250-8f44f8589162" containerName="main" probeResult="failure" output="Get \"https://10.133.0.58:8000/ready\": context deadline exceeded"
Oct 31 17:41:29.603469 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:29.567090    4276 patch_prober.go:29] interesting pod/e2e-worker-w6-d80080-openshift-default-956cddf57-hgtdx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 17:41:29.603469 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:29.567149    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-hgtdx" podUID="a59e3d47-2d57-4bf7-9a6c-5584d2d58d63" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 17:41:29.603469 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:29.602500    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ee816e/premium-w0-ee816e-kserve-65c86cd6db-lnt9p" podUID="e536c902-52a9-4b36-a2ad-eb5500781a5e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.49:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 17:41:29.658474 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:29.602653    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-5c446c/distinct-2-w2-5c446c-kserve-5d9cd79c47-p4j5l" podUID="4c288908-92e3-4c45-8ba5-2f86c9f38b3d" containerName="main" probeResult="failure" output="Get \"https://10.133.0.65:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 17:41:29.845713 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:29.845665    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-d80080/distinct-w6-d80080-kserve-664576c8f5-6jc4d" podUID="b9c2f445-a191-4b00-802a-1b0ded068411" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 17:41:30.033740 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:30.033709    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-d80080/distinct-w6-d80080-kserve-664576c8f5-6jc4d"
Oct 31 17:41:30.058420 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:30.058386    4276 patch_prober.go:29] interesting pod/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-rfwx5 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 17:41:30.058696 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:30.058667    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-rfwx5" podUID="5155d374-e42a-4c40-9485-935443ede2c5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 17:41:30.143617 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:30.143544    4276 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-zsl5x"
Oct 31 17:41:30.149155 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:30.149127    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-zsl5x"
Oct 31 17:41:30.149519 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:30.149496    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-5c446c/distinct-w2-5c446c-kserve-69d97d6968-gtvjv"
Oct 31 17:41:30.161128 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:30.161105    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-zsl5x"
Oct 31 17:41:30.185080 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:30.185055    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-0e59ca/premium-w1-0e59ca-kserve-58765f9746-fqwq7"
Oct 31 17:41:30.257975 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:30.238509    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-1c0bfd/distinct-2-w4-1c0bfd-kserve-847bcf6cf4-lt8ls" podUID="f8a45b7c-e5ae-4321-ad81-fd6353b2495d" containerName="main" probeResult="failure" output="Get \"https://10.133.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 17:41:30.271498 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:30.258482    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-5c446c/distinct-2-w2-5c446c-kserve-5d9cd79c47-p4j5l"
Oct 31 17:41:30.400587 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:30.372564    4276 patch_prober.go:29] interesting pod/e2e-worker-w6-d80080-openshift-default-956cddf57-g646p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 17:41:30.400847 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:30.400815    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-g646p" podUID="14900971-460e-4126-9089-c4b110d539ab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 17:41:30.640703 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:30.640475    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-1c0bfd/distinct-2-w4-1c0bfd-kserve-847bcf6cf4-lt8ls"
Oct 31 17:41:30.645202 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:30.645154    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ee816e/premium-w0-ee816e-kserve-65c86cd6db-lnt9p" podUID="e536c902-52a9-4b36-a2ad-eb5500781a5e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.49:8000/ready\": context deadline exceeded"
Oct 31 17:41:30.663625 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:30.663544    4276 patch_prober.go:29] interesting pod/e2e-worker-w6-d80080-openshift-default-956cddf57-hgtdx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 17:41:30.672589 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:30.672546    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-hgtdx" podUID="a59e3d47-2d57-4bf7-9a6c-5584d2d58d63" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 17:41:30.920798 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:30.920668    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-ee816e/premium-w0-ee816e-kserve-65c86cd6db-lnt9p"
Oct 31 17:41:31.154217 ip-10-0-129-127 kubenswrapper[4276]: E1031 17:41:31.154181    4276 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/system.slice/systemd-tmpfiles-clean.service\": RecentStats: unable to find data in memory cache]"
Oct 31 17:41:31.401366 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:31.401328    4276 patch_prober.go:29] interesting pod/e2e-worker-w6-d80080-openshift-default-956cddf57-g646p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 17:41:31.401563 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:31.401419    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-g646p" podUID="14900971-460e-4126-9089-c4b110d539ab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 17:41:31.641119 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:31.641090    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:41:31.747883 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:31.747764    4276 patch_prober.go:29] interesting pod/e2e-worker-w6-d80080-openshift-default-956cddf57-hgtdx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 17:41:31.748221 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:31.748171    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-hgtdx" podUID="a59e3d47-2d57-4bf7-9a6c-5584d2d58d63" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 17:41:32.217467 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:32.214034    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-g646p"
Oct 31 17:41:32.273467 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:32.266999    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-hgtdx"
Oct 31 17:41:32.394874 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:32.394826    4276 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-g646p"
Oct 31 17:41:32.455656 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:32.455629    4276 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-hgtdx"
Oct 31 17:41:32.882602 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:32.882573    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-hgtdx"
Oct 31 17:41:32.883034 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:32.882639    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-g646p"
Oct 31 17:41:34.173094 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:34.173060    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:41:35.896112 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:35.896076    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:41:37.599611 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:37.599579    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:41:39.362096 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:39.362062    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:41:41.074258 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:41.074228    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:41:42.234403 ip-10-0-129-127 kubenswrapper[4276]: E1031 17:41:42.234367    4276 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tg5qk" containerName="extractor"
Oct 31 17:41:43.246074 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:43.246043    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:41:44.176318 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:44.176284    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:41:45.669738 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:45.669708    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:41:46.559850 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:46.559818    4276 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-bwdwd"]
Oct 31 17:41:46.564356 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:46.564333    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-bwdwd"
Oct 31 17:41:46.575557 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:46.574830    4276 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-8867c"]
Oct 31 17:41:46.579107 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:46.579085    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-8867c"
Oct 31 17:41:46.593689 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:46.593660    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-bwdwd"]
Oct 31 17:41:46.599983 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:46.599960    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-8867c"]
Oct 31 17:41:46.682099 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:46.682064    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d1bf33fd-97d5-462d-8c04-e2697fcf1b05-istio-token\") pod \"e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-8867c\" (UID: \"d1bf33fd-97d5-462d-8c04-e2697fcf1b05\") " pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-8867c"
Oct 31 17:41:46.682549 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:46.682170    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1bf33fd-97d5-462d-8c04-e2697fcf1b05-workload-socket\") pod \"e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-8867c\" (UID: \"d1bf33fd-97d5-462d-8c04-e2697fcf1b05\") " pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-8867c"
Oct 31 17:41:46.682549 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:46.682211    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7ec5066a-6b4d-4669-97f5-8a46a9c99a1b-istiod-ca-cert\") pod \"e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-bwdwd\" (UID: \"7ec5066a-6b4d-4669-97f5-8a46a9c99a1b\") " pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-bwdwd"
Oct 31 17:41:46.682549 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:46.682267    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d1bf33fd-97d5-462d-8c04-e2697fcf1b05-istio-data\") pod \"e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-8867c\" (UID: \"d1bf33fd-97d5-462d-8c04-e2697fcf1b05\") " pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-8867c"
Oct 31 17:41:46.682549 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:46.682290    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7ec5066a-6b4d-4669-97f5-8a46a9c99a1b-istio-podinfo\") pod \"e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-bwdwd\" (UID: \"7ec5066a-6b4d-4669-97f5-8a46a9c99a1b\") " pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-bwdwd"
Oct 31 17:41:46.682549 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:46.682323    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1bf33fd-97d5-462d-8c04-e2697fcf1b05-credential-socket\") pod \"e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-8867c\" (UID: \"d1bf33fd-97d5-462d-8c04-e2697fcf1b05\") " pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-8867c"
Oct 31 17:41:46.682549 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:46.682400    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ec5066a-6b4d-4669-97f5-8a46a9c99a1b-credential-socket\") pod \"e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-bwdwd\" (UID: \"7ec5066a-6b4d-4669-97f5-8a46a9c99a1b\") " pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-bwdwd"
Oct 31 17:41:46.682549 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:46.682424    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d1bf33fd-97d5-462d-8c04-e2697fcf1b05-istio-envoy\") pod \"e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-8867c\" (UID: \"d1bf33fd-97d5-462d-8c04-e2697fcf1b05\") " pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-8867c"
Oct 31 17:41:46.682549 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:46.682469    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7ec5066a-6b4d-4669-97f5-8a46a9c99a1b-istio-envoy\") pod \"e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-bwdwd\" (UID: \"7ec5066a-6b4d-4669-97f5-8a46a9c99a1b\") " pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-bwdwd"
Oct 31 17:41:46.682549 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:46.682528    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7ec5066a-6b4d-4669-97f5-8a46a9c99a1b-istio-data\") pod \"e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-bwdwd\" (UID: \"7ec5066a-6b4d-4669-97f5-8a46a9c99a1b\") " pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-bwdwd"
Oct 31 17:41:46.682938 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:46.682623    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d1bf33fd-97d5-462d-8c04-e2697fcf1b05-workload-certs\") pod \"e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-8867c\" (UID: \"d1bf33fd-97d5-462d-8c04-e2697fcf1b05\") " pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-8867c"
Oct 31 17:41:46.682938 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:46.682659    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-67pn5\" (UniqueName: \"kubernetes.io/projected/7ec5066a-6b4d-4669-97f5-8a46a9c99a1b-kube-api-access-67pn5\") pod \"e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-bwdwd\" (UID: \"7ec5066a-6b4d-4669-97f5-8a46a9c99a1b\") " pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-bwdwd"
Oct 31 17:41:46.682938 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:46.682720    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7ec5066a-6b4d-4669-97f5-8a46a9c99a1b-workload-certs\") pod \"e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-bwdwd\" (UID: \"7ec5066a-6b4d-4669-97f5-8a46a9c99a1b\") " pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-bwdwd"
Oct 31 17:41:46.682938 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:46.682789    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d1bf33fd-97d5-462d-8c04-e2697fcf1b05-istiod-ca-cert\") pod \"e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-8867c\" (UID: \"d1bf33fd-97d5-462d-8c04-e2697fcf1b05\") " pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-8867c"
Oct 31 17:41:46.682938 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:46.682888    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d1bf33fd-97d5-462d-8c04-e2697fcf1b05-istio-podinfo\") pod \"e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-8867c\" (UID: \"d1bf33fd-97d5-462d-8c04-e2697fcf1b05\") " pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-8867c"
Oct 31 17:41:46.682938 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:46.682934    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7ec5066a-6b4d-4669-97f5-8a46a9c99a1b-istio-token\") pod \"e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-bwdwd\" (UID: \"7ec5066a-6b4d-4669-97f5-8a46a9c99a1b\") " pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-bwdwd"
Oct 31 17:41:46.683157 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:46.682955    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ptjgq\" (UniqueName: \"kubernetes.io/projected/d1bf33fd-97d5-462d-8c04-e2697fcf1b05-kube-api-access-ptjgq\") pod \"e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-8867c\" (UID: \"d1bf33fd-97d5-462d-8c04-e2697fcf1b05\") " pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-8867c"
Oct 31 17:41:46.683157 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:46.683012    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ec5066a-6b4d-4669-97f5-8a46a9c99a1b-workload-socket\") pod \"e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-bwdwd\" (UID: \"7ec5066a-6b4d-4669-97f5-8a46a9c99a1b\") " pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-bwdwd"
Oct 31 17:41:46.783711 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:46.783681    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d1bf33fd-97d5-462d-8c04-e2697fcf1b05-istio-data\") pod \"e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-8867c\" (UID: \"d1bf33fd-97d5-462d-8c04-e2697fcf1b05\") " pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-8867c"
Oct 31 17:41:46.783875 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:46.783716    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7ec5066a-6b4d-4669-97f5-8a46a9c99a1b-istio-podinfo\") pod \"e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-bwdwd\" (UID: \"7ec5066a-6b4d-4669-97f5-8a46a9c99a1b\") " pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-bwdwd"
Oct 31 17:41:46.783875 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:46.783757    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1bf33fd-97d5-462d-8c04-e2697fcf1b05-credential-socket\") pod \"e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-8867c\" (UID: \"d1bf33fd-97d5-462d-8c04-e2697fcf1b05\") " pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-8867c"
Oct 31 17:41:46.783875 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:46.783795    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ec5066a-6b4d-4669-97f5-8a46a9c99a1b-credential-socket\") pod \"e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-bwdwd\" (UID: \"7ec5066a-6b4d-4669-97f5-8a46a9c99a1b\") " pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-bwdwd"
Oct 31 17:41:46.783875 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:46.783823    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d1bf33fd-97d5-462d-8c04-e2697fcf1b05-istio-envoy\") pod \"e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-8867c\" (UID: \"d1bf33fd-97d5-462d-8c04-e2697fcf1b05\") " pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-8867c"
Oct 31 17:41:46.783875 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:46.783849    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7ec5066a-6b4d-4669-97f5-8a46a9c99a1b-istio-envoy\") pod \"e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-bwdwd\" (UID: \"7ec5066a-6b4d-4669-97f5-8a46a9c99a1b\") " pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-bwdwd"
Oct 31 17:41:46.784143 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:46.783876    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7ec5066a-6b4d-4669-97f5-8a46a9c99a1b-istio-data\") pod \"e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-bwdwd\" (UID: \"7ec5066a-6b4d-4669-97f5-8a46a9c99a1b\") " pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-bwdwd"
Oct 31 17:41:46.784143 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:46.783970    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d1bf33fd-97d5-462d-8c04-e2697fcf1b05-workload-certs\") pod \"e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-8867c\" (UID: \"d1bf33fd-97d5-462d-8c04-e2697fcf1b05\") " pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-8867c"
Oct 31 17:41:46.784143 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:46.784012    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-67pn5\" (UniqueName: \"kubernetes.io/projected/7ec5066a-6b4d-4669-97f5-8a46a9c99a1b-kube-api-access-67pn5\") pod \"e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-bwdwd\" (UID: \"7ec5066a-6b4d-4669-97f5-8a46a9c99a1b\") " pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-bwdwd"
Oct 31 17:41:46.784143 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:46.784060    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7ec5066a-6b4d-4669-97f5-8a46a9c99a1b-workload-certs\") pod \"e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-bwdwd\" (UID: \"7ec5066a-6b4d-4669-97f5-8a46a9c99a1b\") " pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-bwdwd"
Oct 31 17:41:46.784143 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:46.784105    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d1bf33fd-97d5-462d-8c04-e2697fcf1b05-istiod-ca-cert\") pod \"e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-8867c\" (UID: \"d1bf33fd-97d5-462d-8c04-e2697fcf1b05\") " pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-8867c"
Oct 31 17:41:46.784143 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:46.784124    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d1bf33fd-97d5-462d-8c04-e2697fcf1b05-istio-data\") pod \"e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-8867c\" (UID: \"d1bf33fd-97d5-462d-8c04-e2697fcf1b05\") " pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-8867c"
Oct 31 17:41:46.784476 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:46.784177    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1bf33fd-97d5-462d-8c04-e2697fcf1b05-credential-socket\") pod \"e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-8867c\" (UID: \"d1bf33fd-97d5-462d-8c04-e2697fcf1b05\") " pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-8867c"
Oct 31 17:41:46.784476 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:46.784195    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ec5066a-6b4d-4669-97f5-8a46a9c99a1b-credential-socket\") pod \"e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-bwdwd\" (UID: \"7ec5066a-6b4d-4669-97f5-8a46a9c99a1b\") " pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-bwdwd"
Oct 31 17:41:46.784476 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:46.784285    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d1bf33fd-97d5-462d-8c04-e2697fcf1b05-istio-podinfo\") pod \"e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-8867c\" (UID: \"d1bf33fd-97d5-462d-8c04-e2697fcf1b05\") " pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-8867c"
Oct 31 17:41:46.784476 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:46.784315    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7ec5066a-6b4d-4669-97f5-8a46a9c99a1b-istio-token\") pod \"e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-bwdwd\" (UID: \"7ec5066a-6b4d-4669-97f5-8a46a9c99a1b\") " pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-bwdwd"
Oct 31 17:41:46.784476 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:46.784344    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ptjgq\" (UniqueName: \"kubernetes.io/projected/d1bf33fd-97d5-462d-8c04-e2697fcf1b05-kube-api-access-ptjgq\") pod \"e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-8867c\" (UID: \"d1bf33fd-97d5-462d-8c04-e2697fcf1b05\") " pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-8867c"
Oct 31 17:41:46.784476 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:46.784367    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ec5066a-6b4d-4669-97f5-8a46a9c99a1b-workload-socket\") pod \"e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-bwdwd\" (UID: \"7ec5066a-6b4d-4669-97f5-8a46a9c99a1b\") " pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-bwdwd"
Oct 31 17:41:46.784476 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:46.784428    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d1bf33fd-97d5-462d-8c04-e2697fcf1b05-istio-token\") pod \"e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-8867c\" (UID: \"d1bf33fd-97d5-462d-8c04-e2697fcf1b05\") " pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-8867c"
Oct 31 17:41:46.784864 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:46.784494    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d1bf33fd-97d5-462d-8c04-e2697fcf1b05-workload-certs\") pod \"e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-8867c\" (UID: \"d1bf33fd-97d5-462d-8c04-e2697fcf1b05\") " pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-8867c"
Oct 31 17:41:46.784864 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:46.784498    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7ec5066a-6b4d-4669-97f5-8a46a9c99a1b-istio-data\") pod \"e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-bwdwd\" (UID: \"7ec5066a-6b4d-4669-97f5-8a46a9c99a1b\") " pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-bwdwd"
Oct 31 17:41:46.784864 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:46.784539    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1bf33fd-97d5-462d-8c04-e2697fcf1b05-workload-socket\") pod \"e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-8867c\" (UID: \"d1bf33fd-97d5-462d-8c04-e2697fcf1b05\") " pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-8867c"
Oct 31 17:41:46.784864 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:46.784568    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7ec5066a-6b4d-4669-97f5-8a46a9c99a1b-istiod-ca-cert\") pod \"e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-bwdwd\" (UID: \"7ec5066a-6b4d-4669-97f5-8a46a9c99a1b\") " pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-bwdwd"
Oct 31 17:41:46.784864 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:46.784774    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ec5066a-6b4d-4669-97f5-8a46a9c99a1b-workload-socket\") pod \"e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-bwdwd\" (UID: \"7ec5066a-6b4d-4669-97f5-8a46a9c99a1b\") " pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-bwdwd"
Oct 31 17:41:46.785130 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:46.784865    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d1bf33fd-97d5-462d-8c04-e2697fcf1b05-istiod-ca-cert\") pod \"e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-8867c\" (UID: \"d1bf33fd-97d5-462d-8c04-e2697fcf1b05\") " pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-8867c"
Oct 31 17:41:46.785130 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:46.784945    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7ec5066a-6b4d-4669-97f5-8a46a9c99a1b-workload-certs\") pod \"e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-bwdwd\" (UID: \"7ec5066a-6b4d-4669-97f5-8a46a9c99a1b\") " pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-bwdwd"
Oct 31 17:41:46.785130 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:46.785112    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1bf33fd-97d5-462d-8c04-e2697fcf1b05-workload-socket\") pod \"e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-8867c\" (UID: \"d1bf33fd-97d5-462d-8c04-e2697fcf1b05\") " pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-8867c"
Oct 31 17:41:46.785414 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:46.785391    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7ec5066a-6b4d-4669-97f5-8a46a9c99a1b-istiod-ca-cert\") pod \"e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-bwdwd\" (UID: \"7ec5066a-6b4d-4669-97f5-8a46a9c99a1b\") " pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-bwdwd"
Oct 31 17:41:46.786248 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:46.786229    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7ec5066a-6b4d-4669-97f5-8a46a9c99a1b-istio-envoy\") pod \"e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-bwdwd\" (UID: \"7ec5066a-6b4d-4669-97f5-8a46a9c99a1b\") " pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-bwdwd"
Oct 31 17:41:46.786711 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:46.786685    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7ec5066a-6b4d-4669-97f5-8a46a9c99a1b-istio-podinfo\") pod \"e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-bwdwd\" (UID: \"7ec5066a-6b4d-4669-97f5-8a46a9c99a1b\") " pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-bwdwd"
Oct 31 17:41:46.786826 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:46.786809    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d1bf33fd-97d5-462d-8c04-e2697fcf1b05-istio-envoy\") pod \"e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-8867c\" (UID: \"d1bf33fd-97d5-462d-8c04-e2697fcf1b05\") " pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-8867c"
Oct 31 17:41:46.787293 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:46.787273    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d1bf33fd-97d5-462d-8c04-e2697fcf1b05-istio-podinfo\") pod \"e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-8867c\" (UID: \"d1bf33fd-97d5-462d-8c04-e2697fcf1b05\") " pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-8867c"
Oct 31 17:41:46.808361 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:46.808332    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7ec5066a-6b4d-4669-97f5-8a46a9c99a1b-istio-token\") pod \"e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-bwdwd\" (UID: \"7ec5066a-6b4d-4669-97f5-8a46a9c99a1b\") " pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-bwdwd"
Oct 31 17:41:46.808486 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:46.808370    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-67pn5\" (UniqueName: \"kubernetes.io/projected/7ec5066a-6b4d-4669-97f5-8a46a9c99a1b-kube-api-access-67pn5\") pod \"e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-bwdwd\" (UID: \"7ec5066a-6b4d-4669-97f5-8a46a9c99a1b\") " pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-bwdwd"
Oct 31 17:41:46.808559 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:46.808523    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d1bf33fd-97d5-462d-8c04-e2697fcf1b05-istio-token\") pod \"e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-8867c\" (UID: \"d1bf33fd-97d5-462d-8c04-e2697fcf1b05\") " pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-8867c"
Oct 31 17:41:46.808909 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:46.808887    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ptjgq\" (UniqueName: \"kubernetes.io/projected/d1bf33fd-97d5-462d-8c04-e2697fcf1b05-kube-api-access-ptjgq\") pod \"e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-8867c\" (UID: \"d1bf33fd-97d5-462d-8c04-e2697fcf1b05\") " pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-8867c"
Oct 31 17:41:46.831616 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:46.831555    4276 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-8bf7k"]
Oct 31 17:41:46.836929 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:46.836912    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-8bf7k"
Oct 31 17:41:46.845047 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:46.845024    4276 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-m7vsj"]
Oct 31 17:41:46.849618 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:46.849593    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-m7vsj"
Oct 31 17:41:46.861359 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:46.861336    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-8bf7k"]
Oct 31 17:41:46.894700 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:46.894672    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-m7vsj"]
Oct 31 17:41:46.895041 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:46.894993    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-bwdwd"
Oct 31 17:41:46.904648 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:46.904626    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-8867c"
Oct 31 17:41:46.999303 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:46.998556    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/28cbdb7c-1bc2-49b4-834c-de871be33524-istio-data\") pod \"e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-8bf7k\" (UID: \"28cbdb7c-1bc2-49b4-834c-de871be33524\") " pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-8bf7k"
Oct 31 17:41:46.999303 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:46.998613    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f675ff71-27f9-44e8-aa32-89f4332d4c3e-istio-podinfo\") pod \"e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-m7vsj\" (UID: \"f675ff71-27f9-44e8-aa32-89f4332d4c3e\") " pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-m7vsj"
Oct 31 17:41:46.999303 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:46.998675    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/28cbdb7c-1bc2-49b4-834c-de871be33524-istio-envoy\") pod \"e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-8bf7k\" (UID: \"28cbdb7c-1bc2-49b4-834c-de871be33524\") " pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-8bf7k"
Oct 31 17:41:46.999303 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:46.998703    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/28cbdb7c-1bc2-49b4-834c-de871be33524-workload-socket\") pod \"e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-8bf7k\" (UID: \"28cbdb7c-1bc2-49b4-834c-de871be33524\") " pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-8bf7k"
Oct 31 17:41:46.999303 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:46.998735    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/28cbdb7c-1bc2-49b4-834c-de871be33524-istio-podinfo\") pod \"e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-8bf7k\" (UID: \"28cbdb7c-1bc2-49b4-834c-de871be33524\") " pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-8bf7k"
Oct 31 17:41:46.999303 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:46.998782    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f675ff71-27f9-44e8-aa32-89f4332d4c3e-istio-envoy\") pod \"e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-m7vsj\" (UID: \"f675ff71-27f9-44e8-aa32-89f4332d4c3e\") " pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-m7vsj"
Oct 31 17:41:46.999303 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:46.998817    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-schh7\" (UniqueName: \"kubernetes.io/projected/28cbdb7c-1bc2-49b4-834c-de871be33524-kube-api-access-schh7\") pod \"e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-8bf7k\" (UID: \"28cbdb7c-1bc2-49b4-834c-de871be33524\") " pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-8bf7k"
Oct 31 17:41:46.999303 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:46.998858    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/28cbdb7c-1bc2-49b4-834c-de871be33524-workload-certs\") pod \"e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-8bf7k\" (UID: \"28cbdb7c-1bc2-49b4-834c-de871be33524\") " pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-8bf7k"
Oct 31 17:41:46.999303 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:46.998953    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f675ff71-27f9-44e8-aa32-89f4332d4c3e-istio-token\") pod \"e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-m7vsj\" (UID: \"f675ff71-27f9-44e8-aa32-89f4332d4c3e\") " pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-m7vsj"
Oct 31 17:41:46.999303 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:46.998978    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/28cbdb7c-1bc2-49b4-834c-de871be33524-istio-token\") pod \"e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-8bf7k\" (UID: \"28cbdb7c-1bc2-49b4-834c-de871be33524\") " pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-8bf7k"
Oct 31 17:41:46.999303 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:46.999012    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f675ff71-27f9-44e8-aa32-89f4332d4c3e-istiod-ca-cert\") pod \"e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-m7vsj\" (UID: \"f675ff71-27f9-44e8-aa32-89f4332d4c3e\") " pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-m7vsj"
Oct 31 17:41:46.999303 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:46.999051    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7hrd5\" (UniqueName: \"kubernetes.io/projected/f675ff71-27f9-44e8-aa32-89f4332d4c3e-kube-api-access-7hrd5\") pod \"e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-m7vsj\" (UID: \"f675ff71-27f9-44e8-aa32-89f4332d4c3e\") " pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-m7vsj"
Oct 31 17:41:47.001219 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:46.999094    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f675ff71-27f9-44e8-aa32-89f4332d4c3e-workload-certs\") pod \"e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-m7vsj\" (UID: \"f675ff71-27f9-44e8-aa32-89f4332d4c3e\") " pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-m7vsj"
Oct 31 17:41:47.001219 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:47.000815    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f675ff71-27f9-44e8-aa32-89f4332d4c3e-credential-socket\") pod \"e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-m7vsj\" (UID: \"f675ff71-27f9-44e8-aa32-89f4332d4c3e\") " pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-m7vsj"
Oct 31 17:41:47.001219 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:47.000849    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f675ff71-27f9-44e8-aa32-89f4332d4c3e-istio-data\") pod \"e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-m7vsj\" (UID: \"f675ff71-27f9-44e8-aa32-89f4332d4c3e\") " pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-m7vsj"
Oct 31 17:41:47.001219 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:47.000905    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/28cbdb7c-1bc2-49b4-834c-de871be33524-credential-socket\") pod \"e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-8bf7k\" (UID: \"28cbdb7c-1bc2-49b4-834c-de871be33524\") " pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-8bf7k"
Oct 31 17:41:47.001219 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:47.000983    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/28cbdb7c-1bc2-49b4-834c-de871be33524-istiod-ca-cert\") pod \"e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-8bf7k\" (UID: \"28cbdb7c-1bc2-49b4-834c-de871be33524\") " pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-8bf7k"
Oct 31 17:41:47.001219 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:47.001055    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f675ff71-27f9-44e8-aa32-89f4332d4c3e-workload-socket\") pod \"e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-m7vsj\" (UID: \"f675ff71-27f9-44e8-aa32-89f4332d4c3e\") " pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-m7vsj"
Oct 31 17:41:47.102613 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:47.102527    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/28cbdb7c-1bc2-49b4-834c-de871be33524-credential-socket\") pod \"e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-8bf7k\" (UID: \"28cbdb7c-1bc2-49b4-834c-de871be33524\") " pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-8bf7k"
Oct 31 17:41:47.102769 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:47.102629    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/28cbdb7c-1bc2-49b4-834c-de871be33524-istiod-ca-cert\") pod \"e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-8bf7k\" (UID: \"28cbdb7c-1bc2-49b4-834c-de871be33524\") " pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-8bf7k"
Oct 31 17:41:47.102769 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:47.102705    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f675ff71-27f9-44e8-aa32-89f4332d4c3e-workload-socket\") pod \"e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-m7vsj\" (UID: \"f675ff71-27f9-44e8-aa32-89f4332d4c3e\") " pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-m7vsj"
Oct 31 17:41:47.103071 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:47.102767    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/28cbdb7c-1bc2-49b4-834c-de871be33524-istio-data\") pod \"e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-8bf7k\" (UID: \"28cbdb7c-1bc2-49b4-834c-de871be33524\") " pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-8bf7k"
Oct 31 17:41:47.103071 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:47.102799    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f675ff71-27f9-44e8-aa32-89f4332d4c3e-istio-podinfo\") pod \"e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-m7vsj\" (UID: \"f675ff71-27f9-44e8-aa32-89f4332d4c3e\") " pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-m7vsj"
Oct 31 17:41:47.103071 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:47.102861    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/28cbdb7c-1bc2-49b4-834c-de871be33524-istio-envoy\") pod \"e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-8bf7k\" (UID: \"28cbdb7c-1bc2-49b4-834c-de871be33524\") " pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-8bf7k"
Oct 31 17:41:47.103071 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:47.102888    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/28cbdb7c-1bc2-49b4-834c-de871be33524-workload-socket\") pod \"e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-8bf7k\" (UID: \"28cbdb7c-1bc2-49b4-834c-de871be33524\") " pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-8bf7k"
Oct 31 17:41:47.103071 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:47.102935    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/28cbdb7c-1bc2-49b4-834c-de871be33524-istio-podinfo\") pod \"e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-8bf7k\" (UID: \"28cbdb7c-1bc2-49b4-834c-de871be33524\") " pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-8bf7k"
Oct 31 17:41:47.103071 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:47.102979    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f675ff71-27f9-44e8-aa32-89f4332d4c3e-istio-envoy\") pod \"e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-m7vsj\" (UID: \"f675ff71-27f9-44e8-aa32-89f4332d4c3e\") " pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-m7vsj"
Oct 31 17:41:47.103071 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:47.103034    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/28cbdb7c-1bc2-49b4-834c-de871be33524-credential-socket\") pod \"e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-8bf7k\" (UID: \"28cbdb7c-1bc2-49b4-834c-de871be33524\") " pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-8bf7k"
Oct 31 17:41:47.103428 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:47.103090    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-schh7\" (UniqueName: \"kubernetes.io/projected/28cbdb7c-1bc2-49b4-834c-de871be33524-kube-api-access-schh7\") pod \"e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-8bf7k\" (UID: \"28cbdb7c-1bc2-49b4-834c-de871be33524\") " pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-8bf7k"
Oct 31 17:41:47.103428 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:47.103134    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/28cbdb7c-1bc2-49b4-834c-de871be33524-workload-certs\") pod \"e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-8bf7k\" (UID: \"28cbdb7c-1bc2-49b4-834c-de871be33524\") " pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-8bf7k"
Oct 31 17:41:47.103428 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:47.103226    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f675ff71-27f9-44e8-aa32-89f4332d4c3e-istio-token\") pod \"e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-m7vsj\" (UID: \"f675ff71-27f9-44e8-aa32-89f4332d4c3e\") " pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-m7vsj"
Oct 31 17:41:47.103428 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:47.103253    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/28cbdb7c-1bc2-49b4-834c-de871be33524-istio-token\") pod \"e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-8bf7k\" (UID: \"28cbdb7c-1bc2-49b4-834c-de871be33524\") " pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-8bf7k"
Oct 31 17:41:47.103428 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:47.103287    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f675ff71-27f9-44e8-aa32-89f4332d4c3e-istiod-ca-cert\") pod \"e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-m7vsj\" (UID: \"f675ff71-27f9-44e8-aa32-89f4332d4c3e\") " pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-m7vsj"
Oct 31 17:41:47.103428 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:47.103303    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/28cbdb7c-1bc2-49b4-834c-de871be33524-workload-socket\") pod \"e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-8bf7k\" (UID: \"28cbdb7c-1bc2-49b4-834c-de871be33524\") " pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-8bf7k"
Oct 31 17:41:47.103428 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:47.103328    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7hrd5\" (UniqueName: \"kubernetes.io/projected/f675ff71-27f9-44e8-aa32-89f4332d4c3e-kube-api-access-7hrd5\") pod \"e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-m7vsj\" (UID: \"f675ff71-27f9-44e8-aa32-89f4332d4c3e\") " pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-m7vsj"
Oct 31 17:41:47.103428 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:47.103329    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f675ff71-27f9-44e8-aa32-89f4332d4c3e-workload-socket\") pod \"e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-m7vsj\" (UID: \"f675ff71-27f9-44e8-aa32-89f4332d4c3e\") " pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-m7vsj"
Oct 31 17:41:47.103428 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:47.103373    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f675ff71-27f9-44e8-aa32-89f4332d4c3e-workload-certs\") pod \"e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-m7vsj\" (UID: \"f675ff71-27f9-44e8-aa32-89f4332d4c3e\") " pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-m7vsj"
Oct 31 17:41:47.103906 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:47.103494    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f675ff71-27f9-44e8-aa32-89f4332d4c3e-credential-socket\") pod \"e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-m7vsj\" (UID: \"f675ff71-27f9-44e8-aa32-89f4332d4c3e\") " pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-m7vsj"
Oct 31 17:41:47.103906 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:47.103513    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/28cbdb7c-1bc2-49b4-834c-de871be33524-istio-data\") pod \"e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-8bf7k\" (UID: \"28cbdb7c-1bc2-49b4-834c-de871be33524\") " pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-8bf7k"
Oct 31 17:41:47.103906 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:47.103520    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f675ff71-27f9-44e8-aa32-89f4332d4c3e-istio-data\") pod \"e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-m7vsj\" (UID: \"f675ff71-27f9-44e8-aa32-89f4332d4c3e\") " pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-m7vsj"
Oct 31 17:41:47.103906 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:47.103777    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/28cbdb7c-1bc2-49b4-834c-de871be33524-workload-certs\") pod \"e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-8bf7k\" (UID: \"28cbdb7c-1bc2-49b4-834c-de871be33524\") " pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-8bf7k"
Oct 31 17:41:47.103906 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:47.103783    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f675ff71-27f9-44e8-aa32-89f4332d4c3e-istio-data\") pod \"e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-m7vsj\" (UID: \"f675ff71-27f9-44e8-aa32-89f4332d4c3e\") " pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-m7vsj"
Oct 31 17:41:47.106053 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:47.104002    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f675ff71-27f9-44e8-aa32-89f4332d4c3e-workload-certs\") pod \"e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-m7vsj\" (UID: \"f675ff71-27f9-44e8-aa32-89f4332d4c3e\") " pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-m7vsj"
Oct 31 17:41:47.106053 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:47.104056    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f675ff71-27f9-44e8-aa32-89f4332d4c3e-credential-socket\") pod \"e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-m7vsj\" (UID: \"f675ff71-27f9-44e8-aa32-89f4332d4c3e\") " pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-m7vsj"
Oct 31 17:41:47.106053 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:47.104825    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f675ff71-27f9-44e8-aa32-89f4332d4c3e-istiod-ca-cert\") pod \"e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-m7vsj\" (UID: \"f675ff71-27f9-44e8-aa32-89f4332d4c3e\") " pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-m7vsj"
Oct 31 17:41:47.106053 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:47.104847    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/28cbdb7c-1bc2-49b4-834c-de871be33524-istiod-ca-cert\") pod \"e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-8bf7k\" (UID: \"28cbdb7c-1bc2-49b4-834c-de871be33524\") " pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-8bf7k"
Oct 31 17:41:47.106748 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:47.106724    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f675ff71-27f9-44e8-aa32-89f4332d4c3e-istio-envoy\") pod \"e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-m7vsj\" (UID: \"f675ff71-27f9-44e8-aa32-89f4332d4c3e\") " pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-m7vsj"
Oct 31 17:41:47.107212 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:47.107183    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/28cbdb7c-1bc2-49b4-834c-de871be33524-istio-envoy\") pod \"e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-8bf7k\" (UID: \"28cbdb7c-1bc2-49b4-834c-de871be33524\") " pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-8bf7k"
Oct 31 17:41:47.107438 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:47.107415    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f675ff71-27f9-44e8-aa32-89f4332d4c3e-istio-podinfo\") pod \"e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-m7vsj\" (UID: \"f675ff71-27f9-44e8-aa32-89f4332d4c3e\") " pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-m7vsj"
Oct 31 17:41:47.108484 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:47.108434    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/28cbdb7c-1bc2-49b4-834c-de871be33524-istio-podinfo\") pod \"e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-8bf7k\" (UID: \"28cbdb7c-1bc2-49b4-834c-de871be33524\") " pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-8bf7k"
Oct 31 17:41:47.123592 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:47.123564    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-schh7\" (UniqueName: \"kubernetes.io/projected/28cbdb7c-1bc2-49b4-834c-de871be33524-kube-api-access-schh7\") pod \"e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-8bf7k\" (UID: \"28cbdb7c-1bc2-49b4-834c-de871be33524\") " pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-8bf7k"
Oct 31 17:41:47.123673 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:47.123621    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7hrd5\" (UniqueName: \"kubernetes.io/projected/f675ff71-27f9-44e8-aa32-89f4332d4c3e-kube-api-access-7hrd5\") pod \"e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-m7vsj\" (UID: \"f675ff71-27f9-44e8-aa32-89f4332d4c3e\") " pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-m7vsj"
Oct 31 17:41:47.123768 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:47.123753    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/28cbdb7c-1bc2-49b4-834c-de871be33524-istio-token\") pod \"e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-8bf7k\" (UID: \"28cbdb7c-1bc2-49b4-834c-de871be33524\") " pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-8bf7k"
Oct 31 17:41:47.123807 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:47.123758    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f675ff71-27f9-44e8-aa32-89f4332d4c3e-istio-token\") pod \"e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-m7vsj\" (UID: \"f675ff71-27f9-44e8-aa32-89f4332d4c3e\") " pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-m7vsj"
Oct 31 17:41:47.165122 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:47.165104    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-8bf7k"
Oct 31 17:41:47.177506 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:47.177486    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-m7vsj"
Oct 31 17:41:47.246881 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:47.246838    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-bwdwd"]
Oct 31 17:41:47.267558 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:47.267523    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-8867c"]
Oct 31 17:41:47.270490 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:47.269471    4276 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 17:41:47.270490 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:47.269555    4276 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 17:41:47.270490 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:47.269601    4276 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 17:41:47.273889 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:41:47.273863    4276 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1bf33fd_97d5_462d_8c04_e2697fcf1b05.slice/crio-4784fbfdff6c2ca6d17de2bd266c1c07bfcbeca7735868e90348cccc4d0b76fd WatchSource:0}: Error finding container 4784fbfdff6c2ca6d17de2bd266c1c07bfcbeca7735868e90348cccc4d0b76fd: Status 404 returned error can't find the container with id 4784fbfdff6c2ca6d17de2bd266c1c07bfcbeca7735868e90348cccc4d0b76fd
Oct 31 17:41:47.286369 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:47.286327    4276 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 17:41:47.286520 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:47.286422    4276 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 17:41:47.286520 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:47.286492    4276 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 17:41:47.311513 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:47.311229    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:41:47.512268 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:47.512242    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-8bf7k"]
Oct 31 17:41:47.524639 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:47.524610    4276 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 17:41:47.524800 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:47.524785    4276 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 17:41:47.524893 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:47.524881    4276 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 17:41:47.544501 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:47.544466    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-m7vsj"]
Oct 31 17:41:47.545167 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:41:47.545140    4276 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf675ff71_27f9_44e8_aa32_89f4332d4c3e.slice/crio-a65b177789025d69ab7307359f5a74e5bb514e1f3db3d8ebe1d3ead08b2a7d94 WatchSource:0}: Error finding container a65b177789025d69ab7307359f5a74e5bb514e1f3db3d8ebe1d3ead08b2a7d94: Status 404 returned error can't find the container with id a65b177789025d69ab7307359f5a74e5bb514e1f3db3d8ebe1d3ead08b2a7d94
Oct 31 17:41:47.547651 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:47.547620    4276 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 17:41:47.547755 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:47.547684    4276 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 17:41:47.547755 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:47.547712    4276 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 17:41:47.978379 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:47.978292    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-8bf7k" event={"ID":"28cbdb7c-1bc2-49b4-834c-de871be33524","Type":"ContainerStarted","Data":"1b62347ed05ecd31efa2e9cb7a621deeb263f006afe6411c319be80c5cd86a75"}
Oct 31 17:41:47.978379 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:47.978361    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-8bf7k" event={"ID":"28cbdb7c-1bc2-49b4-834c-de871be33524","Type":"ContainerStarted","Data":"1d7beaa120a8808d12c28e8f6a2848000f03d35d7863fae917d05aa6a96bca72"}
Oct 31 17:41:47.980617 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:47.980586    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-bwdwd" event={"ID":"7ec5066a-6b4d-4669-97f5-8a46a9c99a1b","Type":"ContainerStarted","Data":"a8e133dd63fb045cd89dea6db2b65aef167a4a19b83a9e11d66fbf8406af7271"}
Oct 31 17:41:47.980764 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:47.980648    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-bwdwd" event={"ID":"7ec5066a-6b4d-4669-97f5-8a46a9c99a1b","Type":"ContainerStarted","Data":"00c7fa987e205d9088d626145a761b81be37b140acfcdf621ac325d52b90e060"}
Oct 31 17:41:47.982671 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:47.982645    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-8867c" event={"ID":"d1bf33fd-97d5-462d-8c04-e2697fcf1b05","Type":"ContainerStarted","Data":"9224bcb42e7be2c812e6db5e1f87d5b65af67a6e6c1cea7735c44f82d264ca7b"}
Oct 31 17:41:47.982787 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:47.982693    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-8867c" event={"ID":"d1bf33fd-97d5-462d-8c04-e2697fcf1b05","Type":"ContainerStarted","Data":"4784fbfdff6c2ca6d17de2bd266c1c07bfcbeca7735868e90348cccc4d0b76fd"}
Oct 31 17:41:47.984768 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:47.984742    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-m7vsj" event={"ID":"f675ff71-27f9-44e8-aa32-89f4332d4c3e","Type":"ContainerStarted","Data":"1019d46050d0bb7e6f4da0f693b5f99640f2050d06769321e7a4b15b04524aa5"}
Oct 31 17:41:47.984875 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:47.984793    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-m7vsj" event={"ID":"f675ff71-27f9-44e8-aa32-89f4332d4c3e","Type":"ContainerStarted","Data":"a65b177789025d69ab7307359f5a74e5bb514e1f3db3d8ebe1d3ead08b2a7d94"}
Oct 31 17:41:48.028683 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:48.028630    4276 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-8bf7k" podStartSLOduration=2.028615048 podStartE2EDuration="2.028615048s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 17:41:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 17:41:48.02456505 +0000 UTC m=+870.822309983" watchObservedRunningTime="2026-10-31 17:41:48.028615048 +0000 UTC m=+870.826359976"
Oct 31 17:41:48.071424 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:48.071370    4276 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-m7vsj" podStartSLOduration=2.071351935 podStartE2EDuration="2.071351935s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 17:41:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 17:41:48.06770155 +0000 UTC m=+870.865446481" watchObservedRunningTime="2026-10-31 17:41:48.071351935 +0000 UTC m=+870.869096864"
Oct 31 17:41:48.113026 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:48.112958    4276 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-8867c" podStartSLOduration=2.112938056 podStartE2EDuration="2.112938056s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 17:41:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 17:41:48.109230957 +0000 UTC m=+870.906975889" watchObservedRunningTime="2026-10-31 17:41:48.112938056 +0000 UTC m=+870.910682985"
Oct 31 17:41:48.153124 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:48.153062    4276 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-bwdwd" podStartSLOduration=2.153044244 podStartE2EDuration="2.153044244s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 17:41:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 17:41:48.152073664 +0000 UTC m=+870.949818598" watchObservedRunningTime="2026-10-31 17:41:48.153044244 +0000 UTC m=+870.950789172"
Oct 31 17:41:48.165296 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:48.165261    4276 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-8bf7k"
Oct 31 17:41:48.167189 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:48.167155    4276 patch_prober.go:29] interesting pod/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-8bf7k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused" start-of-body=
Oct 31 17:41:48.167338 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:48.167218    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-8bf7k" podUID="28cbdb7c-1bc2-49b4-834c-de871be33524" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused"
Oct 31 17:41:48.178002 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:48.177967    4276 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-m7vsj"
Oct 31 17:41:48.185979 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:48.185955    4276 patch_prober.go:29] interesting pod/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-m7vsj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" start-of-body=
Oct 31 17:41:48.186475 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:48.186392    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-m7vsj" podUID="f675ff71-27f9-44e8-aa32-89f4332d4c3e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused"
Oct 31 17:41:48.898466 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:48.897858    4276 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-bwdwd"
Oct 31 17:41:48.904511 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:48.903608    4276 patch_prober.go:29] interesting pod/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-bwdwd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" start-of-body=
Oct 31 17:41:48.904511 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:48.903685    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-bwdwd" podUID="7ec5066a-6b4d-4669-97f5-8a46a9c99a1b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused"
Oct 31 17:41:48.917467 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:48.916324    4276 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-8867c"
Oct 31 17:41:48.917467 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:48.916519    4276 patch_prober.go:29] interesting pod/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-8867c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body=
Oct 31 17:41:48.917467 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:48.916563    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-8867c" podUID="d1bf33fd-97d5-462d-8c04-e2697fcf1b05" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused"
Oct 31 17:41:49.197524 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:49.176178    4276 patch_prober.go:29] interesting pod/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-8bf7k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused" start-of-body=
Oct 31 17:41:49.197524 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:49.176235    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-8bf7k" podUID="28cbdb7c-1bc2-49b4-834c-de871be33524" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused"
Oct 31 17:41:49.197524 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:49.180834    4276 patch_prober.go:29] interesting pod/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-m7vsj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" start-of-body=
Oct 31 17:41:49.197524 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:49.180881    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-m7vsj" podUID="f675ff71-27f9-44e8-aa32-89f4332d4c3e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused"
Oct 31 17:41:49.290631 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:49.290529    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:41:49.569474 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:49.568546    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ee816e/premium-w0-ee816e-kserve-65c86cd6db-lnt9p" podUID="e536c902-52a9-4b36-a2ad-eb5500781a5e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.49:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 17:41:49.885552 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:49.879555    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-d80080/distinct-w6-d80080-kserve-664576c8f5-6jc4d" podUID="b9c2f445-a191-4b00-802a-1b0ded068411" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 31 17:41:49.897002 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:49.896970    4276 patch_prober.go:29] interesting pod/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-bwdwd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" start-of-body=
Oct 31 17:41:49.897262 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:49.897232    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-bwdwd" podUID="7ec5066a-6b4d-4669-97f5-8a46a9c99a1b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused"
Oct 31 17:41:49.916561 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:49.916533    4276 patch_prober.go:29] interesting pod/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-8867c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body=
Oct 31 17:41:49.916760 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:49.916733    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-8867c" podUID="d1bf33fd-97d5-462d-8c04-e2697fcf1b05" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused"
Oct 31 17:41:50.292303 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:50.249894    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-1c0bfd/distinct-2-w4-1c0bfd-kserve-847bcf6cf4-lt8ls" podUID="f8a45b7c-e5ae-4321-ad81-fd6353b2495d" containerName="main" probeResult="failure" output="Get \"https://10.133.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 17:41:50.292303 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:50.276031    4276 patch_prober.go:29] interesting pod/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-8bf7k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused" start-of-body=
Oct 31 17:41:50.292303 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:50.276085    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-8bf7k" podUID="28cbdb7c-1bc2-49b4-834c-de871be33524" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused"
Oct 31 17:41:50.341481 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:50.340578    4276 patch_prober.go:29] interesting pod/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-m7vsj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" start-of-body=
Oct 31 17:41:50.341481 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:50.340638    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-m7vsj" podUID="f675ff71-27f9-44e8-aa32-89f4332d4c3e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused"
Oct 31 17:41:50.522469 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:50.484356    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:41:50.522469 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:50.504618    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-5c446c/distinct-w2-5c446c-kserve-69d97d6968-gtvjv" podUID="2d1185cc-bac1-45ca-a250-8f44f8589162" containerName="main" probeResult="failure" output="Get \"https://10.133.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 17:41:50.522469 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:50.505266    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-0e59ca/premium-w1-0e59ca-kserve-58765f9746-fqwq7" podUID="8cf1e52e-70f8-42e2-bdf3-a315e17f10f5" containerName="main" probeResult="failure" output="Get \"https://10.133.0.48:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 17:41:50.573465 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:50.570528    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ee816e/premium-w0-ee816e-kserve-65c86cd6db-lnt9p" podUID="e536c902-52a9-4b36-a2ad-eb5500781a5e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.49:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 17:41:50.573465 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:50.572283    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-ee816e/premium-w0-ee816e-kserve-65c86cd6db-lnt9p"
Oct 31 17:41:50.706467 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:50.703750    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-5c446c/distinct-2-w2-5c446c-kserve-5d9cd79c47-p4j5l" podUID="4c288908-92e3-4c45-8ba5-2f86c9f38b3d" containerName="main" probeResult="failure" output="Get \"https://10.133.0.65:8000/ready\": context deadline exceeded"
Oct 31 17:41:50.977517 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:50.976551    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-d80080/distinct-w6-d80080-kserve-664576c8f5-6jc4d" podUID="b9c2f445-a191-4b00-802a-1b0ded068411" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 17:41:51.031487 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:51.002978    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-d80080/distinct-w6-d80080-kserve-664576c8f5-6jc4d"
Oct 31 17:41:51.130506 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:51.049572    4276 patch_prober.go:29] interesting pod/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-8867c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body=
Oct 31 17:41:51.130506 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:51.049648    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-8867c" podUID="d1bf33fd-97d5-462d-8c04-e2697fcf1b05" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused"
Oct 31 17:41:51.185466 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:51.160027    4276 patch_prober.go:29] interesting pod/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-bwdwd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" start-of-body=
Oct 31 17:41:51.221557 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:51.221517    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-bwdwd" podUID="7ec5066a-6b4d-4669-97f5-8a46a9c99a1b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused"
Oct 31 17:41:51.227910 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:51.192286    4276 patch_prober.go:29] interesting pod/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-8bf7k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused" start-of-body=
Oct 31 17:41:51.228113 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:51.219632    4276 patch_prober.go:29] interesting pod/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-m7vsj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" start-of-body=
Oct 31 17:41:51.235575 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:51.235528    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-m7vsj" podUID="f675ff71-27f9-44e8-aa32-89f4332d4c3e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused"
Oct 31 17:41:51.235797 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:51.228110    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-8bf7k" podUID="28cbdb7c-1bc2-49b4-834c-de871be33524" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused"
Oct 31 17:41:51.251575 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:51.251537    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-1c0bfd/distinct-2-w4-1c0bfd-kserve-847bcf6cf4-lt8ls" podUID="f8a45b7c-e5ae-4321-ad81-fd6353b2495d" containerName="main" probeResult="failure" output="Get \"https://10.133.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 17:41:51.256672 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:51.256649    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-1c0bfd/distinct-2-w4-1c0bfd-kserve-847bcf6cf4-lt8ls"
Oct 31 17:41:51.560524 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:51.518535    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-5c446c/distinct-w2-5c446c-kserve-69d97d6968-gtvjv" podUID="2d1185cc-bac1-45ca-a250-8f44f8589162" containerName="main" probeResult="failure" output="Get \"https://10.133.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 17:41:51.560524 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:51.541436    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-0e59ca/premium-w1-0e59ca-kserve-58765f9746-fqwq7" podUID="8cf1e52e-70f8-42e2-bdf3-a315e17f10f5" containerName="main" probeResult="failure" output="Get \"https://10.133.0.48:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 17:41:51.572707 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:51.572624    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ee816e/premium-w0-ee816e-kserve-65c86cd6db-lnt9p" podUID="e536c902-52a9-4b36-a2ad-eb5500781a5e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.49:8000/ready\": context deadline exceeded"
Oct 31 17:41:51.773465 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:51.770541    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-5c446c/distinct-2-w2-5c446c-kserve-5d9cd79c47-p4j5l" podUID="4c288908-92e3-4c45-8ba5-2f86c9f38b3d" containerName="main" probeResult="failure" output="Get \"https://10.133.0.65:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 17:41:51.998525 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:51.963472    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-5c446c/distinct-w2-5c446c-kserve-69d97d6968-gtvjv"
Oct 31 17:41:52.062660 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:52.062633    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-0e59ca/premium-w1-0e59ca-kserve-58765f9746-fqwq7"
Oct 31 17:41:52.066561 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:52.066533    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-5c446c/distinct-2-w2-5c446c-kserve-5d9cd79c47-p4j5l"
Oct 31 17:41:52.306715 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:52.252458    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-1c0bfd/distinct-2-w4-1c0bfd-kserve-847bcf6cf4-lt8ls" podUID="f8a45b7c-e5ae-4321-ad81-fd6353b2495d" containerName="main" probeResult="failure" output="Get \"https://10.133.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 17:41:52.306715 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:52.039698    4276 patch_prober.go:29] interesting pod/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-bwdwd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" start-of-body=
Oct 31 17:41:52.306715 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:52.252786    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-bwdwd" podUID="7ec5066a-6b4d-4669-97f5-8a46a9c99a1b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused"
Oct 31 17:41:52.306715 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:52.154460    4276 patch_prober.go:29] interesting pod/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-8867c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body=
Oct 31 17:41:52.306715 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:52.252834    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-8867c" podUID="d1bf33fd-97d5-462d-8c04-e2697fcf1b05" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused"
Oct 31 17:41:52.306715 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:52.236607    4276 patch_prober.go:29] interesting pod/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-m7vsj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" start-of-body=
Oct 31 17:41:52.306715 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:52.252872    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-m7vsj" podUID="f675ff71-27f9-44e8-aa32-89f4332d4c3e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused"
Oct 31 17:41:52.306715 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:51.999825    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-d80080/distinct-w6-d80080-kserve-664576c8f5-6jc4d" podUID="b9c2f445-a191-4b00-802a-1b0ded068411" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 17:41:52.316476 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:52.313908    4276 patch_prober.go:29] interesting pod/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-8bf7k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused" start-of-body=
Oct 31 17:41:52.316476 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:52.313965    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-8bf7k" podUID="28cbdb7c-1bc2-49b4-834c-de871be33524" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused"
Oct 31 17:41:52.583812 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:52.576558    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-0e59ca/premium-w1-0e59ca-kserve-58765f9746-fqwq7" podUID="8cf1e52e-70f8-42e2-bdf3-a315e17f10f5" containerName="main" probeResult="failure" output="Get \"https://10.133.0.48:8000/ready\": context deadline exceeded"
Oct 31 17:41:52.611465 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:52.596173    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-5c446c/distinct-w2-5c446c-kserve-69d97d6968-gtvjv" podUID="2d1185cc-bac1-45ca-a250-8f44f8589162" containerName="main" probeResult="failure" output="Get \"https://10.133.0.58:8000/ready\": context deadline exceeded"
Oct 31 17:41:52.611465 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:52.609920    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ee816e/premium-w0-ee816e-kserve-65c86cd6db-lnt9p" podUID="e536c902-52a9-4b36-a2ad-eb5500781a5e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.49:8000/ready\": context deadline exceeded"
Oct 31 17:41:52.669053 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:52.669022    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-0e59ca/premium-w1-0e59ca-kserve-58765f9746-fqwq7"
Oct 31 17:41:52.816469 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:52.789386    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-5c446c/distinct-2-w2-5c446c-kserve-5d9cd79c47-p4j5l" podUID="4c288908-92e3-4c45-8ba5-2f86c9f38b3d" containerName="main" probeResult="failure" output="Get \"https://10.133.0.65:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 17:41:53.203490 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:53.202505    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:41:53.261527 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:53.261488    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-d80080/distinct-w6-d80080-kserve-664576c8f5-6jc4d" podUID="b9c2f445-a191-4b00-802a-1b0ded068411" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 17:41:53.262949 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:53.262913    4276 patch_prober.go:29] interesting pod/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-8bf7k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused" start-of-body=
Oct 31 17:41:53.263165 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:53.263141    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-8bf7k" podUID="28cbdb7c-1bc2-49b4-834c-de871be33524" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused"
Oct 31 17:41:53.264740 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:53.264715    4276 patch_prober.go:29] interesting pod/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-bwdwd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" start-of-body=
Oct 31 17:41:53.264970 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:53.264948    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-bwdwd" podUID="7ec5066a-6b4d-4669-97f5-8a46a9c99a1b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused"
Oct 31 17:41:53.378470 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:53.358417    4276 patch_prober.go:29] interesting pod/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-8867c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body=
Oct 31 17:41:53.382476 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:53.380901    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-8867c" podUID="d1bf33fd-97d5-462d-8c04-e2697fcf1b05" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused"
Oct 31 17:41:53.431472 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:53.361952    4276 patch_prober.go:29] interesting pod/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-m7vsj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" start-of-body=
Oct 31 17:41:53.524613 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:53.524493    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-1c0bfd/distinct-2-w4-1c0bfd-kserve-847bcf6cf4-lt8ls" podUID="f8a45b7c-e5ae-4321-ad81-fd6353b2495d" containerName="main" probeResult="failure" output="Get \"https://10.133.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 17:41:53.524936 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:53.524907    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-m7vsj" podUID="f675ff71-27f9-44e8-aa32-89f4332d4c3e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused"
Oct 31 17:41:53.604662 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:53.604612    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-5c446c/distinct-w2-5c446c-kserve-69d97d6968-gtvjv" podUID="2d1185cc-bac1-45ca-a250-8f44f8589162" containerName="main" probeResult="failure" output="Get \"https://10.133.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 17:41:53.705469 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:53.612609    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ee816e/premium-w0-ee816e-kserve-65c86cd6db-lnt9p" podUID="e536c902-52a9-4b36-a2ad-eb5500781a5e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.49:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 17:41:53.789506 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:53.747590    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-0e59ca/premium-w1-0e59ca-kserve-58765f9746-fqwq7" podUID="8cf1e52e-70f8-42e2-bdf3-a315e17f10f5" containerName="main" probeResult="failure" output="Get \"https://10.133.0.48:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 31 17:41:53.929474 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:53.905810    4276 patch_prober.go:29] interesting pod/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-8867c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body=
Oct 31 17:41:53.941555 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:53.941512    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-8867c" podUID="d1bf33fd-97d5-462d-8c04-e2697fcf1b05" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused"
Oct 31 17:41:53.941771 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:53.938833    4276 patch_prober.go:29] interesting pod/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-bwdwd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" start-of-body=
Oct 31 17:41:53.945527 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:53.945495    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-bwdwd" podUID="7ec5066a-6b4d-4669-97f5-8a46a9c99a1b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused"
Oct 31 17:41:53.970466 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:53.949510    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-5c446c/distinct-2-w2-5c446c-kserve-5d9cd79c47-p4j5l" podUID="4c288908-92e3-4c45-8ba5-2f86c9f38b3d" containerName="main" probeResult="failure" output="Get \"https://10.133.0.65:8000/ready\": context deadline exceeded"
Oct 31 17:41:54.203469 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:54.201255    4276 patch_prober.go:29] interesting pod/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-8bf7k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused" start-of-body=
Oct 31 17:41:54.203469 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:54.201325    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-8bf7k" podUID="28cbdb7c-1bc2-49b4-834c-de871be33524" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused"
Oct 31 17:41:54.203469 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:54.203178    4276 patch_prober.go:29] interesting pod/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-m7vsj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" start-of-body=
Oct 31 17:41:54.203469 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:54.203220    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-m7vsj" podUID="f675ff71-27f9-44e8-aa32-89f4332d4c3e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused"
Oct 31 17:41:54.282680 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:54.282635    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-d80080/distinct-w6-d80080-kserve-664576c8f5-6jc4d" podUID="b9c2f445-a191-4b00-802a-1b0ded068411" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": context deadline exceeded"
Oct 31 17:41:54.388132 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:54.388090    4276 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-ee816e/premium-w0-ee816e-kserve-65c86cd6db-lnt9p" podUID="e536c902-52a9-4b36-a2ad-eb5500781a5e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.49:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 17:41:54.528567 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:54.502474    4276 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-5c446c/distinct-w2-5c446c-kserve-69d97d6968-gtvjv" podUID="2d1185cc-bac1-45ca-a250-8f44f8589162" containerName="main" probeResult="failure" output="Get \"https://10.133.0.58:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 17:41:54.530726 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:54.530690    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-1c0bfd/distinct-2-w4-1c0bfd-kserve-847bcf6cf4-lt8ls" podUID="f8a45b7c-e5ae-4321-ad81-fd6353b2495d" containerName="main" probeResult="failure" output="Get \"https://10.133.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 17:41:54.614497 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:54.587580    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-ee816e/premium-w0-ee816e-kserve-65c86cd6db-lnt9p"
Oct 31 17:41:54.614497 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:54.613139    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-5c446c/distinct-w2-5c446c-kserve-69d97d6968-gtvjv" podUID="2d1185cc-bac1-45ca-a250-8f44f8589162" containerName="main" probeResult="failure" output="Get \"https://10.133.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 17:41:54.812562 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:54.780298    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-0e59ca/premium-w1-0e59ca-kserve-58765f9746-fqwq7" podUID="8cf1e52e-70f8-42e2-bdf3-a315e17f10f5" containerName="main" probeResult="failure" output="Get \"https://10.133.0.48:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 17:41:54.839211 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:54.839180    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-0e59ca/premium-w1-0e59ca-kserve-58765f9746-fqwq7"
Oct 31 17:41:54.932523 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:54.900490    4276 patch_prober.go:29] interesting pod/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-bwdwd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" start-of-body=
Oct 31 17:41:54.932792 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:54.932763    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-bwdwd" podUID="7ec5066a-6b4d-4669-97f5-8a46a9c99a1b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused"
Oct 31 17:41:54.932904 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:54.909852    4276 patch_prober.go:29] interesting pod/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-8867c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body=
Oct 31 17:41:54.932996 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:54.932982    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-8867c" podUID="d1bf33fd-97d5-462d-8c04-e2697fcf1b05" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused"
Oct 31 17:41:54.944305 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:54.942726    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-1c0bfd/distinct-2-w4-1c0bfd-kserve-847bcf6cf4-lt8ls"
Oct 31 17:41:54.976413 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:54.976372    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-5c446c/distinct-2-w2-5c446c-kserve-5d9cd79c47-p4j5l" podUID="4c288908-92e3-4c45-8ba5-2f86c9f38b3d" containerName="main" probeResult="failure" output="Get \"https://10.133.0.65:8000/ready\": context deadline exceeded"
Oct 31 17:41:55.106706 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:55.106622    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-5c446c/distinct-w2-5c446c-kserve-69d97d6968-gtvjv"
Oct 31 17:41:55.140053 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:55.138394    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-0e59ca/premium-w1-0e59ca-kserve-58765f9746-fqwq7"
Oct 31 17:41:55.162694 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:55.140747    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-d80080/distinct-w6-d80080-kserve-664576c8f5-6jc4d"
Oct 31 17:41:55.229624 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:55.229594    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-5c446c/distinct-2-w2-5c446c-kserve-5d9cd79c47-p4j5l"
Oct 31 17:41:55.268223 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:55.255809    4276 patch_prober.go:29] interesting pod/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-m7vsj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" start-of-body=
Oct 31 17:41:55.268223 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:55.255874    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-m7vsj" podUID="f675ff71-27f9-44e8-aa32-89f4332d4c3e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused"
Oct 31 17:41:55.269784 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:55.269601    4276 patch_prober.go:29] interesting pod/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-8bf7k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused" start-of-body=
Oct 31 17:41:55.275427 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:55.275400    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-8bf7k" podUID="28cbdb7c-1bc2-49b4-834c-de871be33524" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused"
Oct 31 17:41:55.523580 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:55.523436    4276 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-5c446c/distinct-2-w2-5c446c-kserve-5d9cd79c47-p4j5l"]
Oct 31 17:41:55.523820 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:55.523801    4276 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-5c446c/distinct-w2-5c446c-kserve-69d97d6968-gtvjv"]
Oct 31 17:41:55.548465 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:55.548418    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:41:55.693387 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:55.693299    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:41:55.947481 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:55.926608    4276 patch_prober.go:29] interesting pod/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-bwdwd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" start-of-body=
Oct 31 17:41:55.947481 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:55.926672    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-bwdwd" podUID="7ec5066a-6b4d-4669-97f5-8a46a9c99a1b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused"
Oct 31 17:41:55.947481 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:55.945288    4276 patch_prober.go:29] interesting pod/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-8867c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body=
Oct 31 17:41:55.947481 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:55.945429    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-8867c" podUID="d1bf33fd-97d5-462d-8c04-e2697fcf1b05" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused"
Oct 31 17:41:56.170493 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:56.170273    4276 patch_prober.go:29] interesting pod/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-8bf7k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused" start-of-body=
Oct 31 17:41:56.170493 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:56.170330    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-8bf7k" podUID="28cbdb7c-1bc2-49b4-834c-de871be33524" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused"
Oct 31 17:41:56.178472 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:56.178122    4276 patch_prober.go:29] interesting pod/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-m7vsj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" start-of-body=
Oct 31 17:41:56.178472 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:56.178169    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-m7vsj" podUID="f675ff71-27f9-44e8-aa32-89f4332d4c3e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused"
Oct 31 17:41:56.896216 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:56.896177    4276 patch_prober.go:29] interesting pod/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-bwdwd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" start-of-body=
Oct 31 17:41:56.908360 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:56.908316    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-bwdwd" podUID="7ec5066a-6b4d-4669-97f5-8a46a9c99a1b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused"
Oct 31 17:41:57.195532 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:57.194529    4276 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-m7vsj"
Oct 31 17:41:57.195861 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:57.195840    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-m7vsj"
Oct 31 17:41:57.204740 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:57.204707    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-m7vsj"
Oct 31 17:41:57.224074 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:57.224038    4276 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-8bf7k"
Oct 31 17:41:57.228524 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:57.225922    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-8bf7k"
Oct 31 17:41:57.230823 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:57.230376    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-8bf7k"
Oct 31 17:41:57.483591 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:57.483396    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:41:57.511412 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:57.511387    4276 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-8867c"
Oct 31 17:41:57.513182 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:57.513163    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-8867c"
Oct 31 17:41:57.528478 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:57.517200    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-8867c"
Oct 31 17:41:57.900954 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:57.900929    4276 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-bwdwd"
Oct 31 17:41:57.901572 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:57.901548    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-bwdwd"
Oct 31 17:41:57.902368 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:57.902348    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-bwdwd"
Oct 31 17:41:58.141710 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:58.141677    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:41:59.175968 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:59.175930    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:41:59.281098 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:59.281071    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:41:59.424479 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:41:59.406154    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:42:00.992952 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:00.992865    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:42:03.214464 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:03.199538    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:42:03.304468 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:03.300567    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-d80080/distinct-w6-d80080-kserve-664576c8f5-6jc4d" podUID="b9c2f445-a191-4b00-802a-1b0ded068411" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 17:42:03.478668 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:03.478556    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:42:03.603467 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:03.602532    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ee816e/premium-w0-ee816e-kserve-65c86cd6db-lnt9p" podUID="e536c902-52a9-4b36-a2ad-eb5500781a5e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.49:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 17:42:03.624469 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:03.613529    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-5c446c/distinct-w2-5c446c-kserve-69d97d6968-gtvjv" podUID="2d1185cc-bac1-45ca-a250-8f44f8589162" containerName="main" probeResult="failure" output="Get \"https://10.133.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 17:42:03.950629 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:03.950575    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-5c446c/distinct-2-w2-5c446c-kserve-5d9cd79c47-p4j5l" podUID="4c288908-92e3-4c45-8ba5-2f86c9f38b3d" containerName="main" probeResult="failure" output="Get \"https://10.133.0.65:8000/ready\": context deadline exceeded"
Oct 31 17:42:04.159396 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:04.147348    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-0e59ca/premium-w1-0e59ca-kserve-58765f9746-fqwq7" podUID="8cf1e52e-70f8-42e2-bdf3-a315e17f10f5" containerName="main" probeResult="failure" output="Get \"https://10.133.0.48:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 17:42:04.224065 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:04.155112    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:42:04.293473 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:04.264509    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-1c0bfd/distinct-2-w4-1c0bfd-kserve-847bcf6cf4-lt8ls" podUID="f8a45b7c-e5ae-4321-ad81-fd6353b2495d" containerName="main" probeResult="failure" output="Get \"https://10.133.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 17:42:04.302479 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:04.302426    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:42:04.356126 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:04.356067    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-d80080/distinct-w6-d80080-kserve-664576c8f5-6jc4d" podUID="b9c2f445-a191-4b00-802a-1b0ded068411" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": context deadline exceeded"
Oct 31 17:42:04.369010 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:04.368977    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-d80080/distinct-w6-d80080-kserve-664576c8f5-6jc4d"
Oct 31 17:42:04.579523 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:04.574363    4276 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-ee816e/premium-w0-ee816e-kserve-65c86cd6db-lnt9p" podUID="e536c902-52a9-4b36-a2ad-eb5500781a5e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.49:8000/health\": context deadline exceeded"
Oct 31 17:42:04.654973 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:04.654906    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-5c446c/distinct-w2-5c446c-kserve-69d97d6968-gtvjv" podUID="2d1185cc-bac1-45ca-a250-8f44f8589162" containerName="main" probeResult="failure" output="Get \"https://10.133.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 17:42:04.680372 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:04.680341    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-5c446c/distinct-w2-5c446c-kserve-69d97d6968-gtvjv"
Oct 31 17:42:05.110528 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:05.086503    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ee816e/premium-w0-ee816e-kserve-65c86cd6db-lnt9p" podUID="e536c902-52a9-4b36-a2ad-eb5500781a5e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.49:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 17:42:05.111299 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:05.111269    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-ee816e/premium-w0-ee816e-kserve-65c86cd6db-lnt9p"
Oct 31 17:42:05.166344 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:05.143678    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:42:05.252464 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:05.216722    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-5c446c/distinct-2-w2-5c446c-kserve-5d9cd79c47-p4j5l" podUID="4c288908-92e3-4c45-8ba5-2f86c9f38b3d" containerName="main" probeResult="failure" output="Get \"https://10.133.0.65:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 17:42:05.296213 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:05.243372    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-0e59ca/premium-w1-0e59ca-kserve-58765f9746-fqwq7" podUID="8cf1e52e-70f8-42e2-bdf3-a315e17f10f5" containerName="main" probeResult="failure" output="Get \"https://10.133.0.48:8000/ready\": context deadline exceeded"
Oct 31 17:42:05.356469 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:05.334819    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-5c446c/distinct-2-w2-5c446c-kserve-5d9cd79c47-p4j5l"
Oct 31 17:42:05.527644 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:05.390726    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-d80080/distinct-w6-d80080-kserve-664576c8f5-6jc4d" podUID="b9c2f445-a191-4b00-802a-1b0ded068411" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 17:42:05.565561 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:05.393322    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-1c0bfd/distinct-2-w4-1c0bfd-kserve-847bcf6cf4-lt8ls" podUID="f8a45b7c-e5ae-4321-ad81-fd6353b2495d" containerName="main" probeResult="failure" output="Get \"https://10.133.0.67:8000/ready\": context deadline exceeded"
Oct 31 17:42:05.565738 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:05.565644    4276 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w6-d80080/distinct-w6-d80080-kserve-664576c8f5-6jc4d" podUID="b9c2f445-a191-4b00-802a-1b0ded068411" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/health\": context deadline exceeded"
Oct 31 17:42:05.609466 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:05.608833    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-0e59ca/premium-w1-0e59ca-kserve-58765f9746-fqwq7"
Oct 31 17:42:05.609466 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:05.608923    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-1c0bfd/distinct-2-w4-1c0bfd-kserve-847bcf6cf4-lt8ls"
Oct 31 17:42:05.918376 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:05.838533    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-5c446c/distinct-w2-5c446c-kserve-69d97d6968-gtvjv" podUID="2d1185cc-bac1-45ca-a250-8f44f8589162" containerName="main" probeResult="failure" output="Get \"https://10.133.0.58:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 31 17:42:05.918376 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:05.842180    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-d80080/distinct-w6-d80080-kserve-664576c8f5-6jc4d"
Oct 31 17:42:06.064469 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:06.063879    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-5c446c/distinct-2-w2-5c446c-kserve-5d9cd79c47-p4j5l"
Oct 31 17:42:06.198553 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:06.171575    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ee816e/premium-w0-ee816e-kserve-65c86cd6db-lnt9p" podUID="e536c902-52a9-4b36-a2ad-eb5500781a5e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.49:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 31 17:42:06.301472 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:06.297203    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-0e59ca/premium-w1-0e59ca-kserve-58765f9746-fqwq7" podUID="8cf1e52e-70f8-42e2-bdf3-a315e17f10f5" containerName="main" probeResult="failure" output="Get \"https://10.133.0.48:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 17:42:06.323768 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:06.323654    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-ee816e/premium-w0-ee816e-kserve-65c86cd6db-lnt9p"
Oct 31 17:42:06.326088 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:06.326068    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-0e59ca/premium-w1-0e59ca-kserve-58765f9746-fqwq7"
Oct 31 17:42:06.326323 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:06.326310    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-5c446c/distinct-w2-5c446c-kserve-69d97d6968-gtvjv"
Oct 31 17:42:06.333681 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:06.330518    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-1c0bfd/distinct-2-w4-1c0bfd-kserve-847bcf6cf4-lt8ls"
Oct 31 17:42:07.348749 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:07.348717    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:42:07.980676 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:07.975695    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:42:09.279767 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:09.279732    4276 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-cf0a86-7bb4c5d6c5-8p8r5"]
Oct 31 17:42:09.405560 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:09.405527    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:42:09.528627 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:09.528596    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:42:11.437792 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:11.437756    4276 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w2-5c446c/distinct-2-w2-5c446c-kserve-5d9cd79c47-p4j5l" podUID="4c288908-92e3-4c45-8ba5-2f86c9f38b3d" containerName="main" containerID="cri-o://6274d0f6e3ab3ef86dc4be5543d1748a8d2c5cfbd4c6bb1c80547c0702dcdd5e" gracePeriod=45
Oct 31 17:42:11.465799 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:11.465732    4276 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w2-5c446c/distinct-w2-5c446c-kserve-69d97d6968-gtvjv" podUID="2d1185cc-bac1-45ca-a250-8f44f8589162" containerName="main" containerID="cri-o://b9f950b5763a04a6e7f73de21e40008851382e1b9e75364b413def2e4f1204fc" gracePeriod=45
Oct 31 17:42:11.576748 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:11.576718    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:42:11.606570 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:11.606524    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-5c446c/distinct-w2-5c446c-kserve-69d97d6968-gtvjv" podUID="2d1185cc-bac1-45ca-a250-8f44f8589162" containerName="main" probeResult="failure" output="Get \"https://10.133.0.58:8000/ready\": dial tcp 10.133.0.58:8000: connect: connection refused"
Oct 31 17:42:11.869120 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:11.869095    4276 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-5c446c/distinct-2-w2-5c446c-kserve-5d9cd79c47-p4j5l"
Oct 31 17:42:11.912428 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:11.912399    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4c288908-92e3-4c45-8ba5-2f86c9f38b3d-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4c288908-92e3-4c45-8ba5-2f86c9f38b3d-model-cache\") pod \"4c288908-92e3-4c45-8ba5-2f86c9f38b3d\" (UID: \"4c288908-92e3-4c45-8ba5-2f86c9f38b3d\") "
Oct 31 17:42:11.912740 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:11.912715    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4c288908-92e3-4c45-8ba5-2f86c9f38b3d-tls-certs\" (UniqueName: \"kubernetes.io/secret/4c288908-92e3-4c45-8ba5-2f86c9f38b3d-tls-certs\") pod \"4c288908-92e3-4c45-8ba5-2f86c9f38b3d\" (UID: \"4c288908-92e3-4c45-8ba5-2f86c9f38b3d\") "
Oct 31 17:42:11.912816 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:11.912760    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4c288908-92e3-4c45-8ba5-2f86c9f38b3d-kube-api-access-phc42\" (UniqueName: \"kubernetes.io/projected/4c288908-92e3-4c45-8ba5-2f86c9f38b3d-kube-api-access-phc42\") pod \"4c288908-92e3-4c45-8ba5-2f86c9f38b3d\" (UID: \"4c288908-92e3-4c45-8ba5-2f86c9f38b3d\") "
Oct 31 17:42:11.912874 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:11.912816    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4c288908-92e3-4c45-8ba5-2f86c9f38b3d-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4c288908-92e3-4c45-8ba5-2f86c9f38b3d-tmp-dir\") pod \"4c288908-92e3-4c45-8ba5-2f86c9f38b3d\" (UID: \"4c288908-92e3-4c45-8ba5-2f86c9f38b3d\") "
Oct 31 17:42:11.912874 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:11.912841    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4c288908-92e3-4c45-8ba5-2f86c9f38b3d-dshm\" (UniqueName: \"kubernetes.io/empty-dir/4c288908-92e3-4c45-8ba5-2f86c9f38b3d-dshm\") pod \"4c288908-92e3-4c45-8ba5-2f86c9f38b3d\" (UID: \"4c288908-92e3-4c45-8ba5-2f86c9f38b3d\") "
Oct 31 17:42:11.912874 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:11.912870    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4c288908-92e3-4c45-8ba5-2f86c9f38b3d-home\" (UniqueName: \"kubernetes.io/empty-dir/4c288908-92e3-4c45-8ba5-2f86c9f38b3d-home\") pod \"4c288908-92e3-4c45-8ba5-2f86c9f38b3d\" (UID: \"4c288908-92e3-4c45-8ba5-2f86c9f38b3d\") "
Oct 31 17:42:11.913884 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:11.913838    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c288908-92e3-4c45-8ba5-2f86c9f38b3d-home" pod "4c288908-92e3-4c45-8ba5-2f86c9f38b3d" (UID: "4c288908-92e3-4c45-8ba5-2f86c9f38b3d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:42:11.914555 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:11.914530    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c288908-92e3-4c45-8ba5-2f86c9f38b3d-model-cache" pod "4c288908-92e3-4c45-8ba5-2f86c9f38b3d" (UID: "4c288908-92e3-4c45-8ba5-2f86c9f38b3d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:42:11.918018 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:11.917840    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c288908-92e3-4c45-8ba5-2f86c9f38b3d-tmp-dir" pod "4c288908-92e3-4c45-8ba5-2f86c9f38b3d" (UID: "4c288908-92e3-4c45-8ba5-2f86c9f38b3d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:42:11.918650 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:11.918626    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4c288908-92e3-4c45-8ba5-2f86c9f38b3d-tls-certs" pod "4c288908-92e3-4c45-8ba5-2f86c9f38b3d" (UID: "4c288908-92e3-4c45-8ba5-2f86c9f38b3d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 17:42:11.921349 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:11.921325    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c288908-92e3-4c45-8ba5-2f86c9f38b3d-dshm" pod "4c288908-92e3-4c45-8ba5-2f86c9f38b3d" (UID: "4c288908-92e3-4c45-8ba5-2f86c9f38b3d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:42:11.923365 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:11.923327    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4c288908-92e3-4c45-8ba5-2f86c9f38b3d-kube-api-access-phc42" pod "4c288908-92e3-4c45-8ba5-2f86c9f38b3d" (UID: "4c288908-92e3-4c45-8ba5-2f86c9f38b3d"). InnerVolumeSpecName "kube-api-access-phc42". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 17:42:11.990624 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:11.990602    4276 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-5c446c/distinct-w2-5c446c-kserve-69d97d6968-gtvjv"
Oct 31 17:42:12.014383 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:12.014351    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2d1185cc-bac1-45ca-a250-8f44f8589162-tls-certs\" (UniqueName: \"kubernetes.io/secret/2d1185cc-bac1-45ca-a250-8f44f8589162-tls-certs\") pod \"2d1185cc-bac1-45ca-a250-8f44f8589162\" (UID: \"2d1185cc-bac1-45ca-a250-8f44f8589162\") "
Oct 31 17:42:12.014525 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:12.014398    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2d1185cc-bac1-45ca-a250-8f44f8589162-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2d1185cc-bac1-45ca-a250-8f44f8589162-tmp-dir\") pod \"2d1185cc-bac1-45ca-a250-8f44f8589162\" (UID: \"2d1185cc-bac1-45ca-a250-8f44f8589162\") "
Oct 31 17:42:12.014574 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:12.014525    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2d1185cc-bac1-45ca-a250-8f44f8589162-home\" (UniqueName: \"kubernetes.io/empty-dir/2d1185cc-bac1-45ca-a250-8f44f8589162-home\") pod \"2d1185cc-bac1-45ca-a250-8f44f8589162\" (UID: \"2d1185cc-bac1-45ca-a250-8f44f8589162\") "
Oct 31 17:42:12.014653 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:12.014636    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2d1185cc-bac1-45ca-a250-8f44f8589162-dshm\" (UniqueName: \"kubernetes.io/empty-dir/2d1185cc-bac1-45ca-a250-8f44f8589162-dshm\") pod \"2d1185cc-bac1-45ca-a250-8f44f8589162\" (UID: \"2d1185cc-bac1-45ca-a250-8f44f8589162\") "
Oct 31 17:42:12.014722 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:12.014650    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2d1185cc-bac1-45ca-a250-8f44f8589162-tmp-dir" pod "2d1185cc-bac1-45ca-a250-8f44f8589162" (UID: "2d1185cc-bac1-45ca-a250-8f44f8589162"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:42:12.014722 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:12.014696    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2d1185cc-bac1-45ca-a250-8f44f8589162-kube-api-access-sl555\" (UniqueName: \"kubernetes.io/projected/2d1185cc-bac1-45ca-a250-8f44f8589162-kube-api-access-sl555\") pod \"2d1185cc-bac1-45ca-a250-8f44f8589162\" (UID: \"2d1185cc-bac1-45ca-a250-8f44f8589162\") "
Oct 31 17:42:12.014834 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:12.014754    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2d1185cc-bac1-45ca-a250-8f44f8589162-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2d1185cc-bac1-45ca-a250-8f44f8589162-model-cache\") pod \"2d1185cc-bac1-45ca-a250-8f44f8589162\" (UID: \"2d1185cc-bac1-45ca-a250-8f44f8589162\") "
Oct 31 17:42:12.014834 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:12.014804    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2d1185cc-bac1-45ca-a250-8f44f8589162-home" pod "2d1185cc-bac1-45ca-a250-8f44f8589162" (UID: "2d1185cc-bac1-45ca-a250-8f44f8589162"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:42:12.015102 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:12.015074    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2d1185cc-bac1-45ca-a250-8f44f8589162-model-cache" pod "2d1185cc-bac1-45ca-a250-8f44f8589162" (UID: "2d1185cc-bac1-45ca-a250-8f44f8589162"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:42:12.015553 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:12.015531    4276 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4c288908-92e3-4c45-8ba5-2f86c9f38b3d-dshm\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:42:12.015639 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:12.015556    4276 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4c288908-92e3-4c45-8ba5-2f86c9f38b3d-home\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:42:12.015639 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:12.015570    4276 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2d1185cc-bac1-45ca-a250-8f44f8589162-tmp-dir\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:42:12.015639 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:12.015585    4276 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4c288908-92e3-4c45-8ba5-2f86c9f38b3d-model-cache\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:42:12.015639 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:12.015597    4276 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2d1185cc-bac1-45ca-a250-8f44f8589162-home\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:42:12.015639 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:12.015608    4276 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2d1185cc-bac1-45ca-a250-8f44f8589162-model-cache\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:42:12.015639 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:12.015620    4276 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4c288908-92e3-4c45-8ba5-2f86c9f38b3d-tls-certs\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:42:12.015639 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:12.015633    4276 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-phc42\" (UniqueName: \"kubernetes.io/projected/4c288908-92e3-4c45-8ba5-2f86c9f38b3d-kube-api-access-phc42\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:42:12.015872 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:12.015648    4276 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4c288908-92e3-4c45-8ba5-2f86c9f38b3d-tmp-dir\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:42:12.016781 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:12.016721    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2d1185cc-bac1-45ca-a250-8f44f8589162-tls-certs" pod "2d1185cc-bac1-45ca-a250-8f44f8589162" (UID: "2d1185cc-bac1-45ca-a250-8f44f8589162"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 17:42:12.016781 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:12.016762    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2d1185cc-bac1-45ca-a250-8f44f8589162-dshm" pod "2d1185cc-bac1-45ca-a250-8f44f8589162" (UID: "2d1185cc-bac1-45ca-a250-8f44f8589162"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:42:12.016912 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:12.016877    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2d1185cc-bac1-45ca-a250-8f44f8589162-kube-api-access-sl555" pod "2d1185cc-bac1-45ca-a250-8f44f8589162" (UID: "2d1185cc-bac1-45ca-a250-8f44f8589162"). InnerVolumeSpecName "kube-api-access-sl555". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 17:42:12.117377 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:12.117342    4276 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2d1185cc-bac1-45ca-a250-8f44f8589162-dshm\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:42:12.117377 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:12.117368    4276 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sl555\" (UniqueName: \"kubernetes.io/projected/2d1185cc-bac1-45ca-a250-8f44f8589162-kube-api-access-sl555\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:42:12.117377 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:12.117376    4276 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2d1185cc-bac1-45ca-a250-8f44f8589162-tls-certs\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:42:12.236740 ip-10-0-129-127 kubenswrapper[4276]: E1031 17:42:12.236660    4276 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tg5qk" containerName="extractor"
Oct 31 17:42:12.407720 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:12.407676    4276 generic.go:356] "Generic (PLEG): container finished" podID="2d1185cc-bac1-45ca-a250-8f44f8589162" containerID="b9f950b5763a04a6e7f73de21e40008851382e1b9e75364b413def2e4f1204fc" exitCode=0
Oct 31 17:42:12.407903 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:12.407731    4276 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-5c446c/distinct-w2-5c446c-kserve-69d97d6968-gtvjv"
Oct 31 17:42:12.407903 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:12.407783    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-5c446c/distinct-w2-5c446c-kserve-69d97d6968-gtvjv" event={"ID":"2d1185cc-bac1-45ca-a250-8f44f8589162","Type":"ContainerDied","Data":"b9f950b5763a04a6e7f73de21e40008851382e1b9e75364b413def2e4f1204fc"}
Oct 31 17:42:12.407903 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:12.407822    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-5c446c/distinct-w2-5c446c-kserve-69d97d6968-gtvjv" event={"ID":"2d1185cc-bac1-45ca-a250-8f44f8589162","Type":"ContainerDied","Data":"805e0d3ccc93d0325525a13ebc031326d1c8e5a9465a55ed4038a06399ec6c60"}
Oct 31 17:42:12.407903 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:12.407848    4276 scope.go:122] "RemoveContainer" containerID="b9f950b5763a04a6e7f73de21e40008851382e1b9e75364b413def2e4f1204fc"
Oct 31 17:42:12.409782 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:12.409717    4276 generic.go:356] "Generic (PLEG): container finished" podID="4c288908-92e3-4c45-8ba5-2f86c9f38b3d" containerID="6274d0f6e3ab3ef86dc4be5543d1748a8d2c5cfbd4c6bb1c80547c0702dcdd5e" exitCode=0
Oct 31 17:42:12.409878 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:12.409785    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-5c446c/distinct-2-w2-5c446c-kserve-5d9cd79c47-p4j5l" event={"ID":"4c288908-92e3-4c45-8ba5-2f86c9f38b3d","Type":"ContainerDied","Data":"6274d0f6e3ab3ef86dc4be5543d1748a8d2c5cfbd4c6bb1c80547c0702dcdd5e"}
Oct 31 17:42:12.409878 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:12.409815    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-5c446c/distinct-2-w2-5c446c-kserve-5d9cd79c47-p4j5l" event={"ID":"4c288908-92e3-4c45-8ba5-2f86c9f38b3d","Type":"ContainerDied","Data":"55b50a582324c4fbc081dff75ede695e54805bbddf4985ce6fd31c2b7e0f2e82"}
Oct 31 17:42:12.409981 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:12.409876    4276 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-5c446c/distinct-2-w2-5c446c-kserve-5d9cd79c47-p4j5l"
Oct 31 17:42:12.422037 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:12.422013    4276 scope.go:122] "RemoveContainer" containerID="b9f950b5763a04a6e7f73de21e40008851382e1b9e75364b413def2e4f1204fc"
Oct 31 17:42:12.422376 ip-10-0-129-127 kubenswrapper[4276]: E1031 17:42:12.422356    4276 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b9f950b5763a04a6e7f73de21e40008851382e1b9e75364b413def2e4f1204fc\": container with ID starting with b9f950b5763a04a6e7f73de21e40008851382e1b9e75364b413def2e4f1204fc not found: ID does not exist" containerID="b9f950b5763a04a6e7f73de21e40008851382e1b9e75364b413def2e4f1204fc"
Oct 31 17:42:12.422504 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:12.422390    4276 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b9f950b5763a04a6e7f73de21e40008851382e1b9e75364b413def2e4f1204fc"} err="failed to get container status \"b9f950b5763a04a6e7f73de21e40008851382e1b9e75364b413def2e4f1204fc\": rpc error: code = NotFound desc = could not find container \"b9f950b5763a04a6e7f73de21e40008851382e1b9e75364b413def2e4f1204fc\": container with ID starting with b9f950b5763a04a6e7f73de21e40008851382e1b9e75364b413def2e4f1204fc not found: ID does not exist"
Oct 31 17:42:12.422504 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:12.422414    4276 scope.go:122] "RemoveContainer" containerID="6274d0f6e3ab3ef86dc4be5543d1748a8d2c5cfbd4c6bb1c80547c0702dcdd5e"
Oct 31 17:42:12.434215 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:12.434197    4276 scope.go:122] "RemoveContainer" containerID="6274d0f6e3ab3ef86dc4be5543d1748a8d2c5cfbd4c6bb1c80547c0702dcdd5e"
Oct 31 17:42:12.434535 ip-10-0-129-127 kubenswrapper[4276]: E1031 17:42:12.434513    4276 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6274d0f6e3ab3ef86dc4be5543d1748a8d2c5cfbd4c6bb1c80547c0702dcdd5e\": container with ID starting with 6274d0f6e3ab3ef86dc4be5543d1748a8d2c5cfbd4c6bb1c80547c0702dcdd5e not found: ID does not exist" containerID="6274d0f6e3ab3ef86dc4be5543d1748a8d2c5cfbd4c6bb1c80547c0702dcdd5e"
Oct 31 17:42:12.434605 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:12.434540    4276 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6274d0f6e3ab3ef86dc4be5543d1748a8d2c5cfbd4c6bb1c80547c0702dcdd5e"} err="failed to get container status \"6274d0f6e3ab3ef86dc4be5543d1748a8d2c5cfbd4c6bb1c80547c0702dcdd5e\": rpc error: code = NotFound desc = could not find container \"6274d0f6e3ab3ef86dc4be5543d1748a8d2c5cfbd4c6bb1c80547c0702dcdd5e\": container with ID starting with 6274d0f6e3ab3ef86dc4be5543d1748a8d2c5cfbd4c6bb1c80547c0702dcdd5e not found: ID does not exist"
Oct 31 17:42:12.464748 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:12.464717    4276 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-5c446c/distinct-2-w2-5c446c-kserve-5d9cd79c47-p4j5l"]
Oct 31 17:42:12.476920 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:12.476894    4276 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w2-5c446c/distinct-2-w2-5c446c-kserve-5d9cd79c47-p4j5l"]
Oct 31 17:42:12.507035 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:12.506980    4276 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-5c446c/distinct-w2-5c446c-kserve-69d97d6968-gtvjv"]
Oct 31 17:42:12.509991 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:12.509969    4276 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w2-5c446c/distinct-w2-5c446c-kserve-69d97d6968-gtvjv"]
Oct 31 17:42:12.910511 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:12.910479    4276 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-0e59ca/premium-w1-0e59ca-kserve-58765f9746-fqwq7"]
Oct 31 17:42:13.030647 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:13.030615    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:42:13.359248 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:13.359218    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:42:13.369334 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:13.369307    4276 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2d1185cc-bac1-45ca-a250-8f44f8589162" path="/var/lib/kubelet/pods/2d1185cc-bac1-45ca-a250-8f44f8589162/volumes"
Oct 31 17:42:13.369907 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:13.369890    4276 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4c288908-92e3-4c45-8ba5-2f86c9f38b3d" path="/var/lib/kubelet/pods/4c288908-92e3-4c45-8ba5-2f86c9f38b3d/volumes"
Oct 31 17:42:14.280173 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:14.280130    4276 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-cf0a86-7bb4c5d6c5-8p8r5" podUID="1bd48c9b-b035-4931-8cf6-dbb821d3f6d5" containerName="payload-pre-processing" containerID="cri-o://fe5960e6e112614bcdb6c6cd41714ed850389b563a08d6c938b3db43b17ebfc4" gracePeriod=25
Oct 31 17:42:15.062886 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:15.062855    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:42:16.680036 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:16.679958    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:42:17.448355 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:17.448327    4276 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zpf4d_5819fd64-9ac0-4436-b8fe-dcd3a27827ac/ovn-acl-logging/0.log"
Oct 31 17:42:17.451727 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:17.451706    4276 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zpf4d_5819fd64-9ac0-4436-b8fe-dcd3a27827ac/ovn-acl-logging/0.log"
Oct 31 17:42:18.353100 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:18.353068    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:42:18.661405 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:18.661334    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-cf0a86-7bb4c5d6c5-8p8r5" podUID="1bd48c9b-b035-4931-8cf6-dbb821d3f6d5" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 31 17:42:18.848883 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:18.848854    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:42:18.978652 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:18.975814    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:42:19.452696 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:19.452661    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:42:20.306936 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:20.306900    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:42:21.293548 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:21.293513    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:42:22.186076 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:22.186045    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:42:23.121639 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:23.121606    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:42:24.040990 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:24.040936    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:42:24.560835 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:24.560803    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:42:25.882885 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:25.882856    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:42:26.215394 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:26.215320    4276 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-1c0bfd/distinct-2-w4-1c0bfd-kserve-847bcf6cf4-lt8ls"]
Oct 31 17:42:26.345266 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:26.345231    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:42:27.653728 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:27.653695    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:42:27.932493 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:27.932399    4276 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w1-0e59ca/premium-w1-0e59ca-kserve-58765f9746-fqwq7" podUID="8cf1e52e-70f8-42e2-bdf3-a315e17f10f5" containerName="main" containerID="cri-o://da6e91bd6c1fc2162a5b6d8134f06ad43a10ec61ac7d785a236ab9032e95cc3c" gracePeriod=45
Oct 31 17:42:28.078489 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:28.078433    4276 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-ee816e/premium-w0-ee816e-kserve-65c86cd6db-lnt9p"]
Oct 31 17:42:28.211965 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:28.211141    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:42:28.245679 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:28.245654    4276 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-0e59ca/premium-w1-0e59ca-kserve-58765f9746-fqwq7"
Oct 31 17:42:28.348162 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:28.348122    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8cf1e52e-70f8-42e2-bdf3-a315e17f10f5-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8cf1e52e-70f8-42e2-bdf3-a315e17f10f5-tmp-dir\") pod \"8cf1e52e-70f8-42e2-bdf3-a315e17f10f5\" (UID: \"8cf1e52e-70f8-42e2-bdf3-a315e17f10f5\") "
Oct 31 17:42:28.348348 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:28.348245    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8cf1e52e-70f8-42e2-bdf3-a315e17f10f5-dshm\" (UniqueName: \"kubernetes.io/empty-dir/8cf1e52e-70f8-42e2-bdf3-a315e17f10f5-dshm\") pod \"8cf1e52e-70f8-42e2-bdf3-a315e17f10f5\" (UID: \"8cf1e52e-70f8-42e2-bdf3-a315e17f10f5\") "
Oct 31 17:42:28.348348 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:28.348282    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8cf1e52e-70f8-42e2-bdf3-a315e17f10f5-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8cf1e52e-70f8-42e2-bdf3-a315e17f10f5-model-cache\") pod \"8cf1e52e-70f8-42e2-bdf3-a315e17f10f5\" (UID: \"8cf1e52e-70f8-42e2-bdf3-a315e17f10f5\") "
Oct 31 17:42:28.348348 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:28.348344    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8cf1e52e-70f8-42e2-bdf3-a315e17f10f5-home\" (UniqueName: \"kubernetes.io/empty-dir/8cf1e52e-70f8-42e2-bdf3-a315e17f10f5-home\") pod \"8cf1e52e-70f8-42e2-bdf3-a315e17f10f5\" (UID: \"8cf1e52e-70f8-42e2-bdf3-a315e17f10f5\") "
Oct 31 17:42:28.348583 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:28.348391    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8cf1e52e-70f8-42e2-bdf3-a315e17f10f5-tls-certs\" (UniqueName: \"kubernetes.io/secret/8cf1e52e-70f8-42e2-bdf3-a315e17f10f5-tls-certs\") pod \"8cf1e52e-70f8-42e2-bdf3-a315e17f10f5\" (UID: \"8cf1e52e-70f8-42e2-bdf3-a315e17f10f5\") "
Oct 31 17:42:28.348583 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:28.348539    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8cf1e52e-70f8-42e2-bdf3-a315e17f10f5-kube-api-access-5lc4s\" (UniqueName: \"kubernetes.io/projected/8cf1e52e-70f8-42e2-bdf3-a315e17f10f5-kube-api-access-5lc4s\") pod \"8cf1e52e-70f8-42e2-bdf3-a315e17f10f5\" (UID: \"8cf1e52e-70f8-42e2-bdf3-a315e17f10f5\") "
Oct 31 17:42:28.350489 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:28.350424    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8cf1e52e-70f8-42e2-bdf3-a315e17f10f5-home" pod "8cf1e52e-70f8-42e2-bdf3-a315e17f10f5" (UID: "8cf1e52e-70f8-42e2-bdf3-a315e17f10f5"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:42:28.350606 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:28.350438    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8cf1e52e-70f8-42e2-bdf3-a315e17f10f5-model-cache" pod "8cf1e52e-70f8-42e2-bdf3-a315e17f10f5" (UID: "8cf1e52e-70f8-42e2-bdf3-a315e17f10f5"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:42:28.350672 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:28.350655    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8cf1e52e-70f8-42e2-bdf3-a315e17f10f5-tmp-dir" pod "8cf1e52e-70f8-42e2-bdf3-a315e17f10f5" (UID: "8cf1e52e-70f8-42e2-bdf3-a315e17f10f5"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:42:28.352385 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:28.352356    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8cf1e52e-70f8-42e2-bdf3-a315e17f10f5-kube-api-access-5lc4s" pod "8cf1e52e-70f8-42e2-bdf3-a315e17f10f5" (UID: "8cf1e52e-70f8-42e2-bdf3-a315e17f10f5"). InnerVolumeSpecName "kube-api-access-5lc4s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 17:42:28.353242 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:28.352829    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8cf1e52e-70f8-42e2-bdf3-a315e17f10f5-dshm" pod "8cf1e52e-70f8-42e2-bdf3-a315e17f10f5" (UID: "8cf1e52e-70f8-42e2-bdf3-a315e17f10f5"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:42:28.355602 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:28.355576    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8cf1e52e-70f8-42e2-bdf3-a315e17f10f5-tls-certs" pod "8cf1e52e-70f8-42e2-bdf3-a315e17f10f5" (UID: "8cf1e52e-70f8-42e2-bdf3-a315e17f10f5"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 17:42:28.451710 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:28.451681    4276 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8cf1e52e-70f8-42e2-bdf3-a315e17f10f5-dshm\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:42:28.451710 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:28.451707    4276 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8cf1e52e-70f8-42e2-bdf3-a315e17f10f5-model-cache\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:42:28.451710 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:28.451717    4276 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8cf1e52e-70f8-42e2-bdf3-a315e17f10f5-home\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:42:28.451947 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:28.451726    4276 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8cf1e52e-70f8-42e2-bdf3-a315e17f10f5-tls-certs\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:42:28.451947 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:28.451734    4276 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5lc4s\" (UniqueName: \"kubernetes.io/projected/8cf1e52e-70f8-42e2-bdf3-a315e17f10f5-kube-api-access-5lc4s\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:42:28.451947 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:28.451743    4276 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8cf1e52e-70f8-42e2-bdf3-a315e17f10f5-tmp-dir\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:42:28.589167 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:28.589130    4276 generic.go:356] "Generic (PLEG): container finished" podID="8cf1e52e-70f8-42e2-bdf3-a315e17f10f5" containerID="da6e91bd6c1fc2162a5b6d8134f06ad43a10ec61ac7d785a236ab9032e95cc3c" exitCode=0
Oct 31 17:42:28.589335 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:28.589176    4276 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-0e59ca/premium-w1-0e59ca-kserve-58765f9746-fqwq7"
Oct 31 17:42:28.589335 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:28.589207    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-0e59ca/premium-w1-0e59ca-kserve-58765f9746-fqwq7" event={"ID":"8cf1e52e-70f8-42e2-bdf3-a315e17f10f5","Type":"ContainerDied","Data":"da6e91bd6c1fc2162a5b6d8134f06ad43a10ec61ac7d785a236ab9032e95cc3c"}
Oct 31 17:42:28.589335 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:28.589248    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-0e59ca/premium-w1-0e59ca-kserve-58765f9746-fqwq7" event={"ID":"8cf1e52e-70f8-42e2-bdf3-a315e17f10f5","Type":"ContainerDied","Data":"6d777ed7b88cd6d8beed8d3294233a5004202e03ff061f5a09410d34ca782e5d"}
Oct 31 17:42:28.589335 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:28.589266    4276 scope.go:122] "RemoveContainer" containerID="da6e91bd6c1fc2162a5b6d8134f06ad43a10ec61ac7d785a236ab9032e95cc3c"
Oct 31 17:42:28.599485 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:28.599464    4276 scope.go:122] "RemoveContainer" containerID="da6e91bd6c1fc2162a5b6d8134f06ad43a10ec61ac7d785a236ab9032e95cc3c"
Oct 31 17:42:28.599787 ip-10-0-129-127 kubenswrapper[4276]: E1031 17:42:28.599767    4276 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"da6e91bd6c1fc2162a5b6d8134f06ad43a10ec61ac7d785a236ab9032e95cc3c\": container with ID starting with da6e91bd6c1fc2162a5b6d8134f06ad43a10ec61ac7d785a236ab9032e95cc3c not found: ID does not exist" containerID="da6e91bd6c1fc2162a5b6d8134f06ad43a10ec61ac7d785a236ab9032e95cc3c"
Oct 31 17:42:28.599837 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:28.599795    4276 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"da6e91bd6c1fc2162a5b6d8134f06ad43a10ec61ac7d785a236ab9032e95cc3c"} err="failed to get container status \"da6e91bd6c1fc2162a5b6d8134f06ad43a10ec61ac7d785a236ab9032e95cc3c\": rpc error: code = NotFound desc = could not find container \"da6e91bd6c1fc2162a5b6d8134f06ad43a10ec61ac7d785a236ab9032e95cc3c\": container with ID starting with da6e91bd6c1fc2162a5b6d8134f06ad43a10ec61ac7d785a236ab9032e95cc3c not found: ID does not exist"
Oct 31 17:42:28.645616 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:28.645593    4276 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-0e59ca/premium-w1-0e59ca-kserve-58765f9746-fqwq7"]
Oct 31 17:42:28.659191 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:28.659171    4276 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w1-0e59ca/premium-w1-0e59ca-kserve-58765f9746-fqwq7"]
Oct 31 17:42:28.661088 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:28.661065    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-cf0a86-7bb4c5d6c5-8p8r5" podUID="1bd48c9b-b035-4931-8cf6-dbb821d3f6d5" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 31 17:42:29.160876 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:29.160842    4276 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-cf0a86-86c86d974b-hkgh2"]
Oct 31 17:42:29.161248 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:29.161219    4276 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-cf0a86-86c86d974b-hkgh2" podUID="d5dd5b19-e930-4afa-abea-db914b9f7e89" containerName="maas-api" containerID="cri-o://b06e1d7ad773d7d8c5bad3f3c6087145a9d0b7fbdae52d69468f818ee0f48a59" gracePeriod=30
Oct 31 17:42:29.307998 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:29.307963    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:42:29.368321 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:29.368294    4276 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8cf1e52e-70f8-42e2-bdf3-a315e17f10f5" path="/var/lib/kubelet/pods/8cf1e52e-70f8-42e2-bdf3-a315e17f10f5/volumes"
Oct 31 17:42:29.572463 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:29.568681    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:42:29.880573 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:29.879156    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:42:29.944363 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:29.944319    4276 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-5c446c-56f77ff564-dd7dx"]
Oct 31 17:42:29.984125 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:29.982777    4276 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-5c446c-56f77ff564-dd7dx" podUID="67250413-5beb-44da-9925-71b1735d723b" containerName="maas-api" containerID="cri-o://c1c500d0691dd556927f205943d108dc8d79ce455dc8c8c53edc3223075e4385" gracePeriod=30
Oct 31 17:42:30.181512 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:30.161456    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:42:30.332231 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:30.332186    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:42:30.394482 ip-10-0-129-127 kubenswrapper[4276]: E1031 17:42:30.346607    4276 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5dd5b19_e930_4afa_abea_db914b9f7e89.slice/crio-conmon-b06e1d7ad773d7d8c5bad3f3c6087145a9d0b7fbdae52d69468f818ee0f48a59.scope\": RecentStats: unable to find data in memory cache]"
Oct 31 17:42:30.722105 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:30.722071    4276 generic.go:356] "Generic (PLEG): container finished" podID="67250413-5beb-44da-9925-71b1735d723b" containerID="c1c500d0691dd556927f205943d108dc8d79ce455dc8c8c53edc3223075e4385" exitCode=0
Oct 31 17:42:30.722430 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:30.722408    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-5c446c-56f77ff564-dd7dx" event={"ID":"67250413-5beb-44da-9925-71b1735d723b","Type":"ContainerDied","Data":"c1c500d0691dd556927f205943d108dc8d79ce455dc8c8c53edc3223075e4385"}
Oct 31 17:42:30.777473 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:30.777368    4276 generic.go:356] "Generic (PLEG): container finished" podID="d5dd5b19-e930-4afa-abea-db914b9f7e89" containerID="b06e1d7ad773d7d8c5bad3f3c6087145a9d0b7fbdae52d69468f818ee0f48a59" exitCode=0
Oct 31 17:42:30.780103 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:30.778137    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-cf0a86-86c86d974b-hkgh2" event={"ID":"d5dd5b19-e930-4afa-abea-db914b9f7e89","Type":"ContainerDied","Data":"b06e1d7ad773d7d8c5bad3f3c6087145a9d0b7fbdae52d69468f818ee0f48a59"}
Oct 31 17:42:30.781030 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:30.781005    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:42:31.149535 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:31.149501    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:42:31.259805 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:31.259772    4276 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-5c446c-56f77ff564-dd7dx"
Oct 31 17:42:31.270104 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:31.270073    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:42:31.335287 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:31.335245    4276 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-cf0a86-86c86d974b-hkgh2"
Oct 31 17:42:31.404135 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:31.404061    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:42:31.440875 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:31.440845    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/67250413-5beb-44da-9925-71b1735d723b-metrics-certs\" (UniqueName: \"kubernetes.io/secret/67250413-5beb-44da-9925-71b1735d723b-metrics-certs\") pod \"67250413-5beb-44da-9925-71b1735d723b\" (UID: \"67250413-5beb-44da-9925-71b1735d723b\") "
Oct 31 17:42:31.441056 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:31.441029    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/67250413-5beb-44da-9925-71b1735d723b-kube-api-access-zpjt2\" (UniqueName: \"kubernetes.io/projected/67250413-5beb-44da-9925-71b1735d723b-kube-api-access-zpjt2\") pod \"67250413-5beb-44da-9925-71b1735d723b\" (UID: \"67250413-5beb-44da-9925-71b1735d723b\") "
Oct 31 17:42:31.441105 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:31.441069    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/67250413-5beb-44da-9925-71b1735d723b-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/67250413-5beb-44da-9925-71b1735d723b-maas-api-tls\") pod \"67250413-5beb-44da-9925-71b1735d723b\" (UID: \"67250413-5beb-44da-9925-71b1735d723b\") "
Oct 31 17:42:31.441105 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:31.441096    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d5dd5b19-e930-4afa-abea-db914b9f7e89-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d5dd5b19-e930-4afa-abea-db914b9f7e89-maas-api-tls\") pod \"d5dd5b19-e930-4afa-abea-db914b9f7e89\" (UID: \"d5dd5b19-e930-4afa-abea-db914b9f7e89\") "
Oct 31 17:42:31.441216 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:31.441125    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d5dd5b19-e930-4afa-abea-db914b9f7e89-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d5dd5b19-e930-4afa-abea-db914b9f7e89-openshift-service-ca\") pod \"d5dd5b19-e930-4afa-abea-db914b9f7e89\" (UID: \"d5dd5b19-e930-4afa-abea-db914b9f7e89\") "
Oct 31 17:42:31.441276 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:31.441252    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/67250413-5beb-44da-9925-71b1735d723b-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/67250413-5beb-44da-9925-71b1735d723b-openshift-service-ca\") pod \"67250413-5beb-44da-9925-71b1735d723b\" (UID: \"67250413-5beb-44da-9925-71b1735d723b\") "
Oct 31 17:42:31.441581 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:31.441551    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d5dd5b19-e930-4afa-abea-db914b9f7e89-openshift-service-ca" pod "d5dd5b19-e930-4afa-abea-db914b9f7e89" (UID: "d5dd5b19-e930-4afa-abea-db914b9f7e89"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 17:42:31.441694 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:31.441559    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d5dd5b19-e930-4afa-abea-db914b9f7e89-metrics-certs\" (UniqueName: \"kubernetes.io/secret/d5dd5b19-e930-4afa-abea-db914b9f7e89-metrics-certs\") pod \"d5dd5b19-e930-4afa-abea-db914b9f7e89\" (UID: \"d5dd5b19-e930-4afa-abea-db914b9f7e89\") "
Oct 31 17:42:31.441694 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:31.441625    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/67250413-5beb-44da-9925-71b1735d723b-openshift-service-ca" pod "67250413-5beb-44da-9925-71b1735d723b" (UID: "67250413-5beb-44da-9925-71b1735d723b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 17:42:31.441806 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:31.441715    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d5dd5b19-e930-4afa-abea-db914b9f7e89-kube-api-access-skb6n\" (UniqueName: \"kubernetes.io/projected/d5dd5b19-e930-4afa-abea-db914b9f7e89-kube-api-access-skb6n\") pod \"d5dd5b19-e930-4afa-abea-db914b9f7e89\" (UID: \"d5dd5b19-e930-4afa-abea-db914b9f7e89\") "
Oct 31 17:42:31.442604 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:31.442436    4276 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d5dd5b19-e930-4afa-abea-db914b9f7e89-openshift-service-ca\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:42:31.442604 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:31.442478    4276 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/67250413-5beb-44da-9925-71b1735d723b-openshift-service-ca\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:42:31.443657 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:31.443633    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/67250413-5beb-44da-9925-71b1735d723b-metrics-certs" pod "67250413-5beb-44da-9925-71b1735d723b" (UID: "67250413-5beb-44da-9925-71b1735d723b"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 17:42:31.443968 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:31.443942    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/67250413-5beb-44da-9925-71b1735d723b-kube-api-access-zpjt2" pod "67250413-5beb-44da-9925-71b1735d723b" (UID: "67250413-5beb-44da-9925-71b1735d723b"). InnerVolumeSpecName "kube-api-access-zpjt2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 17:42:31.444366 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:31.444342    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d5dd5b19-e930-4afa-abea-db914b9f7e89-metrics-certs" pod "d5dd5b19-e930-4afa-abea-db914b9f7e89" (UID: "d5dd5b19-e930-4afa-abea-db914b9f7e89"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 17:42:31.444506 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:31.444377    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d5dd5b19-e930-4afa-abea-db914b9f7e89-maas-api-tls" pod "d5dd5b19-e930-4afa-abea-db914b9f7e89" (UID: "d5dd5b19-e930-4afa-abea-db914b9f7e89"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 17:42:31.444506 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:31.444436    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/67250413-5beb-44da-9925-71b1735d723b-maas-api-tls" pod "67250413-5beb-44da-9925-71b1735d723b" (UID: "67250413-5beb-44da-9925-71b1735d723b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 17:42:31.444710 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:31.444692    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d5dd5b19-e930-4afa-abea-db914b9f7e89-kube-api-access-skb6n" pod "d5dd5b19-e930-4afa-abea-db914b9f7e89" (UID: "d5dd5b19-e930-4afa-abea-db914b9f7e89"). InnerVolumeSpecName "kube-api-access-skb6n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 17:42:31.543545 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:31.543509    4276 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zpjt2\" (UniqueName: \"kubernetes.io/projected/67250413-5beb-44da-9925-71b1735d723b-kube-api-access-zpjt2\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:42:31.543545 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:31.543546    4276 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/67250413-5beb-44da-9925-71b1735d723b-maas-api-tls\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:42:31.543819 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:31.543561    4276 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d5dd5b19-e930-4afa-abea-db914b9f7e89-maas-api-tls\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:42:31.543819 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:31.543573    4276 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d5dd5b19-e930-4afa-abea-db914b9f7e89-metrics-certs\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:42:31.543819 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:31.543587    4276 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-skb6n\" (UniqueName: \"kubernetes.io/projected/d5dd5b19-e930-4afa-abea-db914b9f7e89-kube-api-access-skb6n\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:42:31.543819 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:31.543600    4276 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/67250413-5beb-44da-9925-71b1735d723b-metrics-certs\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:42:31.562767 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:31.562737    4276 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w5-b78c31-6b5448c9d7-rbhfz"]
Oct 31 17:42:31.684248 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:31.684171    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:42:31.814333 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:31.813510    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:42:31.838758 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:31.838723    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-5c446c-56f77ff564-dd7dx" event={"ID":"67250413-5beb-44da-9925-71b1735d723b","Type":"ContainerDied","Data":"7a3d5ec74400f3bbfb541d4986fe636adf3255e27eb600316dac2994eef9bdaa"}
Oct 31 17:42:31.838972 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:31.838795    4276 scope.go:122] "RemoveContainer" containerID="c1c500d0691dd556927f205943d108dc8d79ce455dc8c8c53edc3223075e4385"
Oct 31 17:42:31.838972 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:31.838913    4276 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-5c446c-56f77ff564-dd7dx"
Oct 31 17:42:31.841885 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:31.841866    4276 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-cf0a86-86c86d974b-hkgh2"
Oct 31 17:42:31.842106 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:31.841940    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-cf0a86-86c86d974b-hkgh2" event={"ID":"d5dd5b19-e930-4afa-abea-db914b9f7e89","Type":"ContainerDied","Data":"66503d2e55b63126fd0bbc4e38537377362e4e52228f06c247036f35fba1683f"}
Oct 31 17:42:31.851216 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:31.851194    4276 scope.go:122] "RemoveContainer" containerID="b06e1d7ad773d7d8c5bad3f3c6087145a9d0b7fbdae52d69468f818ee0f48a59"
Oct 31 17:42:31.899129 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:31.899087    4276 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-5c446c-56f77ff564-dd7dx"]
Oct 31 17:42:31.904009 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:31.903981    4276 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-5c446c-56f77ff564-dd7dx"]
Oct 31 17:42:31.939485 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:31.939390    4276 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-cf0a86-86c86d974b-hkgh2"]
Oct 31 17:42:31.945035 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:31.945007    4276 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-cf0a86-86c86d974b-hkgh2"]
Oct 31 17:42:32.356864 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:32.356826    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:42:32.487693 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:32.487610    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:42:32.611891 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:32.611859    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:42:32.853627 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:32.853593    4276 generic.go:356] "Generic (PLEG): container finished" podID="00caa007-32e1-4ff0-a5d5-71ab1b73eca9" containerID="5fa1879574b282c012c1a5593a44970aaff45dac29486f07a895d00e879a92c2" exitCode=2
Oct 31 17:42:32.853812 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:32.853664    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-xxspc" event={"ID":"00caa007-32e1-4ff0-a5d5-71ab1b73eca9","Type":"ContainerDied","Data":"5fa1879574b282c012c1a5593a44970aaff45dac29486f07a895d00e879a92c2"}
Oct 31 17:42:32.854174 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:32.854156    4276 scope.go:122] "RemoveContainer" containerID="5fa1879574b282c012c1a5593a44970aaff45dac29486f07a895d00e879a92c2"
Oct 31 17:42:32.855926 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:32.855893    4276 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 17:42:32.855997 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:32.855976    4276 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 17:42:33.066005 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:33.065967    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:42:33.417885 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:33.417849    4276 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="67250413-5beb-44da-9925-71b1735d723b" path="/var/lib/kubelet/pods/67250413-5beb-44da-9925-71b1735d723b/volumes"
Oct 31 17:42:33.418584 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:33.418562    4276 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d5dd5b19-e930-4afa-abea-db914b9f7e89" path="/var/lib/kubelet/pods/d5dd5b19-e930-4afa-abea-db914b9f7e89/volumes"
Oct 31 17:42:33.948469 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:33.947893    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-xxspc" event={"ID":"00caa007-32e1-4ff0-a5d5-71ab1b73eca9","Type":"ContainerStarted","Data":"594e00325125304f047f781d286f5fba5653108de898b1385b8fbe3f97737124"}
Oct 31 17:42:33.952465 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:33.949291    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-xxspc"
Oct 31 17:42:33.958535 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:33.957361    4276 patch_prober.go:29] interesting pod/istiod-openshift-gateway-7ddd9cfdc8-xxspc container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 31 17:42:33.958535 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:33.957431    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-xxspc" podUID="00caa007-32e1-4ff0-a5d5-71ab1b73eca9" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 17:42:34.074882 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:34.074848    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:42:34.580485 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:34.576542    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:42:34.794469 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:34.744344    4276 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w6-d80080/distinct-w6-d80080-kserve-664576c8f5-6jc4d"]
Oct 31 17:42:34.876525 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:34.868997    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:42:35.185525 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:35.174540    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ee816e/premium-w0-ee816e-kserve-65c86cd6db-lnt9p" podUID="e536c902-52a9-4b36-a2ad-eb5500781a5e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.49:8000/ready\": context deadline exceeded"
Oct 31 17:42:35.223918 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:35.197396    4276 generic.go:356] "Generic (PLEG): container finished" podID="1bd48c9b-b035-4931-8cf6-dbb821d3f6d5" containerID="fe5960e6e112614bcdb6c6cd41714ed850389b563a08d6c938b3db43b17ebfc4" exitCode=0
Oct 31 17:42:35.223918 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:35.205547    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-cf0a86-7bb4c5d6c5-8p8r5" event={"ID":"1bd48c9b-b035-4931-8cf6-dbb821d3f6d5","Type":"ContainerDied","Data":"fe5960e6e112614bcdb6c6cd41714ed850389b563a08d6c938b3db43b17ebfc4"}
Oct 31 17:42:35.224499 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:35.224472    4276 patch_prober.go:29] interesting pod/istiod-openshift-gateway-7ddd9cfdc8-xxspc container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 31 17:42:35.224747 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:35.224705    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-xxspc" podUID="00caa007-32e1-4ff0-a5d5-71ab1b73eca9" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 17:42:35.591855 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:35.591810    4276 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w6-d80080/distinct-w6-d80080-kserve-664576c8f5-6jc4d" podUID="b9c2f445-a191-4b00-802a-1b0ded068411" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 17:42:35.608636 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:35.607369    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-d80080/distinct-w6-d80080-kserve-664576c8f5-6jc4d" podUID="b9c2f445-a191-4b00-802a-1b0ded068411" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": context deadline exceeded"
Oct 31 17:42:35.754138 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:35.754104    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:42:35.982108 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:35.982002    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:42:36.233536 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:36.211803    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ee816e/premium-w0-ee816e-kserve-65c86cd6db-lnt9p" podUID="e536c902-52a9-4b36-a2ad-eb5500781a5e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.49:8000/ready\": context deadline exceeded"
Oct 31 17:42:36.234799 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:36.234758    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-ee816e/premium-w0-ee816e-kserve-65c86cd6db-lnt9p"
Oct 31 17:42:36.508522 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:36.486483    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-1c0bfd/distinct-2-w4-1c0bfd-kserve-847bcf6cf4-lt8ls" podUID="f8a45b7c-e5ae-4321-ad81-fd6353b2495d" containerName="main" probeResult="failure" output="Get \"https://10.133.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 17:42:36.576907 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:36.572576    4276 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-b78c31-6b5448c9d7-rbhfz" podUID="f7894ac3-7f53-4a3f-8d20-17359e24f69e" containerName="payload-pre-processing" containerID="cri-o://9d88d61861ce7bf6e5f3c5befa55815e5e0b74178a89eccd2406911943fbe6b5" gracePeriod=25
Oct 31 17:42:36.639763 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:36.639720    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-d80080/distinct-w6-d80080-kserve-664576c8f5-6jc4d" podUID="b9c2f445-a191-4b00-802a-1b0ded068411" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": context deadline exceeded"
Oct 31 17:42:36.640988 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:36.640963    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-d80080/distinct-w6-d80080-kserve-664576c8f5-6jc4d"
Oct 31 17:42:37.247290 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:37.247235    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ee816e/premium-w0-ee816e-kserve-65c86cd6db-lnt9p" podUID="e536c902-52a9-4b36-a2ad-eb5500781a5e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.49:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 17:42:37.586786 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:37.570966    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-1c0bfd/distinct-2-w4-1c0bfd-kserve-847bcf6cf4-lt8ls" podUID="f8a45b7c-e5ae-4321-ad81-fd6353b2495d" containerName="main" probeResult="failure" output="Get \"https://10.133.0.67:8000/ready\": context deadline exceeded"
Oct 31 17:42:37.678124 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:37.678074    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-d80080/distinct-w6-d80080-kserve-664576c8f5-6jc4d" podUID="b9c2f445-a191-4b00-802a-1b0ded068411" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 17:42:37.928673 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:37.928595    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-1c0bfd/distinct-2-w4-1c0bfd-kserve-847bcf6cf4-lt8ls"
Oct 31 17:42:37.955912 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:37.955068    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:42:38.133713 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:38.129688    4276 patch_prober.go:29] interesting pod/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-c5dc6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.36:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 17:42:38.133713 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:38.129757    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-c5dc6" podUID="22af22dd-1130-409b-9a17-05229dc0de11" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.36:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 17:42:38.226990 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:38.226866    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:42:38.227359 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:38.227335    4276 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-cf0a86-7bb4c5d6c5-8p8r5"
Oct 31 17:42:38.249466 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:38.245664    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-xxspc"
Oct 31 17:42:38.315469 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:38.312544    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ee816e/premium-w0-ee816e-kserve-65c86cd6db-lnt9p" podUID="e536c902-52a9-4b36-a2ad-eb5500781a5e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.49:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 31 17:42:38.315469 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:38.313510    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1bd48c9b-b035-4931-8cf6-dbb821d3f6d5-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1bd48c9b-b035-4931-8cf6-dbb821d3f6d5-plugins-config-volume\") pod \"1bd48c9b-b035-4931-8cf6-dbb821d3f6d5\" (UID: \"1bd48c9b-b035-4931-8cf6-dbb821d3f6d5\") "
Oct 31 17:42:38.315469 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:38.313852    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1bd48c9b-b035-4931-8cf6-dbb821d3f6d5-kube-api-access-nj7st\" (UniqueName: \"kubernetes.io/projected/1bd48c9b-b035-4931-8cf6-dbb821d3f6d5-kube-api-access-nj7st\") pod \"1bd48c9b-b035-4931-8cf6-dbb821d3f6d5\" (UID: \"1bd48c9b-b035-4931-8cf6-dbb821d3f6d5\") "
Oct 31 17:42:38.319477 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:38.316854    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1bd48c9b-b035-4931-8cf6-dbb821d3f6d5-plugins-config-volume" pod "1bd48c9b-b035-4931-8cf6-dbb821d3f6d5" (UID: "1bd48c9b-b035-4931-8cf6-dbb821d3f6d5"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 17:42:38.697327 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:38.674121    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-1c0bfd/distinct-2-w4-1c0bfd-kserve-847bcf6cf4-lt8ls" podUID="f8a45b7c-e5ae-4321-ad81-fd6353b2495d" containerName="main" probeResult="failure" output="Get \"https://10.133.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 17:42:38.777682 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:38.751834    4276 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1bd48c9b-b035-4931-8cf6-dbb821d3f6d5-plugins-config-volume\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:42:38.793475 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:38.788201    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:42:38.946467 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:38.941598    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-d80080/distinct-w6-d80080-kserve-664576c8f5-6jc4d" podUID="b9c2f445-a191-4b00-802a-1b0ded068411" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": context deadline exceeded"
Oct 31 17:42:39.072522 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:39.020291    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:42:39.072522 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:39.039491    4276 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-c5dc6"]
Oct 31 17:42:39.072522 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:39.039746    4276 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-c5dc6" podUID="22af22dd-1130-409b-9a17-05229dc0de11" containerName="istio-proxy" containerID="cri-o://bc9404ff4e456fc89595255741cc75406b890990ce42d69b922760a2527d1feb" gracePeriod=30
Oct 31 17:42:39.102608 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:39.074812    4276 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-bwdwd"]
Oct 31 17:42:39.102608 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:39.075073    4276 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-bwdwd" podUID="7ec5066a-6b4d-4669-97f5-8a46a9c99a1b" containerName="istio-proxy" containerID="cri-o://a8e133dd63fb045cd89dea6db2b65aef167a4a19b83a9e11d66fbf8406af7271" gracePeriod=30
Oct 31 17:42:39.138635 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:39.138594    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1bd48c9b-b035-4931-8cf6-dbb821d3f6d5-kube-api-access-nj7st" pod "1bd48c9b-b035-4931-8cf6-dbb821d3f6d5" (UID: "1bd48c9b-b035-4931-8cf6-dbb821d3f6d5"). InnerVolumeSpecName "kube-api-access-nj7st". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 17:42:39.305512 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:39.301528    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:42:39.490585 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:39.318113    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ee816e/premium-w0-ee816e-kserve-65c86cd6db-lnt9p" podUID="e536c902-52a9-4b36-a2ad-eb5500781a5e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.49:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 17:42:39.556511 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:39.318401    4276 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-rfwx5"]
Oct 31 17:42:39.556511 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:39.552851    4276 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-8867c"]
Oct 31 17:42:39.556511 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:39.552889    4276 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-nqt2x"]
Oct 31 17:42:39.559119 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:39.400287    4276 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nj7st\" (UniqueName: \"kubernetes.io/projected/1bd48c9b-b035-4931-8cf6-dbb821d3f6d5-kube-api-access-nj7st\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:42:39.559119 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:39.557060    4276 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-rfwx5" podUID="5155d374-e42a-4c40-9485-935443ede2c5" containerName="istio-proxy" containerID="cri-o://0707bf2ff1d341d72693257e316834b57c76ced9cad6fe5df2ec01e0ec9c803d" gracePeriod=30
Oct 31 17:42:39.559119 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:39.557815    4276 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-8867c" podUID="d1bf33fd-97d5-462d-8c04-e2697fcf1b05" containerName="istio-proxy" containerID="cri-o://9224bcb42e7be2c812e6db5e1f87d5b65af67a6e6c1cea7735c44f82d264ca7b" gracePeriod=30
Oct 31 17:42:39.559119 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:39.558051    4276 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-nqt2x" podUID="f2899dc7-04e8-40b9-ac3f-6cf6cb019020" containerName="istio-proxy" containerID="cri-o://7ecff18217515087184c4171450b64676aee451bb8da3d527e4059ecb26fcfaf" gracePeriod=30
Oct 31 17:42:39.893952 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:39.797034    4276 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-cf0a86-7bb4c5d6c5-8p8r5"
Oct 31 17:42:39.893952 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:39.805737    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-1c0bfd/distinct-2-w4-1c0bfd-kserve-847bcf6cf4-lt8ls" podUID="f8a45b7c-e5ae-4321-ad81-fd6353b2495d" containerName="main" probeResult="failure" output="Get \"https://10.133.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 17:42:40.004497 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:39.947629    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-d80080/distinct-w6-d80080-kserve-664576c8f5-6jc4d" podUID="b9c2f445-a191-4b00-802a-1b0ded068411" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": context deadline exceeded"
Oct 31 17:42:40.008710 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:40.008609    4276 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-hl287"]
Oct 31 17:42:40.008710 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:40.008700    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-cf0a86-7bb4c5d6c5-8p8r5" event={"ID":"1bd48c9b-b035-4931-8cf6-dbb821d3f6d5","Type":"ContainerDied","Data":"66b7496e33f41a6800ca712c00b0a7967ff7539d57fe57eea77cbb80f482877f"}
Oct 31 17:42:40.049468 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:40.023972    4276 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-hl287" podUID="daf53be5-f832-4696-96b7-310c33b09f03" containerName="istio-proxy" containerID="cri-o://247fc7436ec961ce182dd395a926a2c8ef25f7cc3d398880c0503bd8774291ce" gracePeriod=30
Oct 31 17:42:40.263702 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:40.101501    4276 patch_prober.go:29] interesting pod/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-wm2s9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 17:42:40.263702 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:40.101582    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-wm2s9" podUID="34243752-91ce-4e92-b0a6-72490b1f7c9e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 17:42:40.263702 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:40.222611    4276 scope.go:122] "RemoveContainer" containerID="fe5960e6e112614bcdb6c6cd41714ed850389b563a08d6c938b3db43b17ebfc4"
Oct 31 17:42:40.377469 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:40.377169    4276 patch_prober.go:29] interesting pod/e2e-worker-w6-d80080-openshift-default-956cddf57-25kdb container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 17:42:40.377663 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:40.377627    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-25kdb" podUID="6ec6accb-4b2a-45a8-bab2-fe65f2f5df2d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 17:42:40.493468 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:40.492421    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ee816e/premium-w0-ee816e-kserve-65c86cd6db-lnt9p" podUID="e536c902-52a9-4b36-a2ad-eb5500781a5e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.49:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 17:42:40.780169 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:40.780130    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:42:40.886068 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:40.801794    4276 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-jxpd8"]
Oct 31 17:42:40.886706 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:40.886669    4276 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-jxpd8" podUID="9d24572d-ec8f-4108-a83e-704c5ae976ff" containerName="istio-proxy" containerID="cri-o://0fd95e192f8f3065b709ebc370709b230b43de2bd09f51864596dd4bad244101" gracePeriod=30
Oct 31 17:42:40.886991 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:40.811527    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-1c0bfd/distinct-2-w4-1c0bfd-kserve-847bcf6cf4-lt8ls" podUID="f8a45b7c-e5ae-4321-ad81-fd6353b2495d" containerName="main" probeResult="failure" output="Get \"https://10.133.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 17:42:40.963540 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:40.948954    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-d80080/distinct-w6-d80080-kserve-664576c8f5-6jc4d" podUID="b9c2f445-a191-4b00-802a-1b0ded068411" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 17:42:41.142754 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:41.142718    4276 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-cf0a86-7bb4c5d6c5-8p8r5"]
Oct 31 17:42:41.218586 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:41.218494    4276 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-cf0a86-7bb4c5d6c5-8p8r5"]
Oct 31 17:42:41.539119 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:41.538615    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ee816e/premium-w0-ee816e-kserve-65c86cd6db-lnt9p" podUID="e536c902-52a9-4b36-a2ad-eb5500781a5e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.49:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 17:42:41.564626 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:41.549022    4276 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-1c0bfd/distinct-2-w4-1c0bfd-kserve-847bcf6cf4-lt8ls" podUID="f8a45b7c-e5ae-4321-ad81-fd6353b2495d" containerName="main" containerID="cri-o://14f49c1216072040ea6045823346d1158f41c921240b17b9e18bf79d9258fada" gracePeriod=45
Oct 31 17:42:41.598465 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:41.594577    4276 patch_prober.go:29] interesting pod/e2e-worker-w6-d80080-openshift-default-956cddf57-cfvtd container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 17:42:41.598465 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:41.594632    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-cfvtd" podUID="bc5595c7-f428-4092-ac29-a30af2e01065" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 17:42:41.899912 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:41.896748    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-1c0bfd/distinct-2-w4-1c0bfd-kserve-847bcf6cf4-lt8ls" podUID="f8a45b7c-e5ae-4321-ad81-fd6353b2495d" containerName="main" probeResult="failure" output="Get \"https://10.133.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 17:42:42.189429 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:42.131929    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:42:42.205986 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:42.205949    4276 patch_prober.go:29] interesting pod/e2e-worker-w2-5c446c-openshift-default-797c6dc855-nqt2x container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 31 17:42:42.206261 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:42.206230    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-nqt2x" podUID="f2899dc7-04e8-40b9-ac3f-6cf6cb019020" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 17:42:42.206690 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:42.206664    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-d80080/distinct-w6-d80080-kserve-664576c8f5-6jc4d" podUID="b9c2f445-a191-4b00-802a-1b0ded068411" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 17:42:42.240296 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:42.240262    4276 patch_prober.go:29] interesting pod/e2e-worker-w2-5c446c-openshift-default-797c6dc855-hl287 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 31 17:42:42.240568 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:42.240533    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-hl287" podUID="daf53be5-f832-4696-96b7-310c33b09f03" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 17:42:42.379907 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:42.379867    4276 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1bd48c9b-b035-4931-8cf6-dbb821d3f6d5" path="/var/lib/kubelet/pods/1bd48c9b-b035-4931-8cf6-dbb821d3f6d5/volumes"
Oct 31 17:42:42.399147 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:42.399116    4276 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w6-d80080-59b65f4495-h6gz8"]
Oct 31 17:42:42.492269 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:42.472654    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-1c0bfd/distinct-2-w4-1c0bfd-kserve-847bcf6cf4-lt8ls" podUID="f8a45b7c-e5ae-4321-ad81-fd6353b2495d" containerName="main" probeResult="failure" output="Get \"https://10.133.0.67:8000/ready\": EOF"
Oct 31 17:42:42.492269 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:42.473222    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-ee816e/premium-w0-ee816e-kserve-65c86cd6db-lnt9p"
Oct 31 17:42:42.492269 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:42.476350    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:42:42.536254 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:42.534065    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-1c0bfd/distinct-2-w4-1c0bfd-kserve-847bcf6cf4-lt8ls" podUID="f8a45b7c-e5ae-4321-ad81-fd6353b2495d" containerName="main" probeResult="failure" output="Get \"https://10.133.0.67:8000/ready\": dial tcp 10.133.0.67:8000: connect: connection refused"
Oct 31 17:42:42.549746 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:42.549720    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-d80080/distinct-w6-d80080-kserve-664576c8f5-6jc4d"
Oct 31 17:42:42.562330 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:42.553195    4276 patch_prober.go:29] interesting pod/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-8867c container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 31 17:42:42.562633 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:42.562603    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-8867c" podUID="d1bf33fd-97d5-462d-8c04-e2697fcf1b05" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 17:42:42.588808 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:42.588693    4276 patch_prober.go:29] interesting pod/e2e-worker-w2-5c446c-openshift-default-797c6dc855-jxpd8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 31 17:42:42.588808 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:42.588748    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-jxpd8" podUID="9d24572d-ec8f-4108-a83e-704c5ae976ff" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 17:42:42.589851 ip-10-0-129-127 kubenswrapper[4276]: E1031 17:42:42.589823    4276 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tg5qk" containerName="extractor"
Oct 31 17:42:42.958464 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:42.955021    4276 patch_prober.go:29] interesting pod/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-bwdwd container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 31 17:42:42.958464 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:42.955076    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-bwdwd" podUID="7ec5066a-6b4d-4669-97f5-8a46a9c99a1b" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 17:42:43.012564 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:43.012532    4276 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-1c0bfd/distinct-2-w4-1c0bfd-kserve-847bcf6cf4-lt8ls"
Oct 31 17:42:43.058126 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:43.058100    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f8a45b7c-e5ae-4321-ad81-fd6353b2495d-dshm\" (UniqueName: \"kubernetes.io/empty-dir/f8a45b7c-e5ae-4321-ad81-fd6353b2495d-dshm\") pod \"f8a45b7c-e5ae-4321-ad81-fd6353b2495d\" (UID: \"f8a45b7c-e5ae-4321-ad81-fd6353b2495d\") "
Oct 31 17:42:43.058278 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:43.058215    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f8a45b7c-e5ae-4321-ad81-fd6353b2495d-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f8a45b7c-e5ae-4321-ad81-fd6353b2495d-model-cache\") pod \"f8a45b7c-e5ae-4321-ad81-fd6353b2495d\" (UID: \"f8a45b7c-e5ae-4321-ad81-fd6353b2495d\") "
Oct 31 17:42:43.058278 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:43.058254    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f8a45b7c-e5ae-4321-ad81-fd6353b2495d-tls-certs\" (UniqueName: \"kubernetes.io/secret/f8a45b7c-e5ae-4321-ad81-fd6353b2495d-tls-certs\") pod \"f8a45b7c-e5ae-4321-ad81-fd6353b2495d\" (UID: \"f8a45b7c-e5ae-4321-ad81-fd6353b2495d\") "
Oct 31 17:42:43.058375 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:43.058359    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f8a45b7c-e5ae-4321-ad81-fd6353b2495d-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f8a45b7c-e5ae-4321-ad81-fd6353b2495d-tmp-dir\") pod \"f8a45b7c-e5ae-4321-ad81-fd6353b2495d\" (UID: \"f8a45b7c-e5ae-4321-ad81-fd6353b2495d\") "
Oct 31 17:42:43.058431 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:43.058395    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f8a45b7c-e5ae-4321-ad81-fd6353b2495d-kube-api-access-9l4fz\" (UniqueName: \"kubernetes.io/projected/f8a45b7c-e5ae-4321-ad81-fd6353b2495d-kube-api-access-9l4fz\") pod \"f8a45b7c-e5ae-4321-ad81-fd6353b2495d\" (UID: \"f8a45b7c-e5ae-4321-ad81-fd6353b2495d\") "
Oct 31 17:42:43.058541 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:43.058490    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f8a45b7c-e5ae-4321-ad81-fd6353b2495d-home\" (UniqueName: \"kubernetes.io/empty-dir/f8a45b7c-e5ae-4321-ad81-fd6353b2495d-home\") pod \"f8a45b7c-e5ae-4321-ad81-fd6353b2495d\" (UID: \"f8a45b7c-e5ae-4321-ad81-fd6353b2495d\") "
Oct 31 17:42:43.058541 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:43.058502    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f8a45b7c-e5ae-4321-ad81-fd6353b2495d-model-cache" pod "f8a45b7c-e5ae-4321-ad81-fd6353b2495d" (UID: "f8a45b7c-e5ae-4321-ad81-fd6353b2495d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:42:43.058726 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:43.058705    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f8a45b7c-e5ae-4321-ad81-fd6353b2495d-tmp-dir" pod "f8a45b7c-e5ae-4321-ad81-fd6353b2495d" (UID: "f8a45b7c-e5ae-4321-ad81-fd6353b2495d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:42:43.058891 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:43.058869    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f8a45b7c-e5ae-4321-ad81-fd6353b2495d-home" pod "f8a45b7c-e5ae-4321-ad81-fd6353b2495d" (UID: "f8a45b7c-e5ae-4321-ad81-fd6353b2495d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:42:43.059153 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:43.059136    4276 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f8a45b7c-e5ae-4321-ad81-fd6353b2495d-home\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:42:43.059242 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:43.059155    4276 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f8a45b7c-e5ae-4321-ad81-fd6353b2495d-model-cache\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:42:43.059242 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:43.059165    4276 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f8a45b7c-e5ae-4321-ad81-fd6353b2495d-tmp-dir\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:42:43.061023 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:43.060994    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f8a45b7c-e5ae-4321-ad81-fd6353b2495d-tls-certs" pod "f8a45b7c-e5ae-4321-ad81-fd6353b2495d" (UID: "f8a45b7c-e5ae-4321-ad81-fd6353b2495d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 17:42:43.061227 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:43.061198    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f8a45b7c-e5ae-4321-ad81-fd6353b2495d-kube-api-access-9l4fz" pod "f8a45b7c-e5ae-4321-ad81-fd6353b2495d" (UID: "f8a45b7c-e5ae-4321-ad81-fd6353b2495d"). InnerVolumeSpecName "kube-api-access-9l4fz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 17:42:43.061349 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:43.061332    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f8a45b7c-e5ae-4321-ad81-fd6353b2495d-dshm" pod "f8a45b7c-e5ae-4321-ad81-fd6353b2495d" (UID: "f8a45b7c-e5ae-4321-ad81-fd6353b2495d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:42:43.104134 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:43.104099    4276 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-ee816e/premium-w0-ee816e-kserve-65c86cd6db-lnt9p" podUID="e536c902-52a9-4b36-a2ad-eb5500781a5e" containerName="main" containerID="cri-o://36c6d2d02f285224be746e52aa2bbb9d39a797d025543f7867fdd95f94cb5696" gracePeriod=45
Oct 31 17:42:43.160870 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:43.160833    4276 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f8a45b7c-e5ae-4321-ad81-fd6353b2495d-tls-certs\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:42:43.160870 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:43.160862    4276 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9l4fz\" (UniqueName: \"kubernetes.io/projected/f8a45b7c-e5ae-4321-ad81-fd6353b2495d-kube-api-access-9l4fz\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:42:43.160870 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:43.160872    4276 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f8a45b7c-e5ae-4321-ad81-fd6353b2495d-dshm\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:42:43.173898 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:43.173859    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ee816e/premium-w0-ee816e-kserve-65c86cd6db-lnt9p" podUID="e536c902-52a9-4b36-a2ad-eb5500781a5e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.49:8000/ready\": dial tcp 10.133.0.49:8000: connect: connection refused"
Oct 31 17:42:43.275247 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:43.275217    4276 generic.go:356] "Generic (PLEG): container finished" podID="e536c902-52a9-4b36-a2ad-eb5500781a5e" containerID="36c6d2d02f285224be746e52aa2bbb9d39a797d025543f7867fdd95f94cb5696" exitCode=0
Oct 31 17:42:43.275681 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:43.275291    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-ee816e/premium-w0-ee816e-kserve-65c86cd6db-lnt9p" event={"ID":"e536c902-52a9-4b36-a2ad-eb5500781a5e","Type":"ContainerDied","Data":"36c6d2d02f285224be746e52aa2bbb9d39a797d025543f7867fdd95f94cb5696"}
Oct 31 17:42:43.283972 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:43.277000    4276 generic.go:356] "Generic (PLEG): container finished" podID="f8a45b7c-e5ae-4321-ad81-fd6353b2495d" containerID="14f49c1216072040ea6045823346d1158f41c921240b17b9e18bf79d9258fada" exitCode=0
Oct 31 17:42:43.283972 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:43.277129    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-1c0bfd/distinct-2-w4-1c0bfd-kserve-847bcf6cf4-lt8ls" event={"ID":"f8a45b7c-e5ae-4321-ad81-fd6353b2495d","Type":"ContainerDied","Data":"14f49c1216072040ea6045823346d1158f41c921240b17b9e18bf79d9258fada"}
Oct 31 17:42:43.283972 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:43.277160    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-1c0bfd/distinct-2-w4-1c0bfd-kserve-847bcf6cf4-lt8ls" event={"ID":"f8a45b7c-e5ae-4321-ad81-fd6353b2495d","Type":"ContainerDied","Data":"fe1cf9d5110f8b3acca81bf656e42f5b6911df2bf00432993b110404c38d70b4"}
Oct 31 17:42:43.283972 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:43.277182    4276 scope.go:122] "RemoveContainer" containerID="14f49c1216072040ea6045823346d1158f41c921240b17b9e18bf79d9258fada"
Oct 31 17:42:43.283972 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:43.277287    4276 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-1c0bfd/distinct-2-w4-1c0bfd-kserve-847bcf6cf4-lt8ls"
Oct 31 17:42:43.296148 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:43.296127    4276 scope.go:122] "RemoveContainer" containerID="14f49c1216072040ea6045823346d1158f41c921240b17b9e18bf79d9258fada"
Oct 31 17:42:43.296515 ip-10-0-129-127 kubenswrapper[4276]: E1031 17:42:43.296492    4276 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"14f49c1216072040ea6045823346d1158f41c921240b17b9e18bf79d9258fada\": container with ID starting with 14f49c1216072040ea6045823346d1158f41c921240b17b9e18bf79d9258fada not found: ID does not exist" containerID="14f49c1216072040ea6045823346d1158f41c921240b17b9e18bf79d9258fada"
Oct 31 17:42:43.296600 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:43.296528    4276 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"14f49c1216072040ea6045823346d1158f41c921240b17b9e18bf79d9258fada"} err="failed to get container status \"14f49c1216072040ea6045823346d1158f41c921240b17b9e18bf79d9258fada\": rpc error: code = NotFound desc = could not find container \"14f49c1216072040ea6045823346d1158f41c921240b17b9e18bf79d9258fada\": container with ID starting with 14f49c1216072040ea6045823346d1158f41c921240b17b9e18bf79d9258fada not found: ID does not exist"
Oct 31 17:42:43.344585 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:43.344515    4276 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-1c0bfd/distinct-2-w4-1c0bfd-kserve-847bcf6cf4-lt8ls"]
Oct 31 17:42:43.349338 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:43.349313    4276 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-1c0bfd/distinct-2-w4-1c0bfd-kserve-847bcf6cf4-lt8ls"]
Oct 31 17:42:43.370126 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:43.370100    4276 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f8a45b7c-e5ae-4321-ad81-fd6353b2495d" path="/var/lib/kubelet/pods/f8a45b7c-e5ae-4321-ad81-fd6353b2495d/volumes"
Oct 31 17:42:43.381352 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:43.381331    4276 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-ee816e/premium-w0-ee816e-kserve-65c86cd6db-lnt9p"
Oct 31 17:42:43.464620 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:43.464586    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e536c902-52a9-4b36-a2ad-eb5500781a5e-tls-certs\" (UniqueName: \"kubernetes.io/secret/e536c902-52a9-4b36-a2ad-eb5500781a5e-tls-certs\") pod \"e536c902-52a9-4b36-a2ad-eb5500781a5e\" (UID: \"e536c902-52a9-4b36-a2ad-eb5500781a5e\") "
Oct 31 17:42:43.464800 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:43.464635    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e536c902-52a9-4b36-a2ad-eb5500781a5e-kube-api-access-4jwzd\" (UniqueName: \"kubernetes.io/projected/e536c902-52a9-4b36-a2ad-eb5500781a5e-kube-api-access-4jwzd\") pod \"e536c902-52a9-4b36-a2ad-eb5500781a5e\" (UID: \"e536c902-52a9-4b36-a2ad-eb5500781a5e\") "
Oct 31 17:42:43.464800 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:43.464711    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e536c902-52a9-4b36-a2ad-eb5500781a5e-home\" (UniqueName: \"kubernetes.io/empty-dir/e536c902-52a9-4b36-a2ad-eb5500781a5e-home\") pod \"e536c902-52a9-4b36-a2ad-eb5500781a5e\" (UID: \"e536c902-52a9-4b36-a2ad-eb5500781a5e\") "
Oct 31 17:42:43.464800 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:43.464738    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e536c902-52a9-4b36-a2ad-eb5500781a5e-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e536c902-52a9-4b36-a2ad-eb5500781a5e-tmp-dir\") pod \"e536c902-52a9-4b36-a2ad-eb5500781a5e\" (UID: \"e536c902-52a9-4b36-a2ad-eb5500781a5e\") "
Oct 31 17:42:43.464983 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:43.464803    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e536c902-52a9-4b36-a2ad-eb5500781a5e-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e536c902-52a9-4b36-a2ad-eb5500781a5e-model-cache\") pod \"e536c902-52a9-4b36-a2ad-eb5500781a5e\" (UID: \"e536c902-52a9-4b36-a2ad-eb5500781a5e\") "
Oct 31 17:42:43.464983 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:43.464831    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e536c902-52a9-4b36-a2ad-eb5500781a5e-dshm\" (UniqueName: \"kubernetes.io/empty-dir/e536c902-52a9-4b36-a2ad-eb5500781a5e-dshm\") pod \"e536c902-52a9-4b36-a2ad-eb5500781a5e\" (UID: \"e536c902-52a9-4b36-a2ad-eb5500781a5e\") "
Oct 31 17:42:43.465285 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:43.465253    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e536c902-52a9-4b36-a2ad-eb5500781a5e-tmp-dir" pod "e536c902-52a9-4b36-a2ad-eb5500781a5e" (UID: "e536c902-52a9-4b36-a2ad-eb5500781a5e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:42:43.465572 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:43.465548    4276 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e536c902-52a9-4b36-a2ad-eb5500781a5e-tmp-dir\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:42:43.465788 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:43.465762    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e536c902-52a9-4b36-a2ad-eb5500781a5e-model-cache" pod "e536c902-52a9-4b36-a2ad-eb5500781a5e" (UID: "e536c902-52a9-4b36-a2ad-eb5500781a5e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:42:43.474872 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:43.466703    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e536c902-52a9-4b36-a2ad-eb5500781a5e-home" pod "e536c902-52a9-4b36-a2ad-eb5500781a5e" (UID: "e536c902-52a9-4b36-a2ad-eb5500781a5e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:42:43.474872 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:43.468103    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e536c902-52a9-4b36-a2ad-eb5500781a5e-tls-certs" pod "e536c902-52a9-4b36-a2ad-eb5500781a5e" (UID: "e536c902-52a9-4b36-a2ad-eb5500781a5e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 17:42:43.474872 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:43.469122    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e536c902-52a9-4b36-a2ad-eb5500781a5e-dshm" pod "e536c902-52a9-4b36-a2ad-eb5500781a5e" (UID: "e536c902-52a9-4b36-a2ad-eb5500781a5e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:42:43.474872 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:43.469528    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e536c902-52a9-4b36-a2ad-eb5500781a5e-kube-api-access-4jwzd" pod "e536c902-52a9-4b36-a2ad-eb5500781a5e" (UID: "e536c902-52a9-4b36-a2ad-eb5500781a5e"). InnerVolumeSpecName "kube-api-access-4jwzd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 17:42:43.567106 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:43.567077    4276 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e536c902-52a9-4b36-a2ad-eb5500781a5e-dshm\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:42:43.567106 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:43.567104    4276 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e536c902-52a9-4b36-a2ad-eb5500781a5e-tls-certs\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:42:43.567106 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:43.567114    4276 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4jwzd\" (UniqueName: \"kubernetes.io/projected/e536c902-52a9-4b36-a2ad-eb5500781a5e-kube-api-access-4jwzd\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:42:43.567106 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:43.567126    4276 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e536c902-52a9-4b36-a2ad-eb5500781a5e-home\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:42:43.567506 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:43.567140    4276 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e536c902-52a9-4b36-a2ad-eb5500781a5e-model-cache\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:42:44.004666 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:44.004633    4276 patch_prober.go:29] interesting pod/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-rfwx5 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 31 17:42:44.004833 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:44.004698    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-rfwx5" podUID="5155d374-e42a-4c40-9485-935443ede2c5" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 17:42:44.284853 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:44.284823    4276 generic.go:356] "Generic (PLEG): container finished" podID="7ec5066a-6b4d-4669-97f5-8a46a9c99a1b" containerID="a8e133dd63fb045cd89dea6db2b65aef167a4a19b83a9e11d66fbf8406af7271" exitCode=0
Oct 31 17:42:44.285266 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:44.284898    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-bwdwd" event={"ID":"7ec5066a-6b4d-4669-97f5-8a46a9c99a1b","Type":"ContainerDied","Data":"a8e133dd63fb045cd89dea6db2b65aef167a4a19b83a9e11d66fbf8406af7271"}
Oct 31 17:42:44.287841 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:44.287818    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-ee816e/premium-w0-ee816e-kserve-65c86cd6db-lnt9p" event={"ID":"e536c902-52a9-4b36-a2ad-eb5500781a5e","Type":"ContainerDied","Data":"c61fb67c93b42d3898d2cde64e58720bfd198d0cace91429f7ded9237cc9c4ab"}
Oct 31 17:42:44.287841 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:44.287839    4276 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-ee816e/premium-w0-ee816e-kserve-65c86cd6db-lnt9p"
Oct 31 17:42:44.288038 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:44.287860    4276 scope.go:122] "RemoveContainer" containerID="36c6d2d02f285224be746e52aa2bbb9d39a797d025543f7867fdd95f94cb5696"
Oct 31 17:42:44.337786 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:44.337752    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:42:44.351863 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:44.351838    4276 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-ee816e/premium-w0-ee816e-kserve-65c86cd6db-lnt9p"]
Oct 31 17:42:44.370512 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:44.370422    4276 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-ee816e/premium-w0-ee816e-kserve-65c86cd6db-lnt9p"]
Oct 31 17:42:44.437766 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:44.437742    4276 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-bwdwd"
Oct 31 17:42:44.479208 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:44.479182    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7ec5066a-6b4d-4669-97f5-8a46a9c99a1b-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7ec5066a-6b4d-4669-97f5-8a46a9c99a1b-workload-certs\") pod \"7ec5066a-6b4d-4669-97f5-8a46a9c99a1b\" (UID: \"7ec5066a-6b4d-4669-97f5-8a46a9c99a1b\") "
Oct 31 17:42:44.479378 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:44.479326    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7ec5066a-6b4d-4669-97f5-8a46a9c99a1b-istio-token\" (UniqueName: \"kubernetes.io/projected/7ec5066a-6b4d-4669-97f5-8a46a9c99a1b-istio-token\") pod \"7ec5066a-6b4d-4669-97f5-8a46a9c99a1b\" (UID: \"7ec5066a-6b4d-4669-97f5-8a46a9c99a1b\") "
Oct 31 17:42:44.479378 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:44.479359    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7ec5066a-6b4d-4669-97f5-8a46a9c99a1b-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7ec5066a-6b4d-4669-97f5-8a46a9c99a1b-istio-envoy\") pod \"7ec5066a-6b4d-4669-97f5-8a46a9c99a1b\" (UID: \"7ec5066a-6b4d-4669-97f5-8a46a9c99a1b\") "
Oct 31 17:42:44.479534 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:44.479436    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7ec5066a-6b4d-4669-97f5-8a46a9c99a1b-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ec5066a-6b4d-4669-97f5-8a46a9c99a1b-workload-socket\") pod \"7ec5066a-6b4d-4669-97f5-8a46a9c99a1b\" (UID: \"7ec5066a-6b4d-4669-97f5-8a46a9c99a1b\") "
Oct 31 17:42:44.479534 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:44.479516    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/7ec5066a-6b4d-4669-97f5-8a46a9c99a1b-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7ec5066a-6b4d-4669-97f5-8a46a9c99a1b-istio-podinfo\") pod \"7ec5066a-6b4d-4669-97f5-8a46a9c99a1b\" (UID: \"7ec5066a-6b4d-4669-97f5-8a46a9c99a1b\") "
Oct 31 17:42:44.479642 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:44.479549    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7ec5066a-6b4d-4669-97f5-8a46a9c99a1b-kube-api-access-67pn5\" (UniqueName: \"kubernetes.io/projected/7ec5066a-6b4d-4669-97f5-8a46a9c99a1b-kube-api-access-67pn5\") pod \"7ec5066a-6b4d-4669-97f5-8a46a9c99a1b\" (UID: \"7ec5066a-6b4d-4669-97f5-8a46a9c99a1b\") "
Oct 31 17:42:44.479642 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:44.479512    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ec5066a-6b4d-4669-97f5-8a46a9c99a1b-workload-certs" pod "7ec5066a-6b4d-4669-97f5-8a46a9c99a1b" (UID: "7ec5066a-6b4d-4669-97f5-8a46a9c99a1b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:42:44.479737 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:44.479645    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7ec5066a-6b4d-4669-97f5-8a46a9c99a1b-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7ec5066a-6b4d-4669-97f5-8a46a9c99a1b-istiod-ca-cert\") pod \"7ec5066a-6b4d-4669-97f5-8a46a9c99a1b\" (UID: \"7ec5066a-6b4d-4669-97f5-8a46a9c99a1b\") "
Oct 31 17:42:44.479737 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:44.479708    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7ec5066a-6b4d-4669-97f5-8a46a9c99a1b-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7ec5066a-6b4d-4669-97f5-8a46a9c99a1b-istio-data\") pod \"7ec5066a-6b4d-4669-97f5-8a46a9c99a1b\" (UID: \"7ec5066a-6b4d-4669-97f5-8a46a9c99a1b\") "
Oct 31 17:42:44.479839 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:44.479768    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7ec5066a-6b4d-4669-97f5-8a46a9c99a1b-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ec5066a-6b4d-4669-97f5-8a46a9c99a1b-credential-socket\") pod \"7ec5066a-6b4d-4669-97f5-8a46a9c99a1b\" (UID: \"7ec5066a-6b4d-4669-97f5-8a46a9c99a1b\") "
Oct 31 17:42:44.479839 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:44.479791    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ec5066a-6b4d-4669-97f5-8a46a9c99a1b-workload-socket" pod "7ec5066a-6b4d-4669-97f5-8a46a9c99a1b" (UID: "7ec5066a-6b4d-4669-97f5-8a46a9c99a1b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:42:44.480033 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:44.480015    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ec5066a-6b4d-4669-97f5-8a46a9c99a1b-istio-data" pod "7ec5066a-6b4d-4669-97f5-8a46a9c99a1b" (UID: "7ec5066a-6b4d-4669-97f5-8a46a9c99a1b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:42:44.480152 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:44.480130    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7ec5066a-6b4d-4669-97f5-8a46a9c99a1b-istiod-ca-cert" pod "7ec5066a-6b4d-4669-97f5-8a46a9c99a1b" (UID: "7ec5066a-6b4d-4669-97f5-8a46a9c99a1b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 17:42:44.480152 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:44.480095    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ec5066a-6b4d-4669-97f5-8a46a9c99a1b-credential-socket" pod "7ec5066a-6b4d-4669-97f5-8a46a9c99a1b" (UID: "7ec5066a-6b4d-4669-97f5-8a46a9c99a1b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:42:44.480615 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:44.480593    4276 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ec5066a-6b4d-4669-97f5-8a46a9c99a1b-workload-socket\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:42:44.480704 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:44.480620    4276 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7ec5066a-6b4d-4669-97f5-8a46a9c99a1b-istiod-ca-cert\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:42:44.480704 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:44.480635    4276 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7ec5066a-6b4d-4669-97f5-8a46a9c99a1b-istio-data\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:42:44.480704 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:44.480648    4276 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ec5066a-6b4d-4669-97f5-8a46a9c99a1b-credential-socket\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:42:44.480704 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:44.480661    4276 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7ec5066a-6b4d-4669-97f5-8a46a9c99a1b-workload-certs\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:42:44.481999 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:44.481938    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7ec5066a-6b4d-4669-97f5-8a46a9c99a1b-istio-token" pod "7ec5066a-6b4d-4669-97f5-8a46a9c99a1b" (UID: "7ec5066a-6b4d-4669-97f5-8a46a9c99a1b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 17:42:44.482223 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:44.482193    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7ec5066a-6b4d-4669-97f5-8a46a9c99a1b-istio-podinfo" pod "7ec5066a-6b4d-4669-97f5-8a46a9c99a1b" (UID: "7ec5066a-6b4d-4669-97f5-8a46a9c99a1b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 17:42:44.482343 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:44.482315    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7ec5066a-6b4d-4669-97f5-8a46a9c99a1b-kube-api-access-67pn5" pod "7ec5066a-6b4d-4669-97f5-8a46a9c99a1b" (UID: "7ec5066a-6b4d-4669-97f5-8a46a9c99a1b"). InnerVolumeSpecName "kube-api-access-67pn5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 17:42:44.482405 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:44.482353    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ec5066a-6b4d-4669-97f5-8a46a9c99a1b-istio-envoy" pod "7ec5066a-6b4d-4669-97f5-8a46a9c99a1b" (UID: "7ec5066a-6b4d-4669-97f5-8a46a9c99a1b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:42:44.583952 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:44.583921    4276 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7ec5066a-6b4d-4669-97f5-8a46a9c99a1b-istio-podinfo\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:42:44.583952 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:44.583953    4276 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-67pn5\" (UniqueName: \"kubernetes.io/projected/7ec5066a-6b4d-4669-97f5-8a46a9c99a1b-kube-api-access-67pn5\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:42:44.583952 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:44.583969    4276 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7ec5066a-6b4d-4669-97f5-8a46a9c99a1b-istio-token\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:42:44.584465 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:44.583981    4276 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7ec5066a-6b4d-4669-97f5-8a46a9c99a1b-istio-envoy\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:42:44.613253 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:44.613223    4276 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-0e59ca-77546c9db5-b8b5g"]
Oct 31 17:42:44.613498 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:44.613471    4276 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-0e59ca-77546c9db5-b8b5g" podUID="ca2c3199-efa6-450a-8c46-343e39810896" containerName="maas-api" containerID="cri-o://d825eea5b72ce9edfb3548fad4864d78c62c96b9faa074322379ffa82a993849" gracePeriod=30
Oct 31 17:42:44.714034 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:44.713920    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:42:44.774753 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:44.774715    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-b78c31-6b5448c9d7-rbhfz" podUID="f7894ac3-7f53-4a3f-8d20-17359e24f69e" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 31 17:42:44.946702 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:44.946671    4276 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-8867c"
Oct 31 17:42:44.990278 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:44.990244    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d1bf33fd-97d5-462d-8c04-e2697fcf1b05-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1bf33fd-97d5-462d-8c04-e2697fcf1b05-workload-socket\") pod \"d1bf33fd-97d5-462d-8c04-e2697fcf1b05\" (UID: \"d1bf33fd-97d5-462d-8c04-e2697fcf1b05\") "
Oct 31 17:42:44.990499 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:44.990311    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d1bf33fd-97d5-462d-8c04-e2697fcf1b05-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d1bf33fd-97d5-462d-8c04-e2697fcf1b05-workload-certs\") pod \"d1bf33fd-97d5-462d-8c04-e2697fcf1b05\" (UID: \"d1bf33fd-97d5-462d-8c04-e2697fcf1b05\") "
Oct 31 17:42:44.990499 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:44.990338    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d1bf33fd-97d5-462d-8c04-e2697fcf1b05-istio-token\" (UniqueName: \"kubernetes.io/projected/d1bf33fd-97d5-462d-8c04-e2697fcf1b05-istio-token\") pod \"d1bf33fd-97d5-462d-8c04-e2697fcf1b05\" (UID: \"d1bf33fd-97d5-462d-8c04-e2697fcf1b05\") "
Oct 31 17:42:44.990499 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:44.990409    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/d1bf33fd-97d5-462d-8c04-e2697fcf1b05-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d1bf33fd-97d5-462d-8c04-e2697fcf1b05-istio-podinfo\") pod \"d1bf33fd-97d5-462d-8c04-e2697fcf1b05\" (UID: \"d1bf33fd-97d5-462d-8c04-e2697fcf1b05\") "
Oct 31 17:42:44.990499 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:44.990471    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d1bf33fd-97d5-462d-8c04-e2697fcf1b05-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d1bf33fd-97d5-462d-8c04-e2697fcf1b05-istiod-ca-cert\") pod \"d1bf33fd-97d5-462d-8c04-e2697fcf1b05\" (UID: \"d1bf33fd-97d5-462d-8c04-e2697fcf1b05\") "
Oct 31 17:42:44.990738 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:44.990500    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d1bf33fd-97d5-462d-8c04-e2697fcf1b05-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d1bf33fd-97d5-462d-8c04-e2697fcf1b05-istio-data\") pod \"d1bf33fd-97d5-462d-8c04-e2697fcf1b05\" (UID: \"d1bf33fd-97d5-462d-8c04-e2697fcf1b05\") "
Oct 31 17:42:44.990738 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:44.990524    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d1bf33fd-97d5-462d-8c04-e2697fcf1b05-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d1bf33fd-97d5-462d-8c04-e2697fcf1b05-istio-envoy\") pod \"d1bf33fd-97d5-462d-8c04-e2697fcf1b05\" (UID: \"d1bf33fd-97d5-462d-8c04-e2697fcf1b05\") "
Oct 31 17:42:44.990738 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:44.990649    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d1bf33fd-97d5-462d-8c04-e2697fcf1b05-kube-api-access-ptjgq\" (UniqueName: \"kubernetes.io/projected/d1bf33fd-97d5-462d-8c04-e2697fcf1b05-kube-api-access-ptjgq\") pod \"d1bf33fd-97d5-462d-8c04-e2697fcf1b05\" (UID: \"d1bf33fd-97d5-462d-8c04-e2697fcf1b05\") "
Oct 31 17:42:44.990888 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:44.990734    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1bf33fd-97d5-462d-8c04-e2697fcf1b05-workload-certs" pod "d1bf33fd-97d5-462d-8c04-e2697fcf1b05" (UID: "d1bf33fd-97d5-462d-8c04-e2697fcf1b05"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:42:44.990888 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:44.990750    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d1bf33fd-97d5-462d-8c04-e2697fcf1b05-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1bf33fd-97d5-462d-8c04-e2697fcf1b05-credential-socket\") pod \"d1bf33fd-97d5-462d-8c04-e2697fcf1b05\" (UID: \"d1bf33fd-97d5-462d-8c04-e2697fcf1b05\") "
Oct 31 17:42:44.991005 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:44.990959    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1bf33fd-97d5-462d-8c04-e2697fcf1b05-workload-socket" pod "d1bf33fd-97d5-462d-8c04-e2697fcf1b05" (UID: "d1bf33fd-97d5-462d-8c04-e2697fcf1b05"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:42:44.991557 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:44.991521    4276 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1bf33fd-97d5-462d-8c04-e2697fcf1b05-workload-socket\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:42:44.991557 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:44.991555    4276 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d1bf33fd-97d5-462d-8c04-e2697fcf1b05-workload-certs\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:42:44.991851 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:44.991829    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1bf33fd-97d5-462d-8c04-e2697fcf1b05-credential-socket" pod "d1bf33fd-97d5-462d-8c04-e2697fcf1b05" (UID: "d1bf33fd-97d5-462d-8c04-e2697fcf1b05"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:42:44.992469 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:44.992406    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1bf33fd-97d5-462d-8c04-e2697fcf1b05-istio-data" pod "d1bf33fd-97d5-462d-8c04-e2697fcf1b05" (UID: "d1bf33fd-97d5-462d-8c04-e2697fcf1b05"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:42:44.992782 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:44.992744    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d1bf33fd-97d5-462d-8c04-e2697fcf1b05-istiod-ca-cert" pod "d1bf33fd-97d5-462d-8c04-e2697fcf1b05" (UID: "d1bf33fd-97d5-462d-8c04-e2697fcf1b05"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 17:42:44.995340 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:44.995304    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d1bf33fd-97d5-462d-8c04-e2697fcf1b05-istio-podinfo" pod "d1bf33fd-97d5-462d-8c04-e2697fcf1b05" (UID: "d1bf33fd-97d5-462d-8c04-e2697fcf1b05"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 17:42:44.997784 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:44.997372    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d1bf33fd-97d5-462d-8c04-e2697fcf1b05-kube-api-access-ptjgq" pod "d1bf33fd-97d5-462d-8c04-e2697fcf1b05" (UID: "d1bf33fd-97d5-462d-8c04-e2697fcf1b05"). InnerVolumeSpecName "kube-api-access-ptjgq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 17:42:44.998170 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:44.998135    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d1bf33fd-97d5-462d-8c04-e2697fcf1b05-istio-token" pod "d1bf33fd-97d5-462d-8c04-e2697fcf1b05" (UID: "d1bf33fd-97d5-462d-8c04-e2697fcf1b05"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 17:42:45.005480 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.002589    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1bf33fd-97d5-462d-8c04-e2697fcf1b05-istio-envoy" pod "d1bf33fd-97d5-462d-8c04-e2697fcf1b05" (UID: "d1bf33fd-97d5-462d-8c04-e2697fcf1b05"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:42:45.092799 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.092768    4276 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ptjgq\" (UniqueName: \"kubernetes.io/projected/d1bf33fd-97d5-462d-8c04-e2697fcf1b05-kube-api-access-ptjgq\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:42:45.092799 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.092798    4276 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1bf33fd-97d5-462d-8c04-e2697fcf1b05-credential-socket\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:42:45.093165 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.092811    4276 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d1bf33fd-97d5-462d-8c04-e2697fcf1b05-istio-token\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:42:45.093165 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.092858    4276 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d1bf33fd-97d5-462d-8c04-e2697fcf1b05-istio-podinfo\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:42:45.093165 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.092871    4276 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d1bf33fd-97d5-462d-8c04-e2697fcf1b05-istiod-ca-cert\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:42:45.093165 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.092885    4276 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d1bf33fd-97d5-462d-8c04-e2697fcf1b05-istio-data\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:42:45.093165 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.092895    4276 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d1bf33fd-97d5-462d-8c04-e2697fcf1b05-istio-envoy\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:42:45.163581 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.163495    4276 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-f61ac551-sc5kq"]
Oct 31 17:42:45.164774 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.164748    4276 cpu_manager.go:414] "containerMap: removing container" podUID="1bd48c9b-b035-4931-8cf6-dbb821d3f6d5" containerName="payload-pre-processing"
Oct 31 17:42:45.164932 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.164918    4276 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1bd48c9b-b035-4931-8cf6-dbb821d3f6d5" containerName="payload-pre-processing"
Oct 31 17:42:45.165032 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.165022    4276 cpu_manager.go:414] "containerMap: removing container" podUID="d5dd5b19-e930-4afa-abea-db914b9f7e89" containerName="maas-api"
Oct 31 17:42:45.165108 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.165097    4276 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d5dd5b19-e930-4afa-abea-db914b9f7e89" containerName="maas-api"
Oct 31 17:42:45.165185 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.165175    4276 cpu_manager.go:414] "containerMap: removing container" podUID="2d1185cc-bac1-45ca-a250-8f44f8589162" containerName="main"
Oct 31 17:42:45.165286 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.165274    4276 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2d1185cc-bac1-45ca-a250-8f44f8589162" containerName="main"
Oct 31 17:42:45.165378 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.165367    4276 cpu_manager.go:414] "containerMap: removing container" podUID="4c288908-92e3-4c45-8ba5-2f86c9f38b3d" containerName="main"
Oct 31 17:42:45.165562 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.165542    4276 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4c288908-92e3-4c45-8ba5-2f86c9f38b3d" containerName="main"
Oct 31 17:42:45.165658 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.165647    4276 cpu_manager.go:414] "containerMap: removing container" podUID="f8a45b7c-e5ae-4321-ad81-fd6353b2495d" containerName="main"
Oct 31 17:42:45.165728 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.165718    4276 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f8a45b7c-e5ae-4321-ad81-fd6353b2495d" containerName="main"
Oct 31 17:42:45.165800 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.165789    4276 cpu_manager.go:414] "containerMap: removing container" podUID="d1bf33fd-97d5-462d-8c04-e2697fcf1b05" containerName="istio-proxy"
Oct 31 17:42:45.165938 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.165884    4276 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d1bf33fd-97d5-462d-8c04-e2697fcf1b05" containerName="istio-proxy"
Oct 31 17:42:45.166066 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.166054    4276 cpu_manager.go:414] "containerMap: removing container" podUID="e536c902-52a9-4b36-a2ad-eb5500781a5e" containerName="main"
Oct 31 17:42:45.166143 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.166132    4276 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e536c902-52a9-4b36-a2ad-eb5500781a5e" containerName="main"
Oct 31 17:42:45.166223 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.166213    4276 cpu_manager.go:414] "containerMap: removing container" podUID="8cf1e52e-70f8-42e2-bdf3-a315e17f10f5" containerName="main"
Oct 31 17:42:45.166293 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.166283    4276 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8cf1e52e-70f8-42e2-bdf3-a315e17f10f5" containerName="main"
Oct 31 17:42:45.166370 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.166361    4276 cpu_manager.go:414] "containerMap: removing container" podUID="7ec5066a-6b4d-4669-97f5-8a46a9c99a1b" containerName="istio-proxy"
Oct 31 17:42:45.166465 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.166437    4276 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7ec5066a-6b4d-4669-97f5-8a46a9c99a1b" containerName="istio-proxy"
Oct 31 17:42:45.166586 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.166575    4276 cpu_manager.go:414] "containerMap: removing container" podUID="67250413-5beb-44da-9925-71b1735d723b" containerName="maas-api"
Oct 31 17:42:45.166662 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.166652    4276 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="67250413-5beb-44da-9925-71b1735d723b" containerName="maas-api"
Oct 31 17:42:45.166887 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.166873    4276 memory_manager.go:365] "RemoveStaleState removing state" podUID="67250413-5beb-44da-9925-71b1735d723b" containerName="maas-api"
Oct 31 17:42:45.167006 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.166995    4276 memory_manager.go:365] "RemoveStaleState removing state" podUID="1bd48c9b-b035-4931-8cf6-dbb821d3f6d5" containerName="payload-pre-processing"
Oct 31 17:42:45.167091 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.167082    4276 memory_manager.go:365] "RemoveStaleState removing state" podUID="d5dd5b19-e930-4afa-abea-db914b9f7e89" containerName="maas-api"
Oct 31 17:42:45.167172 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.167162    4276 memory_manager.go:365] "RemoveStaleState removing state" podUID="2d1185cc-bac1-45ca-a250-8f44f8589162" containerName="main"
Oct 31 17:42:45.167242 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.167233    4276 memory_manager.go:365] "RemoveStaleState removing state" podUID="4c288908-92e3-4c45-8ba5-2f86c9f38b3d" containerName="main"
Oct 31 17:42:45.167318 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.167309    4276 memory_manager.go:365] "RemoveStaleState removing state" podUID="f8a45b7c-e5ae-4321-ad81-fd6353b2495d" containerName="main"
Oct 31 17:42:45.167392 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.167382    4276 memory_manager.go:365] "RemoveStaleState removing state" podUID="d1bf33fd-97d5-462d-8c04-e2697fcf1b05" containerName="istio-proxy"
Oct 31 17:42:45.167502 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.167491    4276 memory_manager.go:365] "RemoveStaleState removing state" podUID="e536c902-52a9-4b36-a2ad-eb5500781a5e" containerName="main"
Oct 31 17:42:45.167585 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.167576    4276 memory_manager.go:365] "RemoveStaleState removing state" podUID="8cf1e52e-70f8-42e2-bdf3-a315e17f10f5" containerName="main"
Oct 31 17:42:45.168053 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.168038    4276 memory_manager.go:365] "RemoveStaleState removing state" podUID="7ec5066a-6b4d-4669-97f5-8a46a9c99a1b" containerName="istio-proxy"
Oct 31 17:42:45.174891 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.174871    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-f61ac551-sc5kq"]
Oct 31 17:42:45.175145 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.175130    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f61ac551-sc5kq"
Oct 31 17:42:45.194601 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.194574    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-mr9x2\""
Oct 31 17:42:45.194790 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.194777    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 31 17:42:45.229134 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.229105    4276 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-c5dc6"
Oct 31 17:42:45.276133 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.276108    4276 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-rfwx5"
Oct 31 17:42:45.298894 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.298859    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/22af22dd-1130-409b-9a17-05229dc0de11-kube-api-access-57gbg\" (UniqueName: \"kubernetes.io/projected/22af22dd-1130-409b-9a17-05229dc0de11-kube-api-access-57gbg\") pod \"22af22dd-1130-409b-9a17-05229dc0de11\" (UID: \"22af22dd-1130-409b-9a17-05229dc0de11\") "
Oct 31 17:42:45.303635 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.303605    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/22af22dd-1130-409b-9a17-05229dc0de11-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/22af22dd-1130-409b-9a17-05229dc0de11-workload-socket\") pod \"22af22dd-1130-409b-9a17-05229dc0de11\" (UID: \"22af22dd-1130-409b-9a17-05229dc0de11\") "
Oct 31 17:42:45.303954 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.303903    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/22af22dd-1130-409b-9a17-05229dc0de11-workload-socket" pod "22af22dd-1130-409b-9a17-05229dc0de11" (UID: "22af22dd-1130-409b-9a17-05229dc0de11"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:42:45.303954 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.303938    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/22af22dd-1130-409b-9a17-05229dc0de11-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/22af22dd-1130-409b-9a17-05229dc0de11-istiod-ca-cert\") pod \"22af22dd-1130-409b-9a17-05229dc0de11\" (UID: \"22af22dd-1130-409b-9a17-05229dc0de11\") "
Oct 31 17:42:45.304157 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.304007    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/22af22dd-1130-409b-9a17-05229dc0de11-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/22af22dd-1130-409b-9a17-05229dc0de11-workload-certs\") pod \"22af22dd-1130-409b-9a17-05229dc0de11\" (UID: \"22af22dd-1130-409b-9a17-05229dc0de11\") "
Oct 31 17:42:45.304157 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.304079    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/22af22dd-1130-409b-9a17-05229dc0de11-istio-token\" (UniqueName: \"kubernetes.io/projected/22af22dd-1130-409b-9a17-05229dc0de11-istio-token\") pod \"22af22dd-1130-409b-9a17-05229dc0de11\" (UID: \"22af22dd-1130-409b-9a17-05229dc0de11\") "
Oct 31 17:42:45.304266 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.304173    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/22af22dd-1130-409b-9a17-05229dc0de11-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/22af22dd-1130-409b-9a17-05229dc0de11-credential-socket\") pod \"22af22dd-1130-409b-9a17-05229dc0de11\" (UID: \"22af22dd-1130-409b-9a17-05229dc0de11\") "
Oct 31 17:42:45.304266 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.304223    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/22af22dd-1130-409b-9a17-05229dc0de11-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/22af22dd-1130-409b-9a17-05229dc0de11-istio-data\") pod \"22af22dd-1130-409b-9a17-05229dc0de11\" (UID: \"22af22dd-1130-409b-9a17-05229dc0de11\") "
Oct 31 17:42:45.304358 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.304344    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/22af22dd-1130-409b-9a17-05229dc0de11-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/22af22dd-1130-409b-9a17-05229dc0de11-istio-envoy\") pod \"22af22dd-1130-409b-9a17-05229dc0de11\" (UID: \"22af22dd-1130-409b-9a17-05229dc0de11\") "
Oct 31 17:42:45.304713 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.304488    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/22af22dd-1130-409b-9a17-05229dc0de11-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/22af22dd-1130-409b-9a17-05229dc0de11-istio-podinfo\") pod \"22af22dd-1130-409b-9a17-05229dc0de11\" (UID: \"22af22dd-1130-409b-9a17-05229dc0de11\") "
Oct 31 17:42:45.304903 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.304881    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/9c4606cd-947c-4226-8152-74f8870458e6-maas-api-xks-ca\") pod \"maas-api-revoke-sub-f61ac551-sc5kq\" (UID: \"9c4606cd-947c-4226-8152-74f8870458e6\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f61ac551-sc5kq"
Oct 31 17:42:45.305078 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.305060    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7qp4b\" (UniqueName: \"kubernetes.io/projected/9c4606cd-947c-4226-8152-74f8870458e6-kube-api-access-7qp4b\") pod \"maas-api-revoke-sub-f61ac551-sc5kq\" (UID: \"9c4606cd-947c-4226-8152-74f8870458e6\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f61ac551-sc5kq"
Oct 31 17:42:45.305377 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.305360    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/9c4606cd-947c-4226-8152-74f8870458e6-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-f61ac551-sc5kq\" (UID: \"9c4606cd-947c-4226-8152-74f8870458e6\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f61ac551-sc5kq"
Oct 31 17:42:45.305525 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.305506    4276 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/22af22dd-1130-409b-9a17-05229dc0de11-workload-socket\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:42:45.305852 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.305831    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/22af22dd-1130-409b-9a17-05229dc0de11-credential-socket" pod "22af22dd-1130-409b-9a17-05229dc0de11" (UID: "22af22dd-1130-409b-9a17-05229dc0de11"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:42:45.306077 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.306056    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/22af22dd-1130-409b-9a17-05229dc0de11-istio-data" pod "22af22dd-1130-409b-9a17-05229dc0de11" (UID: "22af22dd-1130-409b-9a17-05229dc0de11"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:42:45.308549 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.308490    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/22af22dd-1130-409b-9a17-05229dc0de11-workload-certs" pod "22af22dd-1130-409b-9a17-05229dc0de11" (UID: "22af22dd-1130-409b-9a17-05229dc0de11"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:42:45.308647 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.308536    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/22af22dd-1130-409b-9a17-05229dc0de11-istiod-ca-cert" pod "22af22dd-1130-409b-9a17-05229dc0de11" (UID: "22af22dd-1130-409b-9a17-05229dc0de11"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 17:42:45.312982 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.312935    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/22af22dd-1130-409b-9a17-05229dc0de11-istio-podinfo" pod "22af22dd-1130-409b-9a17-05229dc0de11" (UID: "22af22dd-1130-409b-9a17-05229dc0de11"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 17:42:45.320752 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.320696    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/22af22dd-1130-409b-9a17-05229dc0de11-istio-token" pod "22af22dd-1130-409b-9a17-05229dc0de11" (UID: "22af22dd-1130-409b-9a17-05229dc0de11"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 17:42:45.322380 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.322358    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/22af22dd-1130-409b-9a17-05229dc0de11-istio-envoy" pod "22af22dd-1130-409b-9a17-05229dc0de11" (UID: "22af22dd-1130-409b-9a17-05229dc0de11"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:42:45.322380 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.322371    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/22af22dd-1130-409b-9a17-05229dc0de11-kube-api-access-57gbg" pod "22af22dd-1130-409b-9a17-05229dc0de11" (UID: "22af22dd-1130-409b-9a17-05229dc0de11"). InnerVolumeSpecName "kube-api-access-57gbg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 17:42:45.323569 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.323542    4276 generic.go:356] "Generic (PLEG): container finished" podID="22af22dd-1130-409b-9a17-05229dc0de11" containerID="bc9404ff4e456fc89595255741cc75406b890990ce42d69b922760a2527d1feb" exitCode=0
Oct 31 17:42:45.323676 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.323599    4276 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-c5dc6"
Oct 31 17:42:45.323676 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.323619    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-c5dc6" event={"ID":"22af22dd-1130-409b-9a17-05229dc0de11","Type":"ContainerDied","Data":"bc9404ff4e456fc89595255741cc75406b890990ce42d69b922760a2527d1feb"}
Oct 31 17:42:45.323787 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.323675    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-c5dc6" event={"ID":"22af22dd-1130-409b-9a17-05229dc0de11","Type":"ContainerDied","Data":"43fcf2d064994ee73dff7f5da6a887f6283bbe413a689094db8a54614c485a06"}
Oct 31 17:42:45.323787 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.323708    4276 scope.go:122] "RemoveContainer" containerID="bc9404ff4e456fc89595255741cc75406b890990ce42d69b922760a2527d1feb"
Oct 31 17:42:45.329026 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.327115    4276 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-bwdwd"
Oct 31 17:42:45.329026 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.327125    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-bwdwd" event={"ID":"7ec5066a-6b4d-4669-97f5-8a46a9c99a1b","Type":"ContainerDied","Data":"00c7fa987e205d9088d626145a761b81be37b140acfcdf621ac325d52b90e060"}
Oct 31 17:42:45.329026 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.328391    4276 generic.go:356] "Generic (PLEG): container finished" podID="d1bf33fd-97d5-462d-8c04-e2697fcf1b05" containerID="9224bcb42e7be2c812e6db5e1f87d5b65af67a6e6c1cea7735c44f82d264ca7b" exitCode=0
Oct 31 17:42:45.329026 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.328468    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-8867c" event={"ID":"d1bf33fd-97d5-462d-8c04-e2697fcf1b05","Type":"ContainerDied","Data":"9224bcb42e7be2c812e6db5e1f87d5b65af67a6e6c1cea7735c44f82d264ca7b"}
Oct 31 17:42:45.329026 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.328481    4276 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-8867c"
Oct 31 17:42:45.329026 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.328506    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-8867c" event={"ID":"d1bf33fd-97d5-462d-8c04-e2697fcf1b05","Type":"ContainerDied","Data":"4784fbfdff6c2ca6d17de2bd266c1c07bfcbeca7735868e90348cccc4d0b76fd"}
Oct 31 17:42:45.336517 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.336496    4276 generic.go:356] "Generic (PLEG): container finished" podID="5155d374-e42a-4c40-9485-935443ede2c5" containerID="0707bf2ff1d341d72693257e316834b57c76ced9cad6fe5df2ec01e0ec9c803d" exitCode=0
Oct 31 17:42:45.336701 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.336584    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-rfwx5" event={"ID":"5155d374-e42a-4c40-9485-935443ede2c5","Type":"ContainerDied","Data":"0707bf2ff1d341d72693257e316834b57c76ced9cad6fe5df2ec01e0ec9c803d"}
Oct 31 17:42:45.336877 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.336860    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-rfwx5" event={"ID":"5155d374-e42a-4c40-9485-935443ede2c5","Type":"ContainerDied","Data":"3c1a889c48e1f72718fae760d28b15b474c59131109f7a65ea2bfd263e5b4e06"}
Oct 31 17:42:45.337035 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.336616    4276 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-rfwx5"
Oct 31 17:42:45.370536 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.367951    4276 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-0a3ab6fd-62wxv"]
Oct 31 17:42:45.370536 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.368720    4276 cpu_manager.go:414] "containerMap: removing container" podUID="5155d374-e42a-4c40-9485-935443ede2c5" containerName="istio-proxy"
Oct 31 17:42:45.370536 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.368740    4276 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5155d374-e42a-4c40-9485-935443ede2c5" containerName="istio-proxy"
Oct 31 17:42:45.370536 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.368786    4276 cpu_manager.go:414] "containerMap: removing container" podUID="22af22dd-1130-409b-9a17-05229dc0de11" containerName="istio-proxy"
Oct 31 17:42:45.370536 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.368794    4276 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="22af22dd-1130-409b-9a17-05229dc0de11" containerName="istio-proxy"
Oct 31 17:42:45.370536 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.368805    4276 cpu_manager.go:414] "containerMap: removing container" podUID="22af22dd-1130-409b-9a17-05229dc0de11" containerName="istio-proxy"
Oct 31 17:42:45.370536 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.368813    4276 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="22af22dd-1130-409b-9a17-05229dc0de11" containerName="istio-proxy"
Oct 31 17:42:45.370536 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.368919    4276 memory_manager.go:365] "RemoveStaleState removing state" podUID="22af22dd-1130-409b-9a17-05229dc0de11" containerName="istio-proxy"
Oct 31 17:42:45.370536 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.368930    4276 memory_manager.go:365] "RemoveStaleState removing state" podUID="22af22dd-1130-409b-9a17-05229dc0de11" containerName="istio-proxy"
Oct 31 17:42:45.370536 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.368942    4276 memory_manager.go:365] "RemoveStaleState removing state" podUID="5155d374-e42a-4c40-9485-935443ede2c5" containerName="istio-proxy"
Oct 31 17:42:45.376172 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.376143    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0a3ab6fd-62wxv"
Oct 31 17:42:45.387488 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.387454    4276 scope.go:122] "RemoveContainer" containerID="5563c62e742bdd643b7b91521fa78ea55c3acbe6686b95695605135a70068dcc"
Oct 31 17:42:45.392008 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.391981    4276 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e536c902-52a9-4b36-a2ad-eb5500781a5e" path="/var/lib/kubelet/pods/e536c902-52a9-4b36-a2ad-eb5500781a5e/volumes"
Oct 31 17:42:45.395866 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.395842    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:42:45.396034 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.396021    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-0a3ab6fd-62wxv"]
Oct 31 17:42:45.406871 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.406847    4276 scope.go:122] "RemoveContainer" containerID="bc9404ff4e456fc89595255741cc75406b890990ce42d69b922760a2527d1feb"
Oct 31 17:42:45.407236 ip-10-0-129-127 kubenswrapper[4276]: E1031 17:42:45.407212    4276 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bc9404ff4e456fc89595255741cc75406b890990ce42d69b922760a2527d1feb\": container with ID starting with bc9404ff4e456fc89595255741cc75406b890990ce42d69b922760a2527d1feb not found: ID does not exist" containerID="bc9404ff4e456fc89595255741cc75406b890990ce42d69b922760a2527d1feb"
Oct 31 17:42:45.407320 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.407248    4276 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bc9404ff4e456fc89595255741cc75406b890990ce42d69b922760a2527d1feb"} err="failed to get container status \"bc9404ff4e456fc89595255741cc75406b890990ce42d69b922760a2527d1feb\": rpc error: code = NotFound desc = could not find container \"bc9404ff4e456fc89595255741cc75406b890990ce42d69b922760a2527d1feb\": container with ID starting with bc9404ff4e456fc89595255741cc75406b890990ce42d69b922760a2527d1feb not found: ID does not exist"
Oct 31 17:42:45.407320 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.407273    4276 scope.go:122] "RemoveContainer" containerID="5563c62e742bdd643b7b91521fa78ea55c3acbe6686b95695605135a70068dcc"
Oct 31 17:42:45.407432 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.407360    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5155d374-e42a-4c40-9485-935443ede2c5-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5155d374-e42a-4c40-9485-935443ede2c5-workload-socket\") pod \"5155d374-e42a-4c40-9485-935443ede2c5\" (UID: \"5155d374-e42a-4c40-9485-935443ede2c5\") "
Oct 31 17:42:45.408045 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.407473    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5155d374-e42a-4c40-9485-935443ede2c5-kube-api-access-pcq9f\" (UniqueName: \"kubernetes.io/projected/5155d374-e42a-4c40-9485-935443ede2c5-kube-api-access-pcq9f\") pod \"5155d374-e42a-4c40-9485-935443ede2c5\" (UID: \"5155d374-e42a-4c40-9485-935443ede2c5\") "
Oct 31 17:42:45.408045 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.407557    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5155d374-e42a-4c40-9485-935443ede2c5-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5155d374-e42a-4c40-9485-935443ede2c5-workload-certs\") pod \"5155d374-e42a-4c40-9485-935443ede2c5\" (UID: \"5155d374-e42a-4c40-9485-935443ede2c5\") "
Oct 31 17:42:45.408045 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.407652    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5155d374-e42a-4c40-9485-935443ede2c5-workload-socket" pod "5155d374-e42a-4c40-9485-935443ede2c5" (UID: "5155d374-e42a-4c40-9485-935443ede2c5"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:42:45.408045 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.407677    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5155d374-e42a-4c40-9485-935443ede2c5-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5155d374-e42a-4c40-9485-935443ede2c5-istio-data\") pod \"5155d374-e42a-4c40-9485-935443ede2c5\" (UID: \"5155d374-e42a-4c40-9485-935443ede2c5\") "
Oct 31 17:42:45.408045 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.407757    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5155d374-e42a-4c40-9485-935443ede2c5-istio-token\" (UniqueName: \"kubernetes.io/projected/5155d374-e42a-4c40-9485-935443ede2c5-istio-token\") pod \"5155d374-e42a-4c40-9485-935443ede2c5\" (UID: \"5155d374-e42a-4c40-9485-935443ede2c5\") "
Oct 31 17:42:45.408045 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.407800    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5155d374-e42a-4c40-9485-935443ede2c5-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5155d374-e42a-4c40-9485-935443ede2c5-credential-socket\") pod \"5155d374-e42a-4c40-9485-935443ede2c5\" (UID: \"5155d374-e42a-4c40-9485-935443ede2c5\") "
Oct 31 17:42:45.408045 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.407895    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5155d374-e42a-4c40-9485-935443ede2c5-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5155d374-e42a-4c40-9485-935443ede2c5-istio-envoy\") pod \"5155d374-e42a-4c40-9485-935443ede2c5\" (UID: \"5155d374-e42a-4c40-9485-935443ede2c5\") "
Oct 31 17:42:45.408045 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.407937    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/5155d374-e42a-4c40-9485-935443ede2c5-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5155d374-e42a-4c40-9485-935443ede2c5-istio-podinfo\") pod \"5155d374-e42a-4c40-9485-935443ede2c5\" (UID: \"5155d374-e42a-4c40-9485-935443ede2c5\") "
Oct 31 17:42:45.408045 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.407984    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5155d374-e42a-4c40-9485-935443ede2c5-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5155d374-e42a-4c40-9485-935443ede2c5-istiod-ca-cert\") pod \"5155d374-e42a-4c40-9485-935443ede2c5\" (UID: \"5155d374-e42a-4c40-9485-935443ede2c5\") "
Oct 31 17:42:45.408500 ip-10-0-129-127 kubenswrapper[4276]: E1031 17:42:45.408102    4276 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5563c62e742bdd643b7b91521fa78ea55c3acbe6686b95695605135a70068dcc\": container with ID starting with 5563c62e742bdd643b7b91521fa78ea55c3acbe6686b95695605135a70068dcc not found: ID does not exist" containerID="5563c62e742bdd643b7b91521fa78ea55c3acbe6686b95695605135a70068dcc"
Oct 31 17:42:45.408500 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.408133    4276 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5563c62e742bdd643b7b91521fa78ea55c3acbe6686b95695605135a70068dcc"} err="failed to get container status \"5563c62e742bdd643b7b91521fa78ea55c3acbe6686b95695605135a70068dcc\": rpc error: code = NotFound desc = could not find container \"5563c62e742bdd643b7b91521fa78ea55c3acbe6686b95695605135a70068dcc\": container with ID starting with 5563c62e742bdd643b7b91521fa78ea55c3acbe6686b95695605135a70068dcc not found: ID does not exist"
Oct 31 17:42:45.408500 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.408149    4276 scope.go:122] "RemoveContainer" containerID="a8e133dd63fb045cd89dea6db2b65aef167a4a19b83a9e11d66fbf8406af7271"
Oct 31 17:42:45.408659 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.408553    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/9c4606cd-947c-4226-8152-74f8870458e6-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-f61ac551-sc5kq\" (UID: \"9c4606cd-947c-4226-8152-74f8870458e6\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f61ac551-sc5kq"
Oct 31 17:42:45.408714 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.408667    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/9c4606cd-947c-4226-8152-74f8870458e6-maas-api-xks-ca\") pod \"maas-api-revoke-sub-f61ac551-sc5kq\" (UID: \"9c4606cd-947c-4226-8152-74f8870458e6\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f61ac551-sc5kq"
Oct 31 17:42:45.408812 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.408794    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7qp4b\" (UniqueName: \"kubernetes.io/projected/9c4606cd-947c-4226-8152-74f8870458e6-kube-api-access-7qp4b\") pod \"maas-api-revoke-sub-f61ac551-sc5kq\" (UID: \"9c4606cd-947c-4226-8152-74f8870458e6\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f61ac551-sc5kq"
Oct 31 17:42:45.409021 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.409006    4276 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/22af22dd-1130-409b-9a17-05229dc0de11-istio-podinfo\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:42:45.409083 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.409027    4276 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-57gbg\" (UniqueName: \"kubernetes.io/projected/22af22dd-1130-409b-9a17-05229dc0de11-kube-api-access-57gbg\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:42:45.409083 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.409044    4276 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5155d374-e42a-4c40-9485-935443ede2c5-workload-socket\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:42:45.409083 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.409067    4276 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/22af22dd-1130-409b-9a17-05229dc0de11-istiod-ca-cert\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:42:45.409228 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.409082    4276 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/22af22dd-1130-409b-9a17-05229dc0de11-workload-certs\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:42:45.409228 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.409098    4276 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/22af22dd-1130-409b-9a17-05229dc0de11-istio-token\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:42:45.409228 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.409114    4276 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/22af22dd-1130-409b-9a17-05229dc0de11-credential-socket\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:42:45.409228 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.409133    4276 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/22af22dd-1130-409b-9a17-05229dc0de11-istio-data\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:42:45.409228 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.409147    4276 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/22af22dd-1130-409b-9a17-05229dc0de11-istio-envoy\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:42:45.409499 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.409314    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5155d374-e42a-4c40-9485-935443ede2c5-workload-certs" pod "5155d374-e42a-4c40-9485-935443ede2c5" (UID: "5155d374-e42a-4c40-9485-935443ede2c5"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:42:45.409569 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.409511    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5155d374-e42a-4c40-9485-935443ede2c5-istio-data" pod "5155d374-e42a-4c40-9485-935443ede2c5" (UID: "5155d374-e42a-4c40-9485-935443ede2c5"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:42:45.409676 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.409654    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5155d374-e42a-4c40-9485-935443ede2c5-credential-socket" pod "5155d374-e42a-4c40-9485-935443ede2c5" (UID: "5155d374-e42a-4c40-9485-935443ede2c5"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:42:45.410148 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.410129    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/9c4606cd-947c-4226-8152-74f8870458e6-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-f61ac551-sc5kq\" (UID: \"9c4606cd-947c-4226-8152-74f8870458e6\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f61ac551-sc5kq"
Oct 31 17:42:45.410432 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.410406    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5155d374-e42a-4c40-9485-935443ede2c5-istiod-ca-cert" pod "5155d374-e42a-4c40-9485-935443ede2c5" (UID: "5155d374-e42a-4c40-9485-935443ede2c5"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 17:42:45.414690 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.414643    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/9c4606cd-947c-4226-8152-74f8870458e6-maas-api-xks-ca\") pod \"maas-api-revoke-sub-f61ac551-sc5kq\" (UID: \"9c4606cd-947c-4226-8152-74f8870458e6\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f61ac551-sc5kq"
Oct 31 17:42:45.421402 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.421343    4276 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-c5dc6"]
Oct 31 17:42:45.421804 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.421766    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5155d374-e42a-4c40-9485-935443ede2c5-istio-envoy" pod "5155d374-e42a-4c40-9485-935443ede2c5" (UID: "5155d374-e42a-4c40-9485-935443ede2c5"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:42:45.421804 ip-10-0-129-127 kubenswrapper[4276]: E1031 17:42:45.421787    4276 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ec5066a_6b4d_4669_97f5_8a46a9c99a1b.slice\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ec5066a_6b4d_4669_97f5_8a46a9c99a1b.slice/crio-00c7fa987e205d9088d626145a761b81be37b140acfcdf621ac325d52b90e060\": RecentStats: unable to find data in memory cache]"
Oct 31 17:42:45.421955 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.421795    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5155d374-e42a-4c40-9485-935443ede2c5-istio-token" pod "5155d374-e42a-4c40-9485-935443ede2c5" (UID: "5155d374-e42a-4c40-9485-935443ede2c5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 17:42:45.422008 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.421961    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5155d374-e42a-4c40-9485-935443ede2c5-kube-api-access-pcq9f" pod "5155d374-e42a-4c40-9485-935443ede2c5" (UID: "5155d374-e42a-4c40-9485-935443ede2c5"). InnerVolumeSpecName "kube-api-access-pcq9f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 17:42:45.423852 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.423825    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5155d374-e42a-4c40-9485-935443ede2c5-istio-podinfo" pod "5155d374-e42a-4c40-9485-935443ede2c5" (UID: "5155d374-e42a-4c40-9485-935443ede2c5"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 17:42:45.429176 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.429151    4276 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-c5dc6"]
Oct 31 17:42:45.431218 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.431200    4276 scope.go:122] "RemoveContainer" containerID="9224bcb42e7be2c812e6db5e1f87d5b65af67a6e6c1cea7735c44f82d264ca7b"
Oct 31 17:42:45.431378 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.431355    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7qp4b\" (UniqueName: \"kubernetes.io/projected/9c4606cd-947c-4226-8152-74f8870458e6-kube-api-access-7qp4b\") pod \"maas-api-revoke-sub-f61ac551-sc5kq\" (UID: \"9c4606cd-947c-4226-8152-74f8870458e6\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f61ac551-sc5kq"
Oct 31 17:42:45.442841 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.442818    4276 scope.go:122] "RemoveContainer" containerID="9224bcb42e7be2c812e6db5e1f87d5b65af67a6e6c1cea7735c44f82d264ca7b"
Oct 31 17:42:45.443122 ip-10-0-129-127 kubenswrapper[4276]: E1031 17:42:45.443103    4276 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9224bcb42e7be2c812e6db5e1f87d5b65af67a6e6c1cea7735c44f82d264ca7b\": container with ID starting with 9224bcb42e7be2c812e6db5e1f87d5b65af67a6e6c1cea7735c44f82d264ca7b not found: ID does not exist" containerID="9224bcb42e7be2c812e6db5e1f87d5b65af67a6e6c1cea7735c44f82d264ca7b"
Oct 31 17:42:45.443196 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.443129    4276 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9224bcb42e7be2c812e6db5e1f87d5b65af67a6e6c1cea7735c44f82d264ca7b"} err="failed to get container status \"9224bcb42e7be2c812e6db5e1f87d5b65af67a6e6c1cea7735c44f82d264ca7b\": rpc error: code = NotFound desc = could not find container \"9224bcb42e7be2c812e6db5e1f87d5b65af67a6e6c1cea7735c44f82d264ca7b\": container with ID starting with 9224bcb42e7be2c812e6db5e1f87d5b65af67a6e6c1cea7735c44f82d264ca7b not found: ID does not exist"
Oct 31 17:42:45.443196 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.443145    4276 scope.go:122] "RemoveContainer" containerID="0707bf2ff1d341d72693257e316834b57c76ced9cad6fe5df2ec01e0ec9c803d"
Oct 31 17:42:45.452308 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.452290    4276 scope.go:122] "RemoveContainer" containerID="0707bf2ff1d341d72693257e316834b57c76ced9cad6fe5df2ec01e0ec9c803d"
Oct 31 17:42:45.452587 ip-10-0-129-127 kubenswrapper[4276]: E1031 17:42:45.452567    4276 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0707bf2ff1d341d72693257e316834b57c76ced9cad6fe5df2ec01e0ec9c803d\": container with ID starting with 0707bf2ff1d341d72693257e316834b57c76ced9cad6fe5df2ec01e0ec9c803d not found: ID does not exist" containerID="0707bf2ff1d341d72693257e316834b57c76ced9cad6fe5df2ec01e0ec9c803d"
Oct 31 17:42:45.452643 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.452593    4276 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0707bf2ff1d341d72693257e316834b57c76ced9cad6fe5df2ec01e0ec9c803d"} err="failed to get container status \"0707bf2ff1d341d72693257e316834b57c76ced9cad6fe5df2ec01e0ec9c803d\": rpc error: code = NotFound desc = could not find container \"0707bf2ff1d341d72693257e316834b57c76ced9cad6fe5df2ec01e0ec9c803d\": container with ID starting with 0707bf2ff1d341d72693257e316834b57c76ced9cad6fe5df2ec01e0ec9c803d not found: ID does not exist"
Oct 31 17:42:45.496544 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.496521    4276 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-8867c"]
Oct 31 17:42:45.497742 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.497711    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f61ac551-sc5kq"
Oct 31 17:42:45.500478 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.500456    4276 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-8867c"]
Oct 31 17:42:45.510982 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.510958    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v7knh\" (UniqueName: \"kubernetes.io/projected/653b8145-5630-4b28-b3c8-624fb92032e3-kube-api-access-v7knh\") pod \"maas-api-revoke-sub-0a3ab6fd-62wxv\" (UID: \"653b8145-5630-4b28-b3c8-624fb92032e3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0a3ab6fd-62wxv"
Oct 31 17:42:45.511210 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.511192    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/653b8145-5630-4b28-b3c8-624fb92032e3-maas-api-xks-ca\") pod \"maas-api-revoke-sub-0a3ab6fd-62wxv\" (UID: \"653b8145-5630-4b28-b3c8-624fb92032e3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0a3ab6fd-62wxv"
Oct 31 17:42:45.511956 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.511933    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/653b8145-5630-4b28-b3c8-624fb92032e3-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-0a3ab6fd-62wxv\" (UID: \"653b8145-5630-4b28-b3c8-624fb92032e3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0a3ab6fd-62wxv"
Oct 31 17:42:45.512474 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.512423    4276 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5155d374-e42a-4c40-9485-935443ede2c5-workload-certs\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:42:45.512616 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.512596    4276 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5155d374-e42a-4c40-9485-935443ede2c5-istio-data\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:42:45.512675 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.512623    4276 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5155d374-e42a-4c40-9485-935443ede2c5-istio-token\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:42:45.512675 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.512640    4276 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5155d374-e42a-4c40-9485-935443ede2c5-credential-socket\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:42:45.512675 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.512654    4276 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5155d374-e42a-4c40-9485-935443ede2c5-istio-envoy\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:42:45.512675 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.512670    4276 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5155d374-e42a-4c40-9485-935443ede2c5-istio-podinfo\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:42:45.512872 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.512685    4276 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5155d374-e42a-4c40-9485-935443ede2c5-istiod-ca-cert\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:42:45.512872 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.512701    4276 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pcq9f\" (UniqueName: \"kubernetes.io/projected/5155d374-e42a-4c40-9485-935443ede2c5-kube-api-access-pcq9f\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:42:45.525064 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.525042    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:42:45.579526 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.579503    4276 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-bwdwd"]
Oct 31 17:42:45.588810 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.588635    4276 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-bwdwd"]
Oct 31 17:42:45.615178 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.615027    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v7knh\" (UniqueName: \"kubernetes.io/projected/653b8145-5630-4b28-b3c8-624fb92032e3-kube-api-access-v7knh\") pod \"maas-api-revoke-sub-0a3ab6fd-62wxv\" (UID: \"653b8145-5630-4b28-b3c8-624fb92032e3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0a3ab6fd-62wxv"
Oct 31 17:42:45.615178 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.615128    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/653b8145-5630-4b28-b3c8-624fb92032e3-maas-api-xks-ca\") pod \"maas-api-revoke-sub-0a3ab6fd-62wxv\" (UID: \"653b8145-5630-4b28-b3c8-624fb92032e3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0a3ab6fd-62wxv"
Oct 31 17:42:45.615654 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.615388    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/653b8145-5630-4b28-b3c8-624fb92032e3-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-0a3ab6fd-62wxv\" (UID: \"653b8145-5630-4b28-b3c8-624fb92032e3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0a3ab6fd-62wxv"
Oct 31 17:42:45.617149 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.617129    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/653b8145-5630-4b28-b3c8-624fb92032e3-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-0a3ab6fd-62wxv\" (UID: \"653b8145-5630-4b28-b3c8-624fb92032e3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0a3ab6fd-62wxv"
Oct 31 17:42:45.619213 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.619185    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/653b8145-5630-4b28-b3c8-624fb92032e3-maas-api-xks-ca\") pod \"maas-api-revoke-sub-0a3ab6fd-62wxv\" (UID: \"653b8145-5630-4b28-b3c8-624fb92032e3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0a3ab6fd-62wxv"
Oct 31 17:42:45.638077 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.638023    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v7knh\" (UniqueName: \"kubernetes.io/projected/653b8145-5630-4b28-b3c8-624fb92032e3-kube-api-access-v7knh\") pod \"maas-api-revoke-sub-0a3ab6fd-62wxv\" (UID: \"653b8145-5630-4b28-b3c8-624fb92032e3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0a3ab6fd-62wxv"
Oct 31 17:42:45.693348 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.693321    4276 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-rfwx5"]
Oct 31 17:42:45.696212 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.696190    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0a3ab6fd-62wxv"
Oct 31 17:42:45.700609 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.700578    4276 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-cf0a86-openshift-default-595bfdc58c-rfwx5"]
Oct 31 17:42:45.829711 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.829627    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-f61ac551-sc5kq"]
Oct 31 17:42:45.842731 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.842629    4276 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 31 17:42:45.880653 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:45.880620    4276 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-nqt2x"
Oct 31 17:42:46.000655 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:46.000628    4276 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-0e59ca-77546c9db5-b8b5g"
Oct 31 17:42:46.024815 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:42:46.024791    4276 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod653b8145_5630_4b28_b3c8_624fb92032e3.slice/crio-c056b6e3301dbd1c102b2f06c3bc975bb709405471db10f0298bec78a48e527c WatchSource:0}: Error finding container c056b6e3301dbd1c102b2f06c3bc975bb709405471db10f0298bec78a48e527c: Status 404 returned error can't find the container with id c056b6e3301dbd1c102b2f06c3bc975bb709405471db10f0298bec78a48e527c
Oct 31 17:42:46.025073 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:46.025046    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-0a3ab6fd-62wxv"]
Oct 31 17:42:46.027402 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:46.027377    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f2899dc7-04e8-40b9-ac3f-6cf6cb019020-kube-api-access-gbw6m\" (UniqueName: \"kubernetes.io/projected/f2899dc7-04e8-40b9-ac3f-6cf6cb019020-kube-api-access-gbw6m\") pod \"f2899dc7-04e8-40b9-ac3f-6cf6cb019020\" (UID: \"f2899dc7-04e8-40b9-ac3f-6cf6cb019020\") "
Oct 31 17:42:46.027549 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:46.027469    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f2899dc7-04e8-40b9-ac3f-6cf6cb019020-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f2899dc7-04e8-40b9-ac3f-6cf6cb019020-istiod-ca-cert\") pod \"f2899dc7-04e8-40b9-ac3f-6cf6cb019020\" (UID: \"f2899dc7-04e8-40b9-ac3f-6cf6cb019020\") "
Oct 31 17:42:46.027549 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:46.027515    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/f2899dc7-04e8-40b9-ac3f-6cf6cb019020-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f2899dc7-04e8-40b9-ac3f-6cf6cb019020-istio-podinfo\") pod \"f2899dc7-04e8-40b9-ac3f-6cf6cb019020\" (UID: \"f2899dc7-04e8-40b9-ac3f-6cf6cb019020\") "
Oct 31 17:42:46.027623 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:46.027550    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f2899dc7-04e8-40b9-ac3f-6cf6cb019020-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f2899dc7-04e8-40b9-ac3f-6cf6cb019020-credential-socket\") pod \"f2899dc7-04e8-40b9-ac3f-6cf6cb019020\" (UID: \"f2899dc7-04e8-40b9-ac3f-6cf6cb019020\") "
Oct 31 17:42:46.027623 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:46.027584    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f2899dc7-04e8-40b9-ac3f-6cf6cb019020-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f2899dc7-04e8-40b9-ac3f-6cf6cb019020-workload-socket\") pod \"f2899dc7-04e8-40b9-ac3f-6cf6cb019020\" (UID: \"f2899dc7-04e8-40b9-ac3f-6cf6cb019020\") "
Oct 31 17:42:46.027681 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:46.027619    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f2899dc7-04e8-40b9-ac3f-6cf6cb019020-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f2899dc7-04e8-40b9-ac3f-6cf6cb019020-istio-data\") pod \"f2899dc7-04e8-40b9-ac3f-6cf6cb019020\" (UID: \"f2899dc7-04e8-40b9-ac3f-6cf6cb019020\") "
Oct 31 17:42:46.027766 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:46.027739    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f2899dc7-04e8-40b9-ac3f-6cf6cb019020-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f2899dc7-04e8-40b9-ac3f-6cf6cb019020-workload-certs\") pod \"f2899dc7-04e8-40b9-ac3f-6cf6cb019020\" (UID: \"f2899dc7-04e8-40b9-ac3f-6cf6cb019020\") "
Oct 31 17:42:46.027830 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:46.027792    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f2899dc7-04e8-40b9-ac3f-6cf6cb019020-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f2899dc7-04e8-40b9-ac3f-6cf6cb019020-istio-envoy\") pod \"f2899dc7-04e8-40b9-ac3f-6cf6cb019020\" (UID: \"f2899dc7-04e8-40b9-ac3f-6cf6cb019020\") "
Oct 31 17:42:46.027943 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:46.027901    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f2899dc7-04e8-40b9-ac3f-6cf6cb019020-istio-token\" (UniqueName: \"kubernetes.io/projected/f2899dc7-04e8-40b9-ac3f-6cf6cb019020-istio-token\") pod \"f2899dc7-04e8-40b9-ac3f-6cf6cb019020\" (UID: \"f2899dc7-04e8-40b9-ac3f-6cf6cb019020\") "
Oct 31 17:42:46.028108 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:46.028089    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f2899dc7-04e8-40b9-ac3f-6cf6cb019020-credential-socket" pod "f2899dc7-04e8-40b9-ac3f-6cf6cb019020" (UID: "f2899dc7-04e8-40b9-ac3f-6cf6cb019020"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:42:46.028836 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:46.028572    4276 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f2899dc7-04e8-40b9-ac3f-6cf6cb019020-credential-socket\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:42:46.028836 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:46.028623    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f2899dc7-04e8-40b9-ac3f-6cf6cb019020-istio-data" pod "f2899dc7-04e8-40b9-ac3f-6cf6cb019020" (UID: "f2899dc7-04e8-40b9-ac3f-6cf6cb019020"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:42:46.028836 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:46.028798    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f2899dc7-04e8-40b9-ac3f-6cf6cb019020-workload-socket" pod "f2899dc7-04e8-40b9-ac3f-6cf6cb019020" (UID: "f2899dc7-04e8-40b9-ac3f-6cf6cb019020"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:42:46.029011 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:46.028976    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f2899dc7-04e8-40b9-ac3f-6cf6cb019020-workload-certs" pod "f2899dc7-04e8-40b9-ac3f-6cf6cb019020" (UID: "f2899dc7-04e8-40b9-ac3f-6cf6cb019020"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:42:46.029673 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:46.029382    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f2899dc7-04e8-40b9-ac3f-6cf6cb019020-istiod-ca-cert" pod "f2899dc7-04e8-40b9-ac3f-6cf6cb019020" (UID: "f2899dc7-04e8-40b9-ac3f-6cf6cb019020"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 17:42:46.031267 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:46.031244    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f2899dc7-04e8-40b9-ac3f-6cf6cb019020-kube-api-access-gbw6m" pod "f2899dc7-04e8-40b9-ac3f-6cf6cb019020" (UID: "f2899dc7-04e8-40b9-ac3f-6cf6cb019020"). InnerVolumeSpecName "kube-api-access-gbw6m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 17:42:46.038709 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:46.038401    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f2899dc7-04e8-40b9-ac3f-6cf6cb019020-istio-podinfo" pod "f2899dc7-04e8-40b9-ac3f-6cf6cb019020" (UID: "f2899dc7-04e8-40b9-ac3f-6cf6cb019020"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 17:42:46.039021 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:46.039000    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f2899dc7-04e8-40b9-ac3f-6cf6cb019020-istio-envoy" pod "f2899dc7-04e8-40b9-ac3f-6cf6cb019020" (UID: "f2899dc7-04e8-40b9-ac3f-6cf6cb019020"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:42:46.042160 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:46.042136    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f2899dc7-04e8-40b9-ac3f-6cf6cb019020-istio-token" pod "f2899dc7-04e8-40b9-ac3f-6cf6cb019020" (UID: "f2899dc7-04e8-40b9-ac3f-6cf6cb019020"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 17:42:46.076927 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:46.076899    4276 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-hl287"
Oct 31 17:42:46.126827 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:46.126791    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:42:46.129043 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:46.129015    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/daf53be5-f832-4696-96b7-310c33b09f03-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/daf53be5-f832-4696-96b7-310c33b09f03-workload-certs\") pod \"daf53be5-f832-4696-96b7-310c33b09f03\" (UID: \"daf53be5-f832-4696-96b7-310c33b09f03\") "
Oct 31 17:42:46.129191 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:46.129159    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/daf53be5-f832-4696-96b7-310c33b09f03-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/daf53be5-f832-4696-96b7-310c33b09f03-istiod-ca-cert\") pod \"daf53be5-f832-4696-96b7-310c33b09f03\" (UID: \"daf53be5-f832-4696-96b7-310c33b09f03\") "
Oct 31 17:42:46.129258 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:46.129190    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/daf53be5-f832-4696-96b7-310c33b09f03-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/daf53be5-f832-4696-96b7-310c33b09f03-istio-podinfo\") pod \"daf53be5-f832-4696-96b7-310c33b09f03\" (UID: \"daf53be5-f832-4696-96b7-310c33b09f03\") "
Oct 31 17:42:46.129315 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:46.129265    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/daf53be5-f832-4696-96b7-310c33b09f03-kube-api-access-q4hzg\" (UniqueName: \"kubernetes.io/projected/daf53be5-f832-4696-96b7-310c33b09f03-kube-api-access-q4hzg\") pod \"daf53be5-f832-4696-96b7-310c33b09f03\" (UID: \"daf53be5-f832-4696-96b7-310c33b09f03\") "
Oct 31 17:42:46.129315 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:46.129295    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/daf53be5-f832-4696-96b7-310c33b09f03-workload-certs" pod "daf53be5-f832-4696-96b7-310c33b09f03" (UID: "daf53be5-f832-4696-96b7-310c33b09f03"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:42:46.129428 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:46.129314    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/daf53be5-f832-4696-96b7-310c33b09f03-istio-token\" (UniqueName: \"kubernetes.io/projected/daf53be5-f832-4696-96b7-310c33b09f03-istio-token\") pod \"daf53be5-f832-4696-96b7-310c33b09f03\" (UID: \"daf53be5-f832-4696-96b7-310c33b09f03\") "
Oct 31 17:42:46.129428 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:46.129350    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ca2c3199-efa6-450a-8c46-343e39810896-metrics-certs\" (UniqueName: \"kubernetes.io/secret/ca2c3199-efa6-450a-8c46-343e39810896-metrics-certs\") pod \"ca2c3199-efa6-450a-8c46-343e39810896\" (UID: \"ca2c3199-efa6-450a-8c46-343e39810896\") "
Oct 31 17:42:46.129428 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:46.129412    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/daf53be5-f832-4696-96b7-310c33b09f03-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/daf53be5-f832-4696-96b7-310c33b09f03-credential-socket\") pod \"daf53be5-f832-4696-96b7-310c33b09f03\" (UID: \"daf53be5-f832-4696-96b7-310c33b09f03\") "
Oct 31 17:42:46.129764 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:46.129512    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/daf53be5-f832-4696-96b7-310c33b09f03-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/daf53be5-f832-4696-96b7-310c33b09f03-istio-data\") pod \"daf53be5-f832-4696-96b7-310c33b09f03\" (UID: \"daf53be5-f832-4696-96b7-310c33b09f03\") "
Oct 31 17:42:46.129764 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:46.129534    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ca2c3199-efa6-450a-8c46-343e39810896-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ca2c3199-efa6-450a-8c46-343e39810896-maas-api-tls\") pod \"ca2c3199-efa6-450a-8c46-343e39810896\" (UID: \"ca2c3199-efa6-450a-8c46-343e39810896\") "
Oct 31 17:42:46.129764 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:46.129564    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ca2c3199-efa6-450a-8c46-343e39810896-kube-api-access-xgvc4\" (UniqueName: \"kubernetes.io/projected/ca2c3199-efa6-450a-8c46-343e39810896-kube-api-access-xgvc4\") pod \"ca2c3199-efa6-450a-8c46-343e39810896\" (UID: \"ca2c3199-efa6-450a-8c46-343e39810896\") "
Oct 31 17:42:46.130205 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:46.130183    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ca2c3199-efa6-450a-8c46-343e39810896-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca2c3199-efa6-450a-8c46-343e39810896-openshift-service-ca\") pod \"ca2c3199-efa6-450a-8c46-343e39810896\" (UID: \"ca2c3199-efa6-450a-8c46-343e39810896\") "
Oct 31 17:42:46.130276 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:46.130241    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/daf53be5-f832-4696-96b7-310c33b09f03-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/daf53be5-f832-4696-96b7-310c33b09f03-istio-envoy\") pod \"daf53be5-f832-4696-96b7-310c33b09f03\" (UID: \"daf53be5-f832-4696-96b7-310c33b09f03\") "
Oct 31 17:42:46.130338 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:46.130294    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/daf53be5-f832-4696-96b7-310c33b09f03-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/daf53be5-f832-4696-96b7-310c33b09f03-workload-socket\") pod \"daf53be5-f832-4696-96b7-310c33b09f03\" (UID: \"daf53be5-f832-4696-96b7-310c33b09f03\") "
Oct 31 17:42:46.130507 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:46.130484    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/daf53be5-f832-4696-96b7-310c33b09f03-istiod-ca-cert" pod "daf53be5-f832-4696-96b7-310c33b09f03" (UID: "daf53be5-f832-4696-96b7-310c33b09f03"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 17:42:46.130910 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:46.130877    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ca2c3199-efa6-450a-8c46-343e39810896-openshift-service-ca" pod "ca2c3199-efa6-450a-8c46-343e39810896" (UID: "ca2c3199-efa6-450a-8c46-343e39810896"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 17:42:46.131066 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:46.131041    4276 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/daf53be5-f832-4696-96b7-310c33b09f03-workload-certs\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:42:46.131157 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:46.131071    4276 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gbw6m\" (UniqueName: \"kubernetes.io/projected/f2899dc7-04e8-40b9-ac3f-6cf6cb019020-kube-api-access-gbw6m\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:42:46.131157 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:46.131085    4276 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f2899dc7-04e8-40b9-ac3f-6cf6cb019020-istiod-ca-cert\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:42:46.131157 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:46.131100    4276 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f2899dc7-04e8-40b9-ac3f-6cf6cb019020-istio-podinfo\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:42:46.131157 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:46.131113    4276 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f2899dc7-04e8-40b9-ac3f-6cf6cb019020-workload-socket\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:42:46.131157 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:46.131126    4276 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f2899dc7-04e8-40b9-ac3f-6cf6cb019020-istio-data\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:42:46.131157 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:46.131140    4276 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/daf53be5-f832-4696-96b7-310c33b09f03-istiod-ca-cert\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:42:46.131157 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:46.131152    4276 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f2899dc7-04e8-40b9-ac3f-6cf6cb019020-workload-certs\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:42:46.131661 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:46.131164    4276 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f2899dc7-04e8-40b9-ac3f-6cf6cb019020-istio-envoy\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:42:46.131661 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:46.131177    4276 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f2899dc7-04e8-40b9-ac3f-6cf6cb019020-istio-token\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:42:46.131661 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:46.131616    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/daf53be5-f832-4696-96b7-310c33b09f03-credential-socket" pod "daf53be5-f832-4696-96b7-310c33b09f03" (UID: "daf53be5-f832-4696-96b7-310c33b09f03"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:42:46.131661 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:46.131645    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/daf53be5-f832-4696-96b7-310c33b09f03-workload-socket" pod "daf53be5-f832-4696-96b7-310c33b09f03" (UID: "daf53be5-f832-4696-96b7-310c33b09f03"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:42:46.131908 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:46.131741    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/daf53be5-f832-4696-96b7-310c33b09f03-istio-data" pod "daf53be5-f832-4696-96b7-310c33b09f03" (UID: "daf53be5-f832-4696-96b7-310c33b09f03"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:42:46.132322 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:46.132285    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/daf53be5-f832-4696-96b7-310c33b09f03-istio-podinfo" pod "daf53be5-f832-4696-96b7-310c33b09f03" (UID: "daf53be5-f832-4696-96b7-310c33b09f03"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 17:42:46.132759 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:46.132722    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ca2c3199-efa6-450a-8c46-343e39810896-kube-api-access-xgvc4" pod "ca2c3199-efa6-450a-8c46-343e39810896" (UID: "ca2c3199-efa6-450a-8c46-343e39810896"). InnerVolumeSpecName "kube-api-access-xgvc4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 17:42:46.132865 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:46.132783    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ca2c3199-efa6-450a-8c46-343e39810896-metrics-certs" pod "ca2c3199-efa6-450a-8c46-343e39810896" (UID: "ca2c3199-efa6-450a-8c46-343e39810896"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 17:42:46.134004 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:46.133977    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/daf53be5-f832-4696-96b7-310c33b09f03-istio-token" pod "daf53be5-f832-4696-96b7-310c33b09f03" (UID: "daf53be5-f832-4696-96b7-310c33b09f03"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 17:42:46.134159 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:46.134140    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ca2c3199-efa6-450a-8c46-343e39810896-maas-api-tls" pod "ca2c3199-efa6-450a-8c46-343e39810896" (UID: "ca2c3199-efa6-450a-8c46-343e39810896"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 17:42:46.134338 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:46.134316    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/daf53be5-f832-4696-96b7-310c33b09f03-istio-envoy" pod "daf53be5-f832-4696-96b7-310c33b09f03" (UID: "daf53be5-f832-4696-96b7-310c33b09f03"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:42:46.135901 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:46.135876    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/daf53be5-f832-4696-96b7-310c33b09f03-kube-api-access-q4hzg" pod "daf53be5-f832-4696-96b7-310c33b09f03" (UID: "daf53be5-f832-4696-96b7-310c33b09f03"). InnerVolumeSpecName "kube-api-access-q4hzg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 17:42:46.139102 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:46.139081    4276 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-jxpd8"
Oct 31 17:42:46.232021 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:46.231949    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9d24572d-ec8f-4108-a83e-704c5ae976ff-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d24572d-ec8f-4108-a83e-704c5ae976ff-credential-socket\") pod \"9d24572d-ec8f-4108-a83e-704c5ae976ff\" (UID: \"9d24572d-ec8f-4108-a83e-704c5ae976ff\") "
Oct 31 17:42:46.232021 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:46.231984    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9d24572d-ec8f-4108-a83e-704c5ae976ff-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d24572d-ec8f-4108-a83e-704c5ae976ff-workload-socket\") pod \"9d24572d-ec8f-4108-a83e-704c5ae976ff\" (UID: \"9d24572d-ec8f-4108-a83e-704c5ae976ff\") "
Oct 31 17:42:46.232021 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:46.232007    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9d24572d-ec8f-4108-a83e-704c5ae976ff-istio-token\" (UniqueName: \"kubernetes.io/projected/9d24572d-ec8f-4108-a83e-704c5ae976ff-istio-token\") pod \"9d24572d-ec8f-4108-a83e-704c5ae976ff\" (UID: \"9d24572d-ec8f-4108-a83e-704c5ae976ff\") "
Oct 31 17:42:46.232289 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:46.232074    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9d24572d-ec8f-4108-a83e-704c5ae976ff-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9d24572d-ec8f-4108-a83e-704c5ae976ff-workload-certs\") pod \"9d24572d-ec8f-4108-a83e-704c5ae976ff\" (UID: \"9d24572d-ec8f-4108-a83e-704c5ae976ff\") "
Oct 31 17:42:46.232289 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:46.232101    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9d24572d-ec8f-4108-a83e-704c5ae976ff-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9d24572d-ec8f-4108-a83e-704c5ae976ff-istio-envoy\") pod \"9d24572d-ec8f-4108-a83e-704c5ae976ff\" (UID: \"9d24572d-ec8f-4108-a83e-704c5ae976ff\") "
Oct 31 17:42:46.232289 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:46.232135    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9d24572d-ec8f-4108-a83e-704c5ae976ff-kube-api-access-sfs6z\" (UniqueName: \"kubernetes.io/projected/9d24572d-ec8f-4108-a83e-704c5ae976ff-kube-api-access-sfs6z\") pod \"9d24572d-ec8f-4108-a83e-704c5ae976ff\" (UID: \"9d24572d-ec8f-4108-a83e-704c5ae976ff\") "
Oct 31 17:42:46.232289 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:46.232218    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/9d24572d-ec8f-4108-a83e-704c5ae976ff-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9d24572d-ec8f-4108-a83e-704c5ae976ff-istio-podinfo\") pod \"9d24572d-ec8f-4108-a83e-704c5ae976ff\" (UID: \"9d24572d-ec8f-4108-a83e-704c5ae976ff\") "
Oct 31 17:42:46.232456 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:46.232297    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d24572d-ec8f-4108-a83e-704c5ae976ff-credential-socket" pod "9d24572d-ec8f-4108-a83e-704c5ae976ff" (UID: "9d24572d-ec8f-4108-a83e-704c5ae976ff"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:42:46.232456 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:46.232310    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d24572d-ec8f-4108-a83e-704c5ae976ff-workload-socket" pod "9d24572d-ec8f-4108-a83e-704c5ae976ff" (UID: "9d24572d-ec8f-4108-a83e-704c5ae976ff"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:42:46.232456 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:46.232405    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9d24572d-ec8f-4108-a83e-704c5ae976ff-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9d24572d-ec8f-4108-a83e-704c5ae976ff-istiod-ca-cert\") pod \"9d24572d-ec8f-4108-a83e-704c5ae976ff\" (UID: \"9d24572d-ec8f-4108-a83e-704c5ae976ff\") "
Oct 31 17:42:46.232620 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:46.232485    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9d24572d-ec8f-4108-a83e-704c5ae976ff-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9d24572d-ec8f-4108-a83e-704c5ae976ff-istio-data\") pod \"9d24572d-ec8f-4108-a83e-704c5ae976ff\" (UID: \"9d24572d-ec8f-4108-a83e-704c5ae976ff\") "
Oct 31 17:42:46.232774 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:46.232751    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d24572d-ec8f-4108-a83e-704c5ae976ff-istio-data" pod "9d24572d-ec8f-4108-a83e-704c5ae976ff" (UID: "9d24572d-ec8f-4108-a83e-704c5ae976ff"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:42:46.232774 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:46.232757    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9d24572d-ec8f-4108-a83e-704c5ae976ff-istiod-ca-cert" pod "9d24572d-ec8f-4108-a83e-704c5ae976ff" (UID: "9d24572d-ec8f-4108-a83e-704c5ae976ff"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 17:42:46.233011 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:46.232988    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d24572d-ec8f-4108-a83e-704c5ae976ff-workload-certs" pod "9d24572d-ec8f-4108-a83e-704c5ae976ff" (UID: "9d24572d-ec8f-4108-a83e-704c5ae976ff"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:42:46.233308 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:46.233287    4276 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/daf53be5-f832-4696-96b7-310c33b09f03-istio-token\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:42:46.233361 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:46.233316    4276 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ca2c3199-efa6-450a-8c46-343e39810896-metrics-certs\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:42:46.233361 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:46.233331    4276 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/daf53be5-f832-4696-96b7-310c33b09f03-credential-socket\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:42:46.233361 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:46.233346    4276 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/daf53be5-f832-4696-96b7-310c33b09f03-istio-data\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:42:46.233470 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:46.233360    4276 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d24572d-ec8f-4108-a83e-704c5ae976ff-credential-socket\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:42:46.233470 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:46.233373    4276 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ca2c3199-efa6-450a-8c46-343e39810896-maas-api-tls\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:42:46.233470 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:46.233387    4276 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d24572d-ec8f-4108-a83e-704c5ae976ff-workload-socket\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:42:46.233470 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:46.233401    4276 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xgvc4\" (UniqueName: \"kubernetes.io/projected/ca2c3199-efa6-450a-8c46-343e39810896-kube-api-access-xgvc4\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:42:46.233470 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:46.233415    4276 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9d24572d-ec8f-4108-a83e-704c5ae976ff-workload-certs\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:42:46.233470 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:46.233429    4276 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca2c3199-efa6-450a-8c46-343e39810896-openshift-service-ca\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:42:46.233648 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:46.233474    4276 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/daf53be5-f832-4696-96b7-310c33b09f03-istio-envoy\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:42:46.233648 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:46.233489    4276 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/daf53be5-f832-4696-96b7-310c33b09f03-workload-socket\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:42:46.233648 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:46.233501    4276 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/daf53be5-f832-4696-96b7-310c33b09f03-istio-podinfo\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:42:46.233648 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:46.233514    4276 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9d24572d-ec8f-4108-a83e-704c5ae976ff-istiod-ca-cert\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:42:46.233648 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:46.233527    4276 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9d24572d-ec8f-4108-a83e-704c5ae976ff-istio-data\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:42:46.233648 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:46.233538    4276 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q4hzg\" (UniqueName: \"kubernetes.io/projected/daf53be5-f832-4696-96b7-310c33b09f03-kube-api-access-q4hzg\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:42:46.234887 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:46.234863    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9d24572d-ec8f-4108-a83e-704c5ae976ff-istio-podinfo" pod "9d24572d-ec8f-4108-a83e-704c5ae976ff" (UID: "9d24572d-ec8f-4108-a83e-704c5ae976ff"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 17:42:46.235022 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:46.234915    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9d24572d-ec8f-4108-a83e-704c5ae976ff-kube-api-access-sfs6z" pod "9d24572d-ec8f-4108-a83e-704c5ae976ff" (UID: "9d24572d-ec8f-4108-a83e-704c5ae976ff"). InnerVolumeSpecName "kube-api-access-sfs6z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 17:42:46.235466 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:46.235432    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d24572d-ec8f-4108-a83e-704c5ae976ff-istio-envoy" pod "9d24572d-ec8f-4108-a83e-704c5ae976ff" (UID: "9d24572d-ec8f-4108-a83e-704c5ae976ff"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:42:46.235960 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:46.235943    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9d24572d-ec8f-4108-a83e-704c5ae976ff-istio-token" pod "9d24572d-ec8f-4108-a83e-704c5ae976ff" (UID: "9d24572d-ec8f-4108-a83e-704c5ae976ff"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 17:42:46.335198 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:46.335167    4276 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9d24572d-ec8f-4108-a83e-704c5ae976ff-istio-token\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:42:46.335198 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:46.335192    4276 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9d24572d-ec8f-4108-a83e-704c5ae976ff-istio-envoy\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:42:46.335198 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:46.335204    4276 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sfs6z\" (UniqueName: \"kubernetes.io/projected/9d24572d-ec8f-4108-a83e-704c5ae976ff-kube-api-access-sfs6z\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:42:46.335601 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:46.335213    4276 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9d24572d-ec8f-4108-a83e-704c5ae976ff-istio-podinfo\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:42:46.347395 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:46.347366    4276 generic.go:356] "Generic (PLEG): container finished" podID="9d24572d-ec8f-4108-a83e-704c5ae976ff" containerID="0fd95e192f8f3065b709ebc370709b230b43de2bd09f51864596dd4bad244101" exitCode=0
Oct 31 17:42:46.347531 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:46.347418    4276 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-jxpd8"
Oct 31 17:42:46.347531 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:46.347480    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-jxpd8" event={"ID":"9d24572d-ec8f-4108-a83e-704c5ae976ff","Type":"ContainerDied","Data":"0fd95e192f8f3065b709ebc370709b230b43de2bd09f51864596dd4bad244101"}
Oct 31 17:42:46.347626 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:46.347542    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-jxpd8" event={"ID":"9d24572d-ec8f-4108-a83e-704c5ae976ff","Type":"ContainerDied","Data":"38671b61e1c1f0b72d1a7065c7eaac7e7a051a87dd9c87cbfb3e321ec89ade6f"}
Oct 31 17:42:46.347626 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:46.347568    4276 scope.go:122] "RemoveContainer" containerID="0fd95e192f8f3065b709ebc370709b230b43de2bd09f51864596dd4bad244101"
Oct 31 17:42:46.348872 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:46.348786    4276 generic.go:356] "Generic (PLEG): container finished" podID="f2899dc7-04e8-40b9-ac3f-6cf6cb019020" containerID="7ecff18217515087184c4171450b64676aee451bb8da3d527e4059ecb26fcfaf" exitCode=0
Oct 31 17:42:46.348872 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:46.348825    4276 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-nqt2x"
Oct 31 17:42:46.348998 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:46.348871    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-nqt2x" event={"ID":"f2899dc7-04e8-40b9-ac3f-6cf6cb019020","Type":"ContainerDied","Data":"7ecff18217515087184c4171450b64676aee451bb8da3d527e4059ecb26fcfaf"}
Oct 31 17:42:46.348998 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:46.348921    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-nqt2x" event={"ID":"f2899dc7-04e8-40b9-ac3f-6cf6cb019020","Type":"ContainerDied","Data":"96a5a35e488f47f60fecf7c894fcefac02d49d50013248f66e8d4844fec4aa9f"}
Oct 31 17:42:46.351536 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:46.351511    4276 generic.go:356] "Generic (PLEG): container finished" podID="ca2c3199-efa6-450a-8c46-343e39810896" containerID="d825eea5b72ce9edfb3548fad4864d78c62c96b9faa074322379ffa82a993849" exitCode=0
Oct 31 17:42:46.351655 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:46.351545    4276 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-0e59ca-77546c9db5-b8b5g"
Oct 31 17:42:46.351655 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:46.351571    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-0e59ca-77546c9db5-b8b5g" event={"ID":"ca2c3199-efa6-450a-8c46-343e39810896","Type":"ContainerDied","Data":"d825eea5b72ce9edfb3548fad4864d78c62c96b9faa074322379ffa82a993849"}
Oct 31 17:42:46.351655 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:46.351628    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-0e59ca-77546c9db5-b8b5g" event={"ID":"ca2c3199-efa6-450a-8c46-343e39810896","Type":"ContainerDied","Data":"a3ec4129734520d4222c317c30fb76919db56d61b26949f87723c35e00661a44"}
Oct 31 17:42:46.355384 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:46.355361    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f61ac551-sc5kq" event={"ID":"9c4606cd-947c-4226-8152-74f8870458e6","Type":"ContainerStarted","Data":"7ceb9a8c44a57a5c19b17420f646fa50ba86b4e81f2d219379f6a407f5bf2f88"}
Oct 31 17:42:46.358747 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:46.358728    4276 generic.go:356] "Generic (PLEG): container finished" podID="daf53be5-f832-4696-96b7-310c33b09f03" containerID="247fc7436ec961ce182dd395a926a2c8ef25f7cc3d398880c0503bd8774291ce" exitCode=0
Oct 31 17:42:46.358846 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:46.358819    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-hl287" event={"ID":"daf53be5-f832-4696-96b7-310c33b09f03","Type":"ContainerDied","Data":"247fc7436ec961ce182dd395a926a2c8ef25f7cc3d398880c0503bd8774291ce"}
Oct 31 17:42:46.358913 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:46.358858    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-hl287" event={"ID":"daf53be5-f832-4696-96b7-310c33b09f03","Type":"ContainerDied","Data":"50bdd6ba867785971a90e7baeb48a1885059eb1f77f71d5873016c03ddb9e397"}
Oct 31 17:42:46.358978 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:46.358923    4276 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-hl287"
Oct 31 17:42:46.360355 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:46.360129    4276 scope.go:122] "RemoveContainer" containerID="e75c910ccf22e78e7ef8f45fcb517c57021f73f93f4dbca44088425c83d8ae28"
Oct 31 17:42:46.361342 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:46.361324    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0a3ab6fd-62wxv" event={"ID":"653b8145-5630-4b28-b3c8-624fb92032e3","Type":"ContainerStarted","Data":"c056b6e3301dbd1c102b2f06c3bc975bb709405471db10f0298bec78a48e527c"}
Oct 31 17:42:46.369372 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:46.369357    4276 scope.go:122] "RemoveContainer" containerID="0fd95e192f8f3065b709ebc370709b230b43de2bd09f51864596dd4bad244101"
Oct 31 17:42:46.369625 ip-10-0-129-127 kubenswrapper[4276]: E1031 17:42:46.369609    4276 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0fd95e192f8f3065b709ebc370709b230b43de2bd09f51864596dd4bad244101\": container with ID starting with 0fd95e192f8f3065b709ebc370709b230b43de2bd09f51864596dd4bad244101 not found: ID does not exist" containerID="0fd95e192f8f3065b709ebc370709b230b43de2bd09f51864596dd4bad244101"
Oct 31 17:42:46.369676 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:46.369632    4276 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0fd95e192f8f3065b709ebc370709b230b43de2bd09f51864596dd4bad244101"} err="failed to get container status \"0fd95e192f8f3065b709ebc370709b230b43de2bd09f51864596dd4bad244101\": rpc error: code = NotFound desc = could not find container \"0fd95e192f8f3065b709ebc370709b230b43de2bd09f51864596dd4bad244101\": container with ID starting with 0fd95e192f8f3065b709ebc370709b230b43de2bd09f51864596dd4bad244101 not found: ID does not exist"
Oct 31 17:42:46.369676 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:46.369646    4276 scope.go:122] "RemoveContainer" containerID="e75c910ccf22e78e7ef8f45fcb517c57021f73f93f4dbca44088425c83d8ae28"
Oct 31 17:42:46.369871 ip-10-0-129-127 kubenswrapper[4276]: E1031 17:42:46.369854    4276 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e75c910ccf22e78e7ef8f45fcb517c57021f73f93f4dbca44088425c83d8ae28\": container with ID starting with e75c910ccf22e78e7ef8f45fcb517c57021f73f93f4dbca44088425c83d8ae28 not found: ID does not exist" containerID="e75c910ccf22e78e7ef8f45fcb517c57021f73f93f4dbca44088425c83d8ae28"
Oct 31 17:42:46.369907 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:46.369877    4276 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e75c910ccf22e78e7ef8f45fcb517c57021f73f93f4dbca44088425c83d8ae28"} err="failed to get container status \"e75c910ccf22e78e7ef8f45fcb517c57021f73f93f4dbca44088425c83d8ae28\": rpc error: code = NotFound desc = could not find container \"e75c910ccf22e78e7ef8f45fcb517c57021f73f93f4dbca44088425c83d8ae28\": container with ID starting with e75c910ccf22e78e7ef8f45fcb517c57021f73f93f4dbca44088425c83d8ae28 not found: ID does not exist"
Oct 31 17:42:46.369907 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:46.369892    4276 scope.go:122] "RemoveContainer" containerID="7ecff18217515087184c4171450b64676aee451bb8da3d527e4059ecb26fcfaf"
Oct 31 17:42:46.378244 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:46.378229    4276 scope.go:122] "RemoveContainer" containerID="7ecff18217515087184c4171450b64676aee451bb8da3d527e4059ecb26fcfaf"
Oct 31 17:42:46.378508 ip-10-0-129-127 kubenswrapper[4276]: E1031 17:42:46.378493    4276 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7ecff18217515087184c4171450b64676aee451bb8da3d527e4059ecb26fcfaf\": container with ID starting with 7ecff18217515087184c4171450b64676aee451bb8da3d527e4059ecb26fcfaf not found: ID does not exist" containerID="7ecff18217515087184c4171450b64676aee451bb8da3d527e4059ecb26fcfaf"
Oct 31 17:42:46.378573 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:46.378514    4276 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7ecff18217515087184c4171450b64676aee451bb8da3d527e4059ecb26fcfaf"} err="failed to get container status \"7ecff18217515087184c4171450b64676aee451bb8da3d527e4059ecb26fcfaf\": rpc error: code = NotFound desc = could not find container \"7ecff18217515087184c4171450b64676aee451bb8da3d527e4059ecb26fcfaf\": container with ID starting with 7ecff18217515087184c4171450b64676aee451bb8da3d527e4059ecb26fcfaf not found: ID does not exist"
Oct 31 17:42:46.378573 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:46.378529    4276 scope.go:122] "RemoveContainer" containerID="d825eea5b72ce9edfb3548fad4864d78c62c96b9faa074322379ffa82a993849"
Oct 31 17:42:46.386768 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:46.386752    4276 scope.go:122] "RemoveContainer" containerID="d825eea5b72ce9edfb3548fad4864d78c62c96b9faa074322379ffa82a993849"
Oct 31 17:42:46.387002 ip-10-0-129-127 kubenswrapper[4276]: E1031 17:42:46.386984    4276 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d825eea5b72ce9edfb3548fad4864d78c62c96b9faa074322379ffa82a993849\": container with ID starting with d825eea5b72ce9edfb3548fad4864d78c62c96b9faa074322379ffa82a993849 not found: ID does not exist" containerID="d825eea5b72ce9edfb3548fad4864d78c62c96b9faa074322379ffa82a993849"
Oct 31 17:42:46.387083 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:46.387012    4276 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d825eea5b72ce9edfb3548fad4864d78c62c96b9faa074322379ffa82a993849"} err="failed to get container status \"d825eea5b72ce9edfb3548fad4864d78c62c96b9faa074322379ffa82a993849\": rpc error: code = NotFound desc = could not find container \"d825eea5b72ce9edfb3548fad4864d78c62c96b9faa074322379ffa82a993849\": container with ID starting with d825eea5b72ce9edfb3548fad4864d78c62c96b9faa074322379ffa82a993849 not found: ID does not exist"
Oct 31 17:42:46.387083 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:46.387035    4276 scope.go:122] "RemoveContainer" containerID="247fc7436ec961ce182dd395a926a2c8ef25f7cc3d398880c0503bd8774291ce"
Oct 31 17:42:46.395187 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:46.395172    4276 scope.go:122] "RemoveContainer" containerID="247fc7436ec961ce182dd395a926a2c8ef25f7cc3d398880c0503bd8774291ce"
Oct 31 17:42:46.395467 ip-10-0-129-127 kubenswrapper[4276]: E1031 17:42:46.395430    4276 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"247fc7436ec961ce182dd395a926a2c8ef25f7cc3d398880c0503bd8774291ce\": container with ID starting with 247fc7436ec961ce182dd395a926a2c8ef25f7cc3d398880c0503bd8774291ce not found: ID does not exist" containerID="247fc7436ec961ce182dd395a926a2c8ef25f7cc3d398880c0503bd8774291ce"
Oct 31 17:42:46.395520 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:46.395471    4276 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"247fc7436ec961ce182dd395a926a2c8ef25f7cc3d398880c0503bd8774291ce"} err="failed to get container status \"247fc7436ec961ce182dd395a926a2c8ef25f7cc3d398880c0503bd8774291ce\": rpc error: code = NotFound desc = could not find container \"247fc7436ec961ce182dd395a926a2c8ef25f7cc3d398880c0503bd8774291ce\": container with ID starting with 247fc7436ec961ce182dd395a926a2c8ef25f7cc3d398880c0503bd8774291ce not found: ID does not exist"
Oct 31 17:42:46.403632 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:46.403608    4276 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-nqt2x"]
Oct 31 17:42:46.410706 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:46.410687    4276 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-nqt2x"]
Oct 31 17:42:46.467639 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:46.467610    4276 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-jxpd8"]
Oct 31 17:42:46.470479 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:46.470460    4276 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-jxpd8"]
Oct 31 17:42:46.508122 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:46.508102    4276 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-0e59ca-77546c9db5-b8b5g"]
Oct 31 17:42:46.515890 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:46.515872    4276 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-0e59ca-77546c9db5-b8b5g"]
Oct 31 17:42:46.552714 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:46.552693    4276 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-hl287"]
Oct 31 17:42:46.557964 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:46.557943    4276 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-5c446c-openshift-default-797c6dc855-hl287"]
Oct 31 17:42:46.948493 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:46.948129    4276 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-94vcs"]
Oct 31 17:42:46.950468 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:46.948815    4276 cpu_manager.go:414] "containerMap: removing container" podUID="9d24572d-ec8f-4108-a83e-704c5ae976ff" containerName="istio-proxy"
Oct 31 17:42:46.950468 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:46.948833    4276 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9d24572d-ec8f-4108-a83e-704c5ae976ff" containerName="istio-proxy"
Oct 31 17:42:46.950468 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:46.948846    4276 cpu_manager.go:414] "containerMap: removing container" podUID="9d24572d-ec8f-4108-a83e-704c5ae976ff" containerName="istio-proxy"
Oct 31 17:42:46.950468 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:46.948851    4276 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9d24572d-ec8f-4108-a83e-704c5ae976ff" containerName="istio-proxy"
Oct 31 17:42:46.950468 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:46.948876    4276 cpu_manager.go:414] "containerMap: removing container" podUID="f2899dc7-04e8-40b9-ac3f-6cf6cb019020" containerName="istio-proxy"
Oct 31 17:42:46.950468 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:46.948882    4276 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f2899dc7-04e8-40b9-ac3f-6cf6cb019020" containerName="istio-proxy"
Oct 31 17:42:46.950468 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:46.948900    4276 cpu_manager.go:414] "containerMap: removing container" podUID="daf53be5-f832-4696-96b7-310c33b09f03" containerName="istio-proxy"
Oct 31 17:42:46.950468 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:46.948906    4276 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="daf53be5-f832-4696-96b7-310c33b09f03" containerName="istio-proxy"
Oct 31 17:42:46.950468 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:46.948920    4276 cpu_manager.go:414] "containerMap: removing container" podUID="ca2c3199-efa6-450a-8c46-343e39810896" containerName="maas-api"
Oct 31 17:42:46.950468 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:46.948925    4276 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ca2c3199-efa6-450a-8c46-343e39810896" containerName="maas-api"
Oct 31 17:42:46.950468 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:46.949010    4276 memory_manager.go:365] "RemoveStaleState removing state" podUID="f2899dc7-04e8-40b9-ac3f-6cf6cb019020" containerName="istio-proxy"
Oct 31 17:42:46.950468 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:46.949020    4276 memory_manager.go:365] "RemoveStaleState removing state" podUID="daf53be5-f832-4696-96b7-310c33b09f03" containerName="istio-proxy"
Oct 31 17:42:46.950468 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:46.949027    4276 memory_manager.go:365] "RemoveStaleState removing state" podUID="ca2c3199-efa6-450a-8c46-343e39810896" containerName="maas-api"
Oct 31 17:42:46.950468 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:46.949034    4276 memory_manager.go:365] "RemoveStaleState removing state" podUID="9d24572d-ec8f-4108-a83e-704c5ae976ff" containerName="istio-proxy"
Oct 31 17:42:46.954540 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:46.953696    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-94vcs"
Oct 31 17:42:46.974425 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:46.972581    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-94vcs"]
Oct 31 17:42:46.984038 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:46.982418    4276 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-2v74j"]
Oct 31 17:42:46.985098 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:46.985074    4276 memory_manager.go:365] "RemoveStaleState removing state" podUID="9d24572d-ec8f-4108-a83e-704c5ae976ff" containerName="istio-proxy"
Oct 31 17:42:46.989108 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:46.988968    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-2v74j"
Oct 31 17:42:47.006380 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:47.004278    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-2v74j"]
Oct 31 17:42:47.063382 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:47.063355    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e948a32e-25a4-4d29-b9dd-01a0539d337c-istio-envoy\") pod \"e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-2v74j\" (UID: \"e948a32e-25a4-4d29-b9dd-01a0539d337c\") " pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-2v74j"
Oct 31 17:42:47.063655 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:47.063633    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4c3cb076-d13d-4212-8fbb-86c851b14fe7-workload-certs\") pod \"e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-94vcs\" (UID: \"4c3cb076-d13d-4212-8fbb-86c851b14fe7\") " pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-94vcs"
Oct 31 17:42:47.063828 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:47.063690    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4c3cb076-d13d-4212-8fbb-86c851b14fe7-istio-podinfo\") pod \"e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-94vcs\" (UID: \"4c3cb076-d13d-4212-8fbb-86c851b14fe7\") " pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-94vcs"
Oct 31 17:42:47.063828 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:47.063724    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c3cb076-d13d-4212-8fbb-86c851b14fe7-credential-socket\") pod \"e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-94vcs\" (UID: \"4c3cb076-d13d-4212-8fbb-86c851b14fe7\") " pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-94vcs"
Oct 31 17:42:47.063950 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:47.063825    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e948a32e-25a4-4d29-b9dd-01a0539d337c-credential-socket\") pod \"e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-2v74j\" (UID: \"e948a32e-25a4-4d29-b9dd-01a0539d337c\") " pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-2v74j"
Oct 31 17:42:47.063950 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:47.063854    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4c3cb076-d13d-4212-8fbb-86c851b14fe7-istiod-ca-cert\") pod \"e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-94vcs\" (UID: \"4c3cb076-d13d-4212-8fbb-86c851b14fe7\") " pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-94vcs"
Oct 31 17:42:47.063950 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:47.063878    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e948a32e-25a4-4d29-b9dd-01a0539d337c-workload-socket\") pod \"e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-2v74j\" (UID: \"e948a32e-25a4-4d29-b9dd-01a0539d337c\") " pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-2v74j"
Oct 31 17:42:47.063950 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:47.063917    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4c3cb076-d13d-4212-8fbb-86c851b14fe7-istio-envoy\") pod \"e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-94vcs\" (UID: \"4c3cb076-d13d-4212-8fbb-86c851b14fe7\") " pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-94vcs"
Oct 31 17:42:47.064180 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:47.063967    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e948a32e-25a4-4d29-b9dd-01a0539d337c-workload-certs\") pod \"e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-2v74j\" (UID: \"e948a32e-25a4-4d29-b9dd-01a0539d337c\") " pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-2v74j"
Oct 31 17:42:47.064180 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:47.063995    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c2x4b\" (UniqueName: \"kubernetes.io/projected/e948a32e-25a4-4d29-b9dd-01a0539d337c-kube-api-access-c2x4b\") pod \"e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-2v74j\" (UID: \"e948a32e-25a4-4d29-b9dd-01a0539d337c\") " pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-2v74j"
Oct 31 17:42:47.064180 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:47.064049    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e948a32e-25a4-4d29-b9dd-01a0539d337c-istio-token\") pod \"e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-2v74j\" (UID: \"e948a32e-25a4-4d29-b9dd-01a0539d337c\") " pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-2v74j"
Oct 31 17:42:47.064180 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:47.064080    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e948a32e-25a4-4d29-b9dd-01a0539d337c-istio-data\") pod \"e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-2v74j\" (UID: \"e948a32e-25a4-4d29-b9dd-01a0539d337c\") " pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-2v74j"
Oct 31 17:42:47.064180 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:47.064119    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e948a32e-25a4-4d29-b9dd-01a0539d337c-istiod-ca-cert\") pod \"e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-2v74j\" (UID: \"e948a32e-25a4-4d29-b9dd-01a0539d337c\") " pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-2v74j"
Oct 31 17:42:47.064180 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:47.064159    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e948a32e-25a4-4d29-b9dd-01a0539d337c-istio-podinfo\") pod \"e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-2v74j\" (UID: \"e948a32e-25a4-4d29-b9dd-01a0539d337c\") " pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-2v74j"
Oct 31 17:42:47.064511 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:47.064198    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c3cb076-d13d-4212-8fbb-86c851b14fe7-workload-socket\") pod \"e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-94vcs\" (UID: \"4c3cb076-d13d-4212-8fbb-86c851b14fe7\") " pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-94vcs"
Oct 31 17:42:47.064511 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:47.064228    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4c3cb076-d13d-4212-8fbb-86c851b14fe7-istio-data\") pod \"e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-94vcs\" (UID: \"4c3cb076-d13d-4212-8fbb-86c851b14fe7\") " pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-94vcs"
Oct 31 17:42:47.064511 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:47.064265    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lnk4k\" (UniqueName: \"kubernetes.io/projected/4c3cb076-d13d-4212-8fbb-86c851b14fe7-kube-api-access-lnk4k\") pod \"e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-94vcs\" (UID: \"4c3cb076-d13d-4212-8fbb-86c851b14fe7\") " pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-94vcs"
Oct 31 17:42:47.064511 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:47.064315    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4c3cb076-d13d-4212-8fbb-86c851b14fe7-istio-token\") pod \"e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-94vcs\" (UID: \"4c3cb076-d13d-4212-8fbb-86c851b14fe7\") " pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-94vcs"
Oct 31 17:42:47.167163 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:47.167133    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e948a32e-25a4-4d29-b9dd-01a0539d337c-credential-socket\") pod \"e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-2v74j\" (UID: \"e948a32e-25a4-4d29-b9dd-01a0539d337c\") " pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-2v74j"
Oct 31 17:42:47.167342 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:47.167172    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4c3cb076-d13d-4212-8fbb-86c851b14fe7-istiod-ca-cert\") pod \"e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-94vcs\" (UID: \"4c3cb076-d13d-4212-8fbb-86c851b14fe7\") " pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-94vcs"
Oct 31 17:42:47.167342 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:47.167207    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e948a32e-25a4-4d29-b9dd-01a0539d337c-workload-socket\") pod \"e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-2v74j\" (UID: \"e948a32e-25a4-4d29-b9dd-01a0539d337c\") " pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-2v74j"
Oct 31 17:42:47.167342 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:47.167229    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4c3cb076-d13d-4212-8fbb-86c851b14fe7-istio-envoy\") pod \"e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-94vcs\" (UID: \"4c3cb076-d13d-4212-8fbb-86c851b14fe7\") " pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-94vcs"
Oct 31 17:42:47.167342 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:47.167266    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e948a32e-25a4-4d29-b9dd-01a0539d337c-workload-certs\") pod \"e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-2v74j\" (UID: \"e948a32e-25a4-4d29-b9dd-01a0539d337c\") " pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-2v74j"
Oct 31 17:42:47.167342 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:47.167292    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c2x4b\" (UniqueName: \"kubernetes.io/projected/e948a32e-25a4-4d29-b9dd-01a0539d337c-kube-api-access-c2x4b\") pod \"e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-2v74j\" (UID: \"e948a32e-25a4-4d29-b9dd-01a0539d337c\") " pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-2v74j"
Oct 31 17:42:47.167342 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:47.167329    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e948a32e-25a4-4d29-b9dd-01a0539d337c-istio-token\") pod \"e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-2v74j\" (UID: \"e948a32e-25a4-4d29-b9dd-01a0539d337c\") " pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-2v74j"
Oct 31 17:42:47.167644 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:47.167353    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e948a32e-25a4-4d29-b9dd-01a0539d337c-istio-data\") pod \"e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-2v74j\" (UID: \"e948a32e-25a4-4d29-b9dd-01a0539d337c\") " pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-2v74j"
Oct 31 17:42:47.167644 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:47.167388    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e948a32e-25a4-4d29-b9dd-01a0539d337c-istiod-ca-cert\") pod \"e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-2v74j\" (UID: \"e948a32e-25a4-4d29-b9dd-01a0539d337c\") " pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-2v74j"
Oct 31 17:42:47.167644 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:47.167421    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e948a32e-25a4-4d29-b9dd-01a0539d337c-istio-podinfo\") pod \"e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-2v74j\" (UID: \"e948a32e-25a4-4d29-b9dd-01a0539d337c\") " pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-2v74j"
Oct 31 17:42:47.167644 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:47.167469    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c3cb076-d13d-4212-8fbb-86c851b14fe7-workload-socket\") pod \"e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-94vcs\" (UID: \"4c3cb076-d13d-4212-8fbb-86c851b14fe7\") " pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-94vcs"
Oct 31 17:42:47.167644 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:47.167501    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4c3cb076-d13d-4212-8fbb-86c851b14fe7-istio-data\") pod \"e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-94vcs\" (UID: \"4c3cb076-d13d-4212-8fbb-86c851b14fe7\") " pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-94vcs"
Oct 31 17:42:47.167644 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:47.167539    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lnk4k\" (UniqueName: \"kubernetes.io/projected/4c3cb076-d13d-4212-8fbb-86c851b14fe7-kube-api-access-lnk4k\") pod \"e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-94vcs\" (UID: \"4c3cb076-d13d-4212-8fbb-86c851b14fe7\") " pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-94vcs"
Oct 31 17:42:47.167644 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:47.167589    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4c3cb076-d13d-4212-8fbb-86c851b14fe7-istio-token\") pod \"e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-94vcs\" (UID: \"4c3cb076-d13d-4212-8fbb-86c851b14fe7\") " pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-94vcs"
Oct 31 17:42:47.167644 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:47.167617    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e948a32e-25a4-4d29-b9dd-01a0539d337c-istio-envoy\") pod \"e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-2v74j\" (UID: \"e948a32e-25a4-4d29-b9dd-01a0539d337c\") " pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-2v74j"
Oct 31 17:42:47.167644 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:47.167616    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e948a32e-25a4-4d29-b9dd-01a0539d337c-credential-socket\") pod \"e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-2v74j\" (UID: \"e948a32e-25a4-4d29-b9dd-01a0539d337c\") " pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-2v74j"
Oct 31 17:42:47.167923 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:47.167741    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4c3cb076-d13d-4212-8fbb-86c851b14fe7-workload-certs\") pod \"e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-94vcs\" (UID: \"4c3cb076-d13d-4212-8fbb-86c851b14fe7\") " pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-94vcs"
Oct 31 17:42:47.167923 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:47.167792    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4c3cb076-d13d-4212-8fbb-86c851b14fe7-istio-podinfo\") pod \"e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-94vcs\" (UID: \"4c3cb076-d13d-4212-8fbb-86c851b14fe7\") " pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-94vcs"
Oct 31 17:42:47.167923 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:47.167826    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c3cb076-d13d-4212-8fbb-86c851b14fe7-credential-socket\") pod \"e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-94vcs\" (UID: \"4c3cb076-d13d-4212-8fbb-86c851b14fe7\") " pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-94vcs"
Oct 31 17:42:47.167923 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:47.167851    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e948a32e-25a4-4d29-b9dd-01a0539d337c-istio-data\") pod \"e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-2v74j\" (UID: \"e948a32e-25a4-4d29-b9dd-01a0539d337c\") " pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-2v74j"
Oct 31 17:42:47.168194 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:47.168175    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e948a32e-25a4-4d29-b9dd-01a0539d337c-workload-socket\") pod \"e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-2v74j\" (UID: \"e948a32e-25a4-4d29-b9dd-01a0539d337c\") " pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-2v74j"
Oct 31 17:42:47.168622 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:47.168596    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c3cb076-d13d-4212-8fbb-86c851b14fe7-credential-socket\") pod \"e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-94vcs\" (UID: \"4c3cb076-d13d-4212-8fbb-86c851b14fe7\") " pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-94vcs"
Oct 31 17:42:47.168786 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:47.168746    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4c3cb076-d13d-4212-8fbb-86c851b14fe7-istio-data\") pod \"e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-94vcs\" (UID: \"4c3cb076-d13d-4212-8fbb-86c851b14fe7\") " pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-94vcs"
Oct 31 17:42:47.169011 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:47.168959    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4c3cb076-d13d-4212-8fbb-86c851b14fe7-workload-certs\") pod \"e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-94vcs\" (UID: \"4c3cb076-d13d-4212-8fbb-86c851b14fe7\") " pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-94vcs"
Oct 31 17:42:47.169224 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:47.169204    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e948a32e-25a4-4d29-b9dd-01a0539d337c-workload-certs\") pod \"e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-2v74j\" (UID: \"e948a32e-25a4-4d29-b9dd-01a0539d337c\") " pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-2v74j"
Oct 31 17:42:47.169628 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:47.169595    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4c3cb076-d13d-4212-8fbb-86c851b14fe7-istiod-ca-cert\") pod \"e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-94vcs\" (UID: \"4c3cb076-d13d-4212-8fbb-86c851b14fe7\") " pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-94vcs"
Oct 31 17:42:47.169825 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:47.169800    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e948a32e-25a4-4d29-b9dd-01a0539d337c-istiod-ca-cert\") pod \"e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-2v74j\" (UID: \"e948a32e-25a4-4d29-b9dd-01a0539d337c\") " pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-2v74j"
Oct 31 17:42:47.170083 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:47.170055    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c3cb076-d13d-4212-8fbb-86c851b14fe7-workload-socket\") pod \"e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-94vcs\" (UID: \"4c3cb076-d13d-4212-8fbb-86c851b14fe7\") " pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-94vcs"
Oct 31 17:42:47.171249 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:47.171224    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4c3cb076-d13d-4212-8fbb-86c851b14fe7-istio-envoy\") pod \"e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-94vcs\" (UID: \"4c3cb076-d13d-4212-8fbb-86c851b14fe7\") " pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-94vcs"
Oct 31 17:42:47.171948 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:47.171924    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e948a32e-25a4-4d29-b9dd-01a0539d337c-istio-envoy\") pod \"e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-2v74j\" (UID: \"e948a32e-25a4-4d29-b9dd-01a0539d337c\") " pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-2v74j"
Oct 31 17:42:47.172166 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:47.172141    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e948a32e-25a4-4d29-b9dd-01a0539d337c-istio-podinfo\") pod \"e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-2v74j\" (UID: \"e948a32e-25a4-4d29-b9dd-01a0539d337c\") " pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-2v74j"
Oct 31 17:42:47.175214 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:47.175191    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4c3cb076-d13d-4212-8fbb-86c851b14fe7-istio-podinfo\") pod \"e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-94vcs\" (UID: \"4c3cb076-d13d-4212-8fbb-86c851b14fe7\") " pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-94vcs"
Oct 31 17:42:47.194805 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:47.194778    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4c3cb076-d13d-4212-8fbb-86c851b14fe7-istio-token\") pod \"e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-94vcs\" (UID: \"4c3cb076-d13d-4212-8fbb-86c851b14fe7\") " pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-94vcs"
Oct 31 17:42:47.196215 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:47.196195    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c2x4b\" (UniqueName: \"kubernetes.io/projected/e948a32e-25a4-4d29-b9dd-01a0539d337c-kube-api-access-c2x4b\") pod \"e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-2v74j\" (UID: \"e948a32e-25a4-4d29-b9dd-01a0539d337c\") " pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-2v74j"
Oct 31 17:42:47.199538 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:47.199424    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e948a32e-25a4-4d29-b9dd-01a0539d337c-istio-token\") pod \"e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-2v74j\" (UID: \"e948a32e-25a4-4d29-b9dd-01a0539d337c\") " pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-2v74j"
Oct 31 17:42:47.202855 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:47.202831    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lnk4k\" (UniqueName: \"kubernetes.io/projected/4c3cb076-d13d-4212-8fbb-86c851b14fe7-kube-api-access-lnk4k\") pod \"e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-94vcs\" (UID: \"4c3cb076-d13d-4212-8fbb-86c851b14fe7\") " pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-94vcs"
Oct 31 17:42:47.280681 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:47.280649    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-94vcs"
Oct 31 17:42:47.310754 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:47.310726    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-2v74j"
Oct 31 17:42:47.386551 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:47.385109    4276 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="22af22dd-1130-409b-9a17-05229dc0de11" path="/var/lib/kubelet/pods/22af22dd-1130-409b-9a17-05229dc0de11/volumes"
Oct 31 17:42:47.386551 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:47.385648    4276 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5155d374-e42a-4c40-9485-935443ede2c5" path="/var/lib/kubelet/pods/5155d374-e42a-4c40-9485-935443ede2c5/volumes"
Oct 31 17:42:47.386551 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:47.386059    4276 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7ec5066a-6b4d-4669-97f5-8a46a9c99a1b" path="/var/lib/kubelet/pods/7ec5066a-6b4d-4669-97f5-8a46a9c99a1b/volumes"
Oct 31 17:42:47.387670 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:47.387134    4276 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9d24572d-ec8f-4108-a83e-704c5ae976ff" path="/var/lib/kubelet/pods/9d24572d-ec8f-4108-a83e-704c5ae976ff/volumes"
Oct 31 17:42:47.387670 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:47.387543    4276 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ca2c3199-efa6-450a-8c46-343e39810896" path="/var/lib/kubelet/pods/ca2c3199-efa6-450a-8c46-343e39810896/volumes"
Oct 31 17:42:47.387943 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:47.387930    4276 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d1bf33fd-97d5-462d-8c04-e2697fcf1b05" path="/var/lib/kubelet/pods/d1bf33fd-97d5-462d-8c04-e2697fcf1b05/volumes"
Oct 31 17:42:47.388998 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:47.388697    4276 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="daf53be5-f832-4696-96b7-310c33b09f03" path="/var/lib/kubelet/pods/daf53be5-f832-4696-96b7-310c33b09f03/volumes"
Oct 31 17:42:47.389141 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:47.389118    4276 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f2899dc7-04e8-40b9-ac3f-6cf6cb019020" path="/var/lib/kubelet/pods/f2899dc7-04e8-40b9-ac3f-6cf6cb019020/volumes"
Oct 31 17:42:47.400150 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:47.400044    4276 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-d80080-59b65f4495-h6gz8" podUID="6e4109e9-1a59-48c7-9143-e14a01758994" containerName="payload-pre-processing" containerID="cri-o://4a0dc01c7432bd39e6e6f75d3eac48c4c6452840954242f79abbd2bfb8d4600e" gracePeriod=25
Oct 31 17:42:47.629198 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:42:47.629161    4276 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c3cb076_d13d_4212_8fbb_86c851b14fe7.slice/crio-b4f6b85db1f316c047778c680a50778a4224607a471d957434b3ba5c1b189518 WatchSource:0}: Error finding container b4f6b85db1f316c047778c680a50778a4224607a471d957434b3ba5c1b189518: Status 404 returned error can't find the container with id b4f6b85db1f316c047778c680a50778a4224607a471d957434b3ba5c1b189518
Oct 31 17:42:47.629488 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:47.629462    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-94vcs"]
Oct 31 17:42:47.632314 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:47.632278    4276 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 17:42:47.632434 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:47.632358    4276 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 17:42:47.632434 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:47.632405    4276 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 17:42:47.667234 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:47.667206    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-2v74j"]
Oct 31 17:42:47.682636 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:47.679612    4276 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 17:42:47.682636 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:47.679691    4276 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 17:42:47.682636 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:47.679764    4276 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 17:42:48.404673 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:48.404629    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-2v74j" event={"ID":"e948a32e-25a4-4d29-b9dd-01a0539d337c","Type":"ContainerStarted","Data":"06a63ac7ce2591fda4bf36eab5086dc1977b339ae31f3a5f73921a0d03fe94c2"}
Oct 31 17:42:48.405144 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:48.404695    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-2v74j" event={"ID":"e948a32e-25a4-4d29-b9dd-01a0539d337c","Type":"ContainerStarted","Data":"593776d33d32ef9e459bdf8d3e69ddc5c8559b337359c156a6a4d00e748a32da"}
Oct 31 17:42:48.407364 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:48.407336    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-94vcs" event={"ID":"4c3cb076-d13d-4212-8fbb-86c851b14fe7","Type":"ContainerStarted","Data":"f55bc0bfbf838cd79b47049ad0a717173d0a9bf489fa6d669724fff4c02d1776"}
Oct 31 17:42:48.407534 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:48.407385    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-94vcs" event={"ID":"4c3cb076-d13d-4212-8fbb-86c851b14fe7","Type":"ContainerStarted","Data":"b4f6b85db1f316c047778c680a50778a4224607a471d957434b3ba5c1b189518"}
Oct 31 17:42:48.469470 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:48.453043    4276 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-2v74j" podStartSLOduration=2.453024295 podStartE2EDuration="2.453024295s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 17:42:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 17:42:48.450620392 +0000 UTC m=+931.248365343" watchObservedRunningTime="2026-10-31 17:42:48.453024295 +0000 UTC m=+931.250769223"
Oct 31 17:42:48.539590 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:48.539521    4276 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-94vcs" podStartSLOduration=2.539480307 podStartE2EDuration="2.539480307s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 17:42:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 17:42:48.518622253 +0000 UTC m=+931.316367181" watchObservedRunningTime="2026-10-31 17:42:48.539480307 +0000 UTC m=+931.337225239"
Oct 31 17:42:49.307487 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:49.306536    4276 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-94vcs"
Oct 31 17:42:49.307487 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:49.306718    4276 patch_prober.go:29] interesting pod/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-94vcs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" start-of-body=
Oct 31 17:42:49.307487 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:49.306768    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-94vcs" podUID="4c3cb076-d13d-4212-8fbb-86c851b14fe7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused"
Oct 31 17:42:49.312054 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:49.312021    4276 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-2v74j"
Oct 31 17:42:49.313576 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:49.313554    4276 patch_prober.go:29] interesting pod/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-2v74j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused" start-of-body=
Oct 31 17:42:49.313768 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:49.313743    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-2v74j" podUID="e948a32e-25a4-4d29-b9dd-01a0539d337c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused"
Oct 31 17:42:49.711881 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:49.711762    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-d80080-59b65f4495-h6gz8" podUID="6e4109e9-1a59-48c7-9143-e14a01758994" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 31 17:42:50.285512 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:50.284833    4276 patch_prober.go:29] interesting pod/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-94vcs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" start-of-body=
Oct 31 17:42:50.285512 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:50.284890    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-94vcs" podUID="4c3cb076-d13d-4212-8fbb-86c851b14fe7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused"
Oct 31 17:42:50.311351 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:50.311221    4276 patch_prober.go:29] interesting pod/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-2v74j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused" start-of-body=
Oct 31 17:42:50.311351 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:50.311265    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-2v74j" podUID="e948a32e-25a4-4d29-b9dd-01a0539d337c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused"
Oct 31 17:42:50.311351 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:50.311299    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:42:50.434731 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:50.434602    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:42:50.685475 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:50.675813    4276 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w6-d80080/distinct-w6-d80080-kserve-664576c8f5-6jc4d" podUID="b9c2f445-a191-4b00-802a-1b0ded068411" containerName="main" containerID="cri-o://423c6b9d0c9fafdfabe87587397a88aa7ad3576dc56f7b8b69c86bceb33fbf23" gracePeriod=45
Oct 31 17:42:51.045674 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:51.045648    4276 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-d80080/distinct-w6-d80080-kserve-664576c8f5-6jc4d"
Oct 31 17:42:51.110500 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:51.109868    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b9c2f445-a191-4b00-802a-1b0ded068411-tls-certs\" (UniqueName: \"kubernetes.io/secret/b9c2f445-a191-4b00-802a-1b0ded068411-tls-certs\") pod \"b9c2f445-a191-4b00-802a-1b0ded068411\" (UID: \"b9c2f445-a191-4b00-802a-1b0ded068411\") "
Oct 31 17:42:51.127433 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:51.127393    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b9c2f445-a191-4b00-802a-1b0ded068411-tls-certs" pod "b9c2f445-a191-4b00-802a-1b0ded068411" (UID: "b9c2f445-a191-4b00-802a-1b0ded068411"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 17:42:51.213014 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:51.212924    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b9c2f445-a191-4b00-802a-1b0ded068411-dshm\" (UniqueName: \"kubernetes.io/empty-dir/b9c2f445-a191-4b00-802a-1b0ded068411-dshm\") pod \"b9c2f445-a191-4b00-802a-1b0ded068411\" (UID: \"b9c2f445-a191-4b00-802a-1b0ded068411\") "
Oct 31 17:42:51.213014 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:51.213000    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b9c2f445-a191-4b00-802a-1b0ded068411-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b9c2f445-a191-4b00-802a-1b0ded068411-tmp-dir\") pod \"b9c2f445-a191-4b00-802a-1b0ded068411\" (UID: \"b9c2f445-a191-4b00-802a-1b0ded068411\") "
Oct 31 17:42:51.213244 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:51.213055    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b9c2f445-a191-4b00-802a-1b0ded068411-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b9c2f445-a191-4b00-802a-1b0ded068411-model-cache\") pod \"b9c2f445-a191-4b00-802a-1b0ded068411\" (UID: \"b9c2f445-a191-4b00-802a-1b0ded068411\") "
Oct 31 17:42:51.213244 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:51.213202    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b9c2f445-a191-4b00-802a-1b0ded068411-kube-api-access-48bv5\" (UniqueName: \"kubernetes.io/projected/b9c2f445-a191-4b00-802a-1b0ded068411-kube-api-access-48bv5\") pod \"b9c2f445-a191-4b00-802a-1b0ded068411\" (UID: \"b9c2f445-a191-4b00-802a-1b0ded068411\") "
Oct 31 17:42:51.213356 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:51.213283    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b9c2f445-a191-4b00-802a-1b0ded068411-home\" (UniqueName: \"kubernetes.io/empty-dir/b9c2f445-a191-4b00-802a-1b0ded068411-home\") pod \"b9c2f445-a191-4b00-802a-1b0ded068411\" (UID: \"b9c2f445-a191-4b00-802a-1b0ded068411\") "
Oct 31 17:42:51.213899 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:51.213709    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b9c2f445-a191-4b00-802a-1b0ded068411-tmp-dir" pod "b9c2f445-a191-4b00-802a-1b0ded068411" (UID: "b9c2f445-a191-4b00-802a-1b0ded068411"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:42:51.214020 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:51.213933    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b9c2f445-a191-4b00-802a-1b0ded068411-home" pod "b9c2f445-a191-4b00-802a-1b0ded068411" (UID: "b9c2f445-a191-4b00-802a-1b0ded068411"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:42:51.214085 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:51.214065    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b9c2f445-a191-4b00-802a-1b0ded068411-model-cache" pod "b9c2f445-a191-4b00-802a-1b0ded068411" (UID: "b9c2f445-a191-4b00-802a-1b0ded068411"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:42:51.214250 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:51.214231    4276 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b9c2f445-a191-4b00-802a-1b0ded068411-home\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:42:51.214309 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:51.214256    4276 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b9c2f445-a191-4b00-802a-1b0ded068411-tls-certs\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:42:51.214309 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:51.214271    4276 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b9c2f445-a191-4b00-802a-1b0ded068411-tmp-dir\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:42:51.214309 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:51.214283    4276 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b9c2f445-a191-4b00-802a-1b0ded068411-model-cache\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:42:51.216105 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:51.216080    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b9c2f445-a191-4b00-802a-1b0ded068411-dshm" pod "b9c2f445-a191-4b00-802a-1b0ded068411" (UID: "b9c2f445-a191-4b00-802a-1b0ded068411"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:42:51.216911 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:51.216887    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b9c2f445-a191-4b00-802a-1b0ded068411-kube-api-access-48bv5" pod "b9c2f445-a191-4b00-802a-1b0ded068411" (UID: "b9c2f445-a191-4b00-802a-1b0ded068411"). InnerVolumeSpecName "kube-api-access-48bv5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 17:42:51.285146 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:51.283857    4276 patch_prober.go:29] interesting pod/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-94vcs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" start-of-body=
Oct 31 17:42:51.285146 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:51.283915    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-94vcs" podUID="4c3cb076-d13d-4212-8fbb-86c851b14fe7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused"
Oct 31 17:42:51.311933 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:51.311901    4276 patch_prober.go:29] interesting pod/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-2v74j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused" start-of-body=
Oct 31 17:42:51.312112 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:51.311949    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-2v74j" podUID="e948a32e-25a4-4d29-b9dd-01a0539d337c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused"
Oct 31 17:42:51.315095 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:51.315071    4276 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-48bv5\" (UniqueName: \"kubernetes.io/projected/b9c2f445-a191-4b00-802a-1b0ded068411-kube-api-access-48bv5\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:42:51.315095 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:51.315096    4276 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b9c2f445-a191-4b00-802a-1b0ded068411-dshm\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:42:51.444892 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:51.444858    4276 generic.go:356] "Generic (PLEG): container finished" podID="9c4606cd-947c-4226-8152-74f8870458e6" containerID="3dc6908f46c76cdd2f58bb838c9aa84727471f93069a53ce36c5a8b0d9cfc74a" exitCode=0
Oct 31 17:42:51.445052 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:51.444960    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f61ac551-sc5kq" event={"ID":"9c4606cd-947c-4226-8152-74f8870458e6","Type":"ContainerDied","Data":"3dc6908f46c76cdd2f58bb838c9aa84727471f93069a53ce36c5a8b0d9cfc74a"}
Oct 31 17:42:51.447277 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:51.447250    4276 generic.go:356] "Generic (PLEG): container finished" podID="653b8145-5630-4b28-b3c8-624fb92032e3" containerID="823176f79415fffcda43732788e03905152faf95571fd7a2f72569d59ccde231" exitCode=0
Oct 31 17:42:51.447434 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:51.447416    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0a3ab6fd-62wxv" event={"ID":"653b8145-5630-4b28-b3c8-624fb92032e3","Type":"ContainerDied","Data":"823176f79415fffcda43732788e03905152faf95571fd7a2f72569d59ccde231"}
Oct 31 17:42:51.448801 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:51.448780    4276 generic.go:356] "Generic (PLEG): container finished" podID="b9c2f445-a191-4b00-802a-1b0ded068411" containerID="423c6b9d0c9fafdfabe87587397a88aa7ad3576dc56f7b8b69c86bceb33fbf23" exitCode=0
Oct 31 17:42:51.448885 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:51.448837    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-d80080/distinct-w6-d80080-kserve-664576c8f5-6jc4d" event={"ID":"b9c2f445-a191-4b00-802a-1b0ded068411","Type":"ContainerDied","Data":"423c6b9d0c9fafdfabe87587397a88aa7ad3576dc56f7b8b69c86bceb33fbf23"}
Oct 31 17:42:51.448885 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:51.448857    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-d80080/distinct-w6-d80080-kserve-664576c8f5-6jc4d" event={"ID":"b9c2f445-a191-4b00-802a-1b0ded068411","Type":"ContainerDied","Data":"99d360bd5b1b96dd72f3b1c35791cba7629c1f6416af3b08729d9157d1a97097"}
Oct 31 17:42:51.448885 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:51.448873    4276 scope.go:122] "RemoveContainer" containerID="423c6b9d0c9fafdfabe87587397a88aa7ad3576dc56f7b8b69c86bceb33fbf23"
Oct 31 17:42:51.449051 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:51.448978    4276 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-d80080/distinct-w6-d80080-kserve-664576c8f5-6jc4d"
Oct 31 17:42:51.537406 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:51.537373    4276 scope.go:122] "RemoveContainer" containerID="423c6b9d0c9fafdfabe87587397a88aa7ad3576dc56f7b8b69c86bceb33fbf23"
Oct 31 17:42:51.538278 ip-10-0-129-127 kubenswrapper[4276]: E1031 17:42:51.537882    4276 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"423c6b9d0c9fafdfabe87587397a88aa7ad3576dc56f7b8b69c86bceb33fbf23\": container with ID starting with 423c6b9d0c9fafdfabe87587397a88aa7ad3576dc56f7b8b69c86bceb33fbf23 not found: ID does not exist" containerID="423c6b9d0c9fafdfabe87587397a88aa7ad3576dc56f7b8b69c86bceb33fbf23"
Oct 31 17:42:51.538278 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:51.537924    4276 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"423c6b9d0c9fafdfabe87587397a88aa7ad3576dc56f7b8b69c86bceb33fbf23"} err="failed to get container status \"423c6b9d0c9fafdfabe87587397a88aa7ad3576dc56f7b8b69c86bceb33fbf23\": rpc error: code = NotFound desc = could not find container \"423c6b9d0c9fafdfabe87587397a88aa7ad3576dc56f7b8b69c86bceb33fbf23\": container with ID starting with 423c6b9d0c9fafdfabe87587397a88aa7ad3576dc56f7b8b69c86bceb33fbf23 not found: ID does not exist"
Oct 31 17:42:51.622194 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:51.622162    4276 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w6-d80080/distinct-w6-d80080-kserve-664576c8f5-6jc4d"]
Oct 31 17:42:51.624848 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:51.624828    4276 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w6-d80080/distinct-w6-d80080-kserve-664576c8f5-6jc4d"]
Oct 31 17:42:51.978466 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:51.978387    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:42:52.110589 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:52.110552    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:42:52.291522 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:52.288579    4276 patch_prober.go:29] interesting pod/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-94vcs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" start-of-body=
Oct 31 17:42:52.291522 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:52.288637    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-94vcs" podUID="4c3cb076-d13d-4212-8fbb-86c851b14fe7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused"
Oct 31 17:42:52.291522 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:52.289027    4276 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-fgvsp"]
Oct 31 17:42:52.291522 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:52.289266    4276 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-fgvsp" podUID="d46713a5-7bba-4dfb-b77f-8fae0b39271e" containerName="istio-proxy" containerID="cri-o://80478be634138f6b35e4e949b2560185d9c16c8c3466e1bcc2b3320bbff17148" gracePeriod=30
Oct 31 17:42:52.311835 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:52.311557    4276 patch_prober.go:29] interesting pod/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-2v74j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused" start-of-body=
Oct 31 17:42:52.311835 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:52.311620    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-2v74j" podUID="e948a32e-25a4-4d29-b9dd-01a0539d337c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused"
Oct 31 17:42:52.312606 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:52.312568    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:42:52.324913 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:52.324171    4276 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-9lts8"]
Oct 31 17:42:52.324913 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:52.324393    4276 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-9lts8" podUID="79a9c2b2-8458-4a74-afde-ee00374beb5b" containerName="istio-proxy" containerID="cri-o://086d73b8f16757218462e0350e934c0a704569110746210e0031445ea87e9d90" gracePeriod=30
Oct 31 17:42:52.330476 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:52.330433    4276 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-f8xx7"]
Oct 31 17:42:52.330721 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:52.330693    4276 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-f8xx7" podUID="6418b389-728b-4040-9b73-50cf013c00f3" containerName="istio-proxy" containerID="cri-o://704cbbdc75a171c7759a0446b73f665cfcb7e7ae497a3c319af6a38912b2391a" gracePeriod=30
Oct 31 17:42:53.084496 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:53.070618    4276 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f61ac551-sc5kq"
Oct 31 17:42:53.112361 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:53.110049    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9c4606cd-947c-4226-8152-74f8870458e6-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/9c4606cd-947c-4226-8152-74f8870458e6-maas-api-openshift-ca\") pod \"9c4606cd-947c-4226-8152-74f8870458e6\" (UID: \"9c4606cd-947c-4226-8152-74f8870458e6\") "
Oct 31 17:42:53.112361 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:53.110136    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9c4606cd-947c-4226-8152-74f8870458e6-kube-api-access-7qp4b\" (UniqueName: \"kubernetes.io/projected/9c4606cd-947c-4226-8152-74f8870458e6-kube-api-access-7qp4b\") pod \"9c4606cd-947c-4226-8152-74f8870458e6\" (UID: \"9c4606cd-947c-4226-8152-74f8870458e6\") "
Oct 31 17:42:53.112361 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:53.110207    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9c4606cd-947c-4226-8152-74f8870458e6-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/9c4606cd-947c-4226-8152-74f8870458e6-maas-api-xks-ca\") pod \"9c4606cd-947c-4226-8152-74f8870458e6\" (UID: \"9c4606cd-947c-4226-8152-74f8870458e6\") "
Oct 31 17:42:53.112361 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:53.110840    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9c4606cd-947c-4226-8152-74f8870458e6-maas-api-openshift-ca" pod "9c4606cd-947c-4226-8152-74f8870458e6" (UID: "9c4606cd-947c-4226-8152-74f8870458e6"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 17:42:53.112361 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:53.111031    4276 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/9c4606cd-947c-4226-8152-74f8870458e6-maas-api-openshift-ca\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:42:53.122610 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:53.119582    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9c4606cd-947c-4226-8152-74f8870458e6-maas-api-xks-ca" pod "9c4606cd-947c-4226-8152-74f8870458e6" (UID: "9c4606cd-947c-4226-8152-74f8870458e6"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 17:42:53.122610 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:53.122308    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:42:53.126732 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:53.126706    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9c4606cd-947c-4226-8152-74f8870458e6-kube-api-access-7qp4b" pod "9c4606cd-947c-4226-8152-74f8870458e6" (UID: "9c4606cd-947c-4226-8152-74f8870458e6"). InnerVolumeSpecName "kube-api-access-7qp4b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 17:42:53.168467 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:53.167630    4276 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0a3ab6fd-62wxv"
Oct 31 17:42:53.212031 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:53.211943    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/653b8145-5630-4b28-b3c8-624fb92032e3-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/653b8145-5630-4b28-b3c8-624fb92032e3-maas-api-xks-ca\") pod \"653b8145-5630-4b28-b3c8-624fb92032e3\" (UID: \"653b8145-5630-4b28-b3c8-624fb92032e3\") "
Oct 31 17:42:53.212470 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:53.212429    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/653b8145-5630-4b28-b3c8-624fb92032e3-kube-api-access-v7knh\" (UniqueName: \"kubernetes.io/projected/653b8145-5630-4b28-b3c8-624fb92032e3-kube-api-access-v7knh\") pod \"653b8145-5630-4b28-b3c8-624fb92032e3\" (UID: \"653b8145-5630-4b28-b3c8-624fb92032e3\") "
Oct 31 17:42:53.212583 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:53.212516    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/653b8145-5630-4b28-b3c8-624fb92032e3-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/653b8145-5630-4b28-b3c8-624fb92032e3-maas-api-openshift-ca\") pod \"653b8145-5630-4b28-b3c8-624fb92032e3\" (UID: \"653b8145-5630-4b28-b3c8-624fb92032e3\") "
Oct 31 17:42:53.213285 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:53.213255    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/653b8145-5630-4b28-b3c8-624fb92032e3-maas-api-openshift-ca" pod "653b8145-5630-4b28-b3c8-624fb92032e3" (UID: "653b8145-5630-4b28-b3c8-624fb92032e3"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 17:42:53.213809 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:53.213784    4276 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/653b8145-5630-4b28-b3c8-624fb92032e3-maas-api-openshift-ca\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:42:53.213939 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:53.213814    4276 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7qp4b\" (UniqueName: \"kubernetes.io/projected/9c4606cd-947c-4226-8152-74f8870458e6-kube-api-access-7qp4b\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:42:53.213939 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:53.213829    4276 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/9c4606cd-947c-4226-8152-74f8870458e6-maas-api-xks-ca\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:42:53.214163 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:53.214142    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/653b8145-5630-4b28-b3c8-624fb92032e3-maas-api-xks-ca" pod "653b8145-5630-4b28-b3c8-624fb92032e3" (UID: "653b8145-5630-4b28-b3c8-624fb92032e3"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 17:42:53.217183 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:53.217159    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/653b8145-5630-4b28-b3c8-624fb92032e3-kube-api-access-v7knh" pod "653b8145-5630-4b28-b3c8-624fb92032e3" (UID: "653b8145-5630-4b28-b3c8-624fb92032e3"). InnerVolumeSpecName "kube-api-access-v7knh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 17:42:53.285012 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:53.284870    4276 patch_prober.go:29] interesting pod/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-94vcs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" start-of-body=
Oct 31 17:42:53.285012 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:53.284946    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-94vcs" podUID="4c3cb076-d13d-4212-8fbb-86c851b14fe7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused"
Oct 31 17:42:53.311376 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:53.311337    4276 patch_prober.go:29] interesting pod/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-2v74j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused" start-of-body=
Oct 31 17:42:53.311601 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:53.311398    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-2v74j" podUID="e948a32e-25a4-4d29-b9dd-01a0539d337c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused"
Oct 31 17:42:53.314754 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:53.314728    4276 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/653b8145-5630-4b28-b3c8-624fb92032e3-maas-api-xks-ca\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:42:53.314754 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:53.314753    4276 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v7knh\" (UniqueName: \"kubernetes.io/projected/653b8145-5630-4b28-b3c8-624fb92032e3-kube-api-access-v7knh\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:42:53.408752 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:53.408720    4276 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b9c2f445-a191-4b00-802a-1b0ded068411" path="/var/lib/kubelet/pods/b9c2f445-a191-4b00-802a-1b0ded068411/volumes"
Oct 31 17:42:53.638136 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:53.638106    4276 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f61ac551-sc5kq"
Oct 31 17:42:53.638136 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:53.638125    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f61ac551-sc5kq" event={"ID":"9c4606cd-947c-4226-8152-74f8870458e6","Type":"ContainerDied","Data":"7ceb9a8c44a57a5c19b17420f646fa50ba86b4e81f2d219379f6a407f5bf2f88"}
Oct 31 17:42:53.638400 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:53.638166    4276 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7ceb9a8c44a57a5c19b17420f646fa50ba86b4e81f2d219379f6a407f5bf2f88"
Oct 31 17:42:53.639756 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:53.639741    4276 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0a3ab6fd-62wxv"
Oct 31 17:42:53.639991 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:53.639780    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0a3ab6fd-62wxv" event={"ID":"653b8145-5630-4b28-b3c8-624fb92032e3","Type":"ContainerDied","Data":"c056b6e3301dbd1c102b2f06c3bc975bb709405471db10f0298bec78a48e527c"}
Oct 31 17:42:53.640160 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:53.640143    4276 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c056b6e3301dbd1c102b2f06c3bc975bb709405471db10f0298bec78a48e527c"
Oct 31 17:42:54.281665 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:54.281627    4276 patch_prober.go:29] interesting pod/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-94vcs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" start-of-body=
Oct 31 17:42:54.282171 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:54.281689    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-94vcs" podUID="4c3cb076-d13d-4212-8fbb-86c851b14fe7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused"
Oct 31 17:42:54.312091 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:54.312059    4276 patch_prober.go:29] interesting pod/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-2v74j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused" start-of-body=
Oct 31 17:42:54.312240 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:54.312146    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-2v74j" podUID="e948a32e-25a4-4d29-b9dd-01a0539d337c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused"
Oct 31 17:42:54.773705 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:54.773666    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-b78c31-6b5448c9d7-rbhfz" podUID="f7894ac3-7f53-4a3f-8d20-17359e24f69e" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 31 17:42:55.315764 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:55.315733    4276 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-2v74j"
Oct 31 17:42:55.316151 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:55.316051    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-2v74j"
Oct 31 17:42:55.316896 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:55.316878    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-2v74j"
Oct 31 17:42:55.355846 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:55.355817    4276 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-94vcs"
Oct 31 17:42:55.356222 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:55.356197    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-94vcs"
Oct 31 17:42:55.357562 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:55.357540    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-94vcs"
Oct 31 17:42:55.830994 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:55.830968    4276 patch_prober.go:29] interesting pod/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-f8xx7 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 31 17:42:55.831166 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:55.831016    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-f8xx7" podUID="6418b389-728b-4040-9b73-50cf013c00f3" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 17:42:56.798332 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:56.798308    4276 patch_prober.go:29] interesting pod/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-9lts8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 31 17:42:56.798652 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:56.798360    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-9lts8" podUID="79a9c2b2-8458-4a74-afde-ee00374beb5b" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 17:42:57.025151 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:57.025128    4276 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-b78c31-6b5448c9d7-rbhfz"
Oct 31 17:42:57.063803 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:57.063780    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f7894ac3-7f53-4a3f-8d20-17359e24f69e-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f7894ac3-7f53-4a3f-8d20-17359e24f69e-plugins-config-volume\") pod \"f7894ac3-7f53-4a3f-8d20-17359e24f69e\" (UID: \"f7894ac3-7f53-4a3f-8d20-17359e24f69e\") "
Oct 31 17:42:57.064194 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:57.064158    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f7894ac3-7f53-4a3f-8d20-17359e24f69e-plugins-config-volume" pod "f7894ac3-7f53-4a3f-8d20-17359e24f69e" (UID: "f7894ac3-7f53-4a3f-8d20-17359e24f69e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 17:42:57.064194 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:57.064167    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f7894ac3-7f53-4a3f-8d20-17359e24f69e-kube-api-access-qk78z\" (UniqueName: \"kubernetes.io/projected/f7894ac3-7f53-4a3f-8d20-17359e24f69e-kube-api-access-qk78z\") pod \"f7894ac3-7f53-4a3f-8d20-17359e24f69e\" (UID: \"f7894ac3-7f53-4a3f-8d20-17359e24f69e\") "
Oct 31 17:42:57.064689 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:57.064665    4276 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f7894ac3-7f53-4a3f-8d20-17359e24f69e-plugins-config-volume\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:42:57.067522 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:57.067501    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f7894ac3-7f53-4a3f-8d20-17359e24f69e-kube-api-access-qk78z" pod "f7894ac3-7f53-4a3f-8d20-17359e24f69e" (UID: "f7894ac3-7f53-4a3f-8d20-17359e24f69e"). InnerVolumeSpecName "kube-api-access-qk78z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 17:42:57.165684 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:57.165659    4276 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qk78z\" (UniqueName: \"kubernetes.io/projected/f7894ac3-7f53-4a3f-8d20-17359e24f69e-kube-api-access-qk78z\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:42:57.567987 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:57.567957    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:42:57.634944 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:57.634911    4276 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-fgvsp"
Oct 31 17:42:57.655219 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:57.655197    4276 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-f8xx7"
Oct 31 17:42:57.667338 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:57.667312    4276 generic.go:356] "Generic (PLEG): container finished" podID="d46713a5-7bba-4dfb-b77f-8fae0b39271e" containerID="80478be634138f6b35e4e949b2560185d9c16c8c3466e1bcc2b3320bbff17148" exitCode=0
Oct 31 17:42:57.667476 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:57.667384    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-fgvsp" event={"ID":"d46713a5-7bba-4dfb-b77f-8fae0b39271e","Type":"ContainerDied","Data":"80478be634138f6b35e4e949b2560185d9c16c8c3466e1bcc2b3320bbff17148"}
Oct 31 17:42:57.667476 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:57.667430    4276 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-fgvsp"
Oct 31 17:42:57.667608 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:57.667503    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-fgvsp" event={"ID":"d46713a5-7bba-4dfb-b77f-8fae0b39271e","Type":"ContainerDied","Data":"216c2271f7cc28d5be2d78234482aabc108227f2ac122befeb33cfac184871ac"}
Oct 31 17:42:57.667608 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:57.667540    4276 scope.go:122] "RemoveContainer" containerID="80478be634138f6b35e4e949b2560185d9c16c8c3466e1bcc2b3320bbff17148"
Oct 31 17:42:57.669363 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:57.669342    4276 generic.go:356] "Generic (PLEG): container finished" podID="79a9c2b2-8458-4a74-afde-ee00374beb5b" containerID="086d73b8f16757218462e0350e934c0a704569110746210e0031445ea87e9d90" exitCode=0
Oct 31 17:42:57.669527 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:57.669428    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-9lts8" event={"ID":"79a9c2b2-8458-4a74-afde-ee00374beb5b","Type":"ContainerDied","Data":"086d73b8f16757218462e0350e934c0a704569110746210e0031445ea87e9d90"}
Oct 31 17:42:57.671114 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:57.670951    4276 generic.go:356] "Generic (PLEG): container finished" podID="f7894ac3-7f53-4a3f-8d20-17359e24f69e" containerID="9d88d61861ce7bf6e5f3c5befa55815e5e0b74178a89eccd2406911943fbe6b5" exitCode=0
Oct 31 17:42:57.671114 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:57.670982    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-b78c31-6b5448c9d7-rbhfz" event={"ID":"f7894ac3-7f53-4a3f-8d20-17359e24f69e","Type":"ContainerDied","Data":"9d88d61861ce7bf6e5f3c5befa55815e5e0b74178a89eccd2406911943fbe6b5"}
Oct 31 17:42:57.671114 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:57.670994    4276 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-b78c31-6b5448c9d7-rbhfz"
Oct 31 17:42:57.671114 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:57.671039    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-b78c31-6b5448c9d7-rbhfz" event={"ID":"f7894ac3-7f53-4a3f-8d20-17359e24f69e","Type":"ContainerDied","Data":"fe2323db58dbc676e651f9d07ce0d826eee4ced26901b1fa88931e8bb5eb551e"}
Oct 31 17:42:57.673013 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:57.672341    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d46713a5-7bba-4dfb-b77f-8fae0b39271e-istio-token\" (UniqueName: \"kubernetes.io/projected/d46713a5-7bba-4dfb-b77f-8fae0b39271e-istio-token\") pod \"d46713a5-7bba-4dfb-b77f-8fae0b39271e\" (UID: \"d46713a5-7bba-4dfb-b77f-8fae0b39271e\") "
Oct 31 17:42:57.673013 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:57.672392    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d46713a5-7bba-4dfb-b77f-8fae0b39271e-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d46713a5-7bba-4dfb-b77f-8fae0b39271e-istio-data\") pod \"d46713a5-7bba-4dfb-b77f-8fae0b39271e\" (UID: \"d46713a5-7bba-4dfb-b77f-8fae0b39271e\") "
Oct 31 17:42:57.673013 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:57.672517    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d46713a5-7bba-4dfb-b77f-8fae0b39271e-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d46713a5-7bba-4dfb-b77f-8fae0b39271e-workload-certs\") pod \"d46713a5-7bba-4dfb-b77f-8fae0b39271e\" (UID: \"d46713a5-7bba-4dfb-b77f-8fae0b39271e\") "
Oct 31 17:42:57.673013 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:57.672632    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/d46713a5-7bba-4dfb-b77f-8fae0b39271e-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d46713a5-7bba-4dfb-b77f-8fae0b39271e-istio-podinfo\") pod \"d46713a5-7bba-4dfb-b77f-8fae0b39271e\" (UID: \"d46713a5-7bba-4dfb-b77f-8fae0b39271e\") "
Oct 31 17:42:57.673013 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:57.672683    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d46713a5-7bba-4dfb-b77f-8fae0b39271e-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d46713a5-7bba-4dfb-b77f-8fae0b39271e-workload-socket\") pod \"d46713a5-7bba-4dfb-b77f-8fae0b39271e\" (UID: \"d46713a5-7bba-4dfb-b77f-8fae0b39271e\") "
Oct 31 17:42:57.673013 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:57.672708    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d46713a5-7bba-4dfb-b77f-8fae0b39271e-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d46713a5-7bba-4dfb-b77f-8fae0b39271e-istiod-ca-cert\") pod \"d46713a5-7bba-4dfb-b77f-8fae0b39271e\" (UID: \"d46713a5-7bba-4dfb-b77f-8fae0b39271e\") "
Oct 31 17:42:57.673013 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:57.672740    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d46713a5-7bba-4dfb-b77f-8fae0b39271e-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d46713a5-7bba-4dfb-b77f-8fae0b39271e-istio-envoy\") pod \"d46713a5-7bba-4dfb-b77f-8fae0b39271e\" (UID: \"d46713a5-7bba-4dfb-b77f-8fae0b39271e\") "
Oct 31 17:42:57.673013 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:57.672803    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d46713a5-7bba-4dfb-b77f-8fae0b39271e-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d46713a5-7bba-4dfb-b77f-8fae0b39271e-credential-socket\") pod \"d46713a5-7bba-4dfb-b77f-8fae0b39271e\" (UID: \"d46713a5-7bba-4dfb-b77f-8fae0b39271e\") "
Oct 31 17:42:57.673013 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:57.672868    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d46713a5-7bba-4dfb-b77f-8fae0b39271e-kube-api-access-g8pph\" (UniqueName: \"kubernetes.io/projected/d46713a5-7bba-4dfb-b77f-8fae0b39271e-kube-api-access-g8pph\") pod \"d46713a5-7bba-4dfb-b77f-8fae0b39271e\" (UID: \"d46713a5-7bba-4dfb-b77f-8fae0b39271e\") "
Oct 31 17:42:57.673531 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:57.673276    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d46713a5-7bba-4dfb-b77f-8fae0b39271e-istio-data" pod "d46713a5-7bba-4dfb-b77f-8fae0b39271e" (UID: "d46713a5-7bba-4dfb-b77f-8fae0b39271e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:42:57.673588 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:57.673573    4276 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d46713a5-7bba-4dfb-b77f-8fae0b39271e-istio-data\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:42:57.674905 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:57.674549    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d46713a5-7bba-4dfb-b77f-8fae0b39271e-istiod-ca-cert" pod "d46713a5-7bba-4dfb-b77f-8fae0b39271e" (UID: "d46713a5-7bba-4dfb-b77f-8fae0b39271e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 17:42:57.674905 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:57.674621    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d46713a5-7bba-4dfb-b77f-8fae0b39271e-workload-socket" pod "d46713a5-7bba-4dfb-b77f-8fae0b39271e" (UID: "d46713a5-7bba-4dfb-b77f-8fae0b39271e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:42:57.674905 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:57.674790    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d46713a5-7bba-4dfb-b77f-8fae0b39271e-workload-certs" pod "d46713a5-7bba-4dfb-b77f-8fae0b39271e" (UID: "d46713a5-7bba-4dfb-b77f-8fae0b39271e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:42:57.675452 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:57.675076    4276 generic.go:356] "Generic (PLEG): container finished" podID="6418b389-728b-4040-9b73-50cf013c00f3" containerID="704cbbdc75a171c7759a0446b73f665cfcb7e7ae497a3c319af6a38912b2391a" exitCode=0
Oct 31 17:42:57.675452 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:57.675122    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-f8xx7" event={"ID":"6418b389-728b-4040-9b73-50cf013c00f3","Type":"ContainerDied","Data":"704cbbdc75a171c7759a0446b73f665cfcb7e7ae497a3c319af6a38912b2391a"}
Oct 31 17:42:57.675452 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:57.675163    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-f8xx7" event={"ID":"6418b389-728b-4040-9b73-50cf013c00f3","Type":"ContainerDied","Data":"b4f7da10e113f7e2313e27cdadbe89772b76ca704549a277cb7c5505ff4c0465"}
Oct 31 17:42:57.675452 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:57.675213    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d46713a5-7bba-4dfb-b77f-8fae0b39271e-credential-socket" pod "d46713a5-7bba-4dfb-b77f-8fae0b39271e" (UID: "d46713a5-7bba-4dfb-b77f-8fae0b39271e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:42:57.675452 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:57.675230    4276 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-f8xx7"
Oct 31 17:42:57.675452 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:57.675347    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d46713a5-7bba-4dfb-b77f-8fae0b39271e-istio-token" pod "d46713a5-7bba-4dfb-b77f-8fae0b39271e" (UID: "d46713a5-7bba-4dfb-b77f-8fae0b39271e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 17:42:57.676708 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:57.675692    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d46713a5-7bba-4dfb-b77f-8fae0b39271e-kube-api-access-g8pph" pod "d46713a5-7bba-4dfb-b77f-8fae0b39271e" (UID: "d46713a5-7bba-4dfb-b77f-8fae0b39271e"). InnerVolumeSpecName "kube-api-access-g8pph". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 17:42:57.677109 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:57.677082    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d46713a5-7bba-4dfb-b77f-8fae0b39271e-istio-podinfo" pod "d46713a5-7bba-4dfb-b77f-8fae0b39271e" (UID: "d46713a5-7bba-4dfb-b77f-8fae0b39271e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 17:42:57.679304 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:57.679197    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d46713a5-7bba-4dfb-b77f-8fae0b39271e-istio-envoy" pod "d46713a5-7bba-4dfb-b77f-8fae0b39271e" (UID: "d46713a5-7bba-4dfb-b77f-8fae0b39271e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:42:57.686235 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:57.686213    4276 scope.go:122] "RemoveContainer" containerID="80478be634138f6b35e4e949b2560185d9c16c8c3466e1bcc2b3320bbff17148"
Oct 31 17:42:57.686560 ip-10-0-129-127 kubenswrapper[4276]: E1031 17:42:57.686540    4276 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"80478be634138f6b35e4e949b2560185d9c16c8c3466e1bcc2b3320bbff17148\": container with ID starting with 80478be634138f6b35e4e949b2560185d9c16c8c3466e1bcc2b3320bbff17148 not found: ID does not exist" containerID="80478be634138f6b35e4e949b2560185d9c16c8c3466e1bcc2b3320bbff17148"
Oct 31 17:42:57.686653 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:57.686568    4276 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"80478be634138f6b35e4e949b2560185d9c16c8c3466e1bcc2b3320bbff17148"} err="failed to get container status \"80478be634138f6b35e4e949b2560185d9c16c8c3466e1bcc2b3320bbff17148\": rpc error: code = NotFound desc = could not find container \"80478be634138f6b35e4e949b2560185d9c16c8c3466e1bcc2b3320bbff17148\": container with ID starting with 80478be634138f6b35e4e949b2560185d9c16c8c3466e1bcc2b3320bbff17148 not found: ID does not exist"
Oct 31 17:42:57.686653 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:57.686588    4276 scope.go:122] "RemoveContainer" containerID="9d88d61861ce7bf6e5f3c5befa55815e5e0b74178a89eccd2406911943fbe6b5"
Oct 31 17:42:57.703635 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:57.703618    4276 scope.go:122] "RemoveContainer" containerID="9d88d61861ce7bf6e5f3c5befa55815e5e0b74178a89eccd2406911943fbe6b5"
Oct 31 17:42:57.704016 ip-10-0-129-127 kubenswrapper[4276]: E1031 17:42:57.703992    4276 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9d88d61861ce7bf6e5f3c5befa55815e5e0b74178a89eccd2406911943fbe6b5\": container with ID starting with 9d88d61861ce7bf6e5f3c5befa55815e5e0b74178a89eccd2406911943fbe6b5 not found: ID does not exist" containerID="9d88d61861ce7bf6e5f3c5befa55815e5e0b74178a89eccd2406911943fbe6b5"
Oct 31 17:42:57.704132 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:57.704020    4276 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9d88d61861ce7bf6e5f3c5befa55815e5e0b74178a89eccd2406911943fbe6b5"} err="failed to get container status \"9d88d61861ce7bf6e5f3c5befa55815e5e0b74178a89eccd2406911943fbe6b5\": rpc error: code = NotFound desc = could not find container \"9d88d61861ce7bf6e5f3c5befa55815e5e0b74178a89eccd2406911943fbe6b5\": container with ID starting with 9d88d61861ce7bf6e5f3c5befa55815e5e0b74178a89eccd2406911943fbe6b5 not found: ID does not exist"
Oct 31 17:42:57.704132 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:57.704037    4276 scope.go:122] "RemoveContainer" containerID="704cbbdc75a171c7759a0446b73f665cfcb7e7ae497a3c319af6a38912b2391a"
Oct 31 17:42:57.715763 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:57.715742    4276 scope.go:122] "RemoveContainer" containerID="ab10caa28f333e0be2ae6f6f40c4d874ad5fcc690e27588b9729c82508d8e8a0"
Oct 31 17:42:57.737020 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:57.736952    4276 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w5-b78c31-6b5448c9d7-rbhfz"]
Oct 31 17:42:57.742223 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:57.742190    4276 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w5-b78c31-6b5448c9d7-rbhfz"]
Oct 31 17:42:57.767342 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:57.767238    4276 scope.go:122] "RemoveContainer" containerID="704cbbdc75a171c7759a0446b73f665cfcb7e7ae497a3c319af6a38912b2391a"
Oct 31 17:42:57.768085 ip-10-0-129-127 kubenswrapper[4276]: E1031 17:42:57.767783    4276 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"704cbbdc75a171c7759a0446b73f665cfcb7e7ae497a3c319af6a38912b2391a\": container with ID starting with 704cbbdc75a171c7759a0446b73f665cfcb7e7ae497a3c319af6a38912b2391a not found: ID does not exist" containerID="704cbbdc75a171c7759a0446b73f665cfcb7e7ae497a3c319af6a38912b2391a"
Oct 31 17:42:57.768085 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:57.767815    4276 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"704cbbdc75a171c7759a0446b73f665cfcb7e7ae497a3c319af6a38912b2391a"} err="failed to get container status \"704cbbdc75a171c7759a0446b73f665cfcb7e7ae497a3c319af6a38912b2391a\": rpc error: code = NotFound desc = could not find container \"704cbbdc75a171c7759a0446b73f665cfcb7e7ae497a3c319af6a38912b2391a\": container with ID starting with 704cbbdc75a171c7759a0446b73f665cfcb7e7ae497a3c319af6a38912b2391a not found: ID does not exist"
Oct 31 17:42:57.768085 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:57.768012    4276 scope.go:122] "RemoveContainer" containerID="ab10caa28f333e0be2ae6f6f40c4d874ad5fcc690e27588b9729c82508d8e8a0"
Oct 31 17:42:57.768995 ip-10-0-129-127 kubenswrapper[4276]: E1031 17:42:57.768934    4276 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ab10caa28f333e0be2ae6f6f40c4d874ad5fcc690e27588b9729c82508d8e8a0\": container with ID starting with ab10caa28f333e0be2ae6f6f40c4d874ad5fcc690e27588b9729c82508d8e8a0 not found: ID does not exist" containerID="ab10caa28f333e0be2ae6f6f40c4d874ad5fcc690e27588b9729c82508d8e8a0"
Oct 31 17:42:57.768995 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:57.768965    4276 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ab10caa28f333e0be2ae6f6f40c4d874ad5fcc690e27588b9729c82508d8e8a0"} err="failed to get container status \"ab10caa28f333e0be2ae6f6f40c4d874ad5fcc690e27588b9729c82508d8e8a0\": rpc error: code = NotFound desc = could not find container \"ab10caa28f333e0be2ae6f6f40c4d874ad5fcc690e27588b9729c82508d8e8a0\": container with ID starting with ab10caa28f333e0be2ae6f6f40c4d874ad5fcc690e27588b9729c82508d8e8a0 not found: ID does not exist"
Oct 31 17:42:57.806780 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:57.796900    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6418b389-728b-4040-9b73-50cf013c00f3-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6418b389-728b-4040-9b73-50cf013c00f3-istio-envoy\") pod \"6418b389-728b-4040-9b73-50cf013c00f3\" (UID: \"6418b389-728b-4040-9b73-50cf013c00f3\") "
Oct 31 17:42:57.806780 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:57.797063    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6418b389-728b-4040-9b73-50cf013c00f3-kube-api-access-tlzgw\" (UniqueName: \"kubernetes.io/projected/6418b389-728b-4040-9b73-50cf013c00f3-kube-api-access-tlzgw\") pod \"6418b389-728b-4040-9b73-50cf013c00f3\" (UID: \"6418b389-728b-4040-9b73-50cf013c00f3\") "
Oct 31 17:42:57.806780 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:57.797117    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6418b389-728b-4040-9b73-50cf013c00f3-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6418b389-728b-4040-9b73-50cf013c00f3-istiod-ca-cert\") pod \"6418b389-728b-4040-9b73-50cf013c00f3\" (UID: \"6418b389-728b-4040-9b73-50cf013c00f3\") "
Oct 31 17:42:57.806780 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:57.797171    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6418b389-728b-4040-9b73-50cf013c00f3-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6418b389-728b-4040-9b73-50cf013c00f3-workload-certs\") pod \"6418b389-728b-4040-9b73-50cf013c00f3\" (UID: \"6418b389-728b-4040-9b73-50cf013c00f3\") "
Oct 31 17:42:57.806780 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:57.797198    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6418b389-728b-4040-9b73-50cf013c00f3-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6418b389-728b-4040-9b73-50cf013c00f3-credential-socket\") pod \"6418b389-728b-4040-9b73-50cf013c00f3\" (UID: \"6418b389-728b-4040-9b73-50cf013c00f3\") "
Oct 31 17:42:57.806780 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:57.797259    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/6418b389-728b-4040-9b73-50cf013c00f3-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6418b389-728b-4040-9b73-50cf013c00f3-istio-podinfo\") pod \"6418b389-728b-4040-9b73-50cf013c00f3\" (UID: \"6418b389-728b-4040-9b73-50cf013c00f3\") "
Oct 31 17:42:57.806780 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:57.797331    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6418b389-728b-4040-9b73-50cf013c00f3-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6418b389-728b-4040-9b73-50cf013c00f3-istio-data\") pod \"6418b389-728b-4040-9b73-50cf013c00f3\" (UID: \"6418b389-728b-4040-9b73-50cf013c00f3\") "
Oct 31 17:42:57.806780 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:57.797368    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6418b389-728b-4040-9b73-50cf013c00f3-istio-token\" (UniqueName: \"kubernetes.io/projected/6418b389-728b-4040-9b73-50cf013c00f3-istio-token\") pod \"6418b389-728b-4040-9b73-50cf013c00f3\" (UID: \"6418b389-728b-4040-9b73-50cf013c00f3\") "
Oct 31 17:42:57.806780 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:57.797417    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6418b389-728b-4040-9b73-50cf013c00f3-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6418b389-728b-4040-9b73-50cf013c00f3-workload-socket\") pod \"6418b389-728b-4040-9b73-50cf013c00f3\" (UID: \"6418b389-728b-4040-9b73-50cf013c00f3\") "
Oct 31 17:42:57.806780 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:57.798078    4276 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g8pph\" (UniqueName: \"kubernetes.io/projected/d46713a5-7bba-4dfb-b77f-8fae0b39271e-kube-api-access-g8pph\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:42:57.806780 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:57.798100    4276 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d46713a5-7bba-4dfb-b77f-8fae0b39271e-istio-token\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:42:57.806780 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:57.798115    4276 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d46713a5-7bba-4dfb-b77f-8fae0b39271e-workload-certs\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:42:57.806780 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:57.798129    4276 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d46713a5-7bba-4dfb-b77f-8fae0b39271e-istio-podinfo\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:42:57.806780 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:57.798141    4276 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d46713a5-7bba-4dfb-b77f-8fae0b39271e-workload-socket\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:42:57.806780 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:57.798154    4276 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d46713a5-7bba-4dfb-b77f-8fae0b39271e-istiod-ca-cert\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:42:57.806780 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:57.798169    4276 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d46713a5-7bba-4dfb-b77f-8fae0b39271e-istio-envoy\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:42:57.806780 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:57.798183    4276 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d46713a5-7bba-4dfb-b77f-8fae0b39271e-credential-socket\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:42:57.817347 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:57.817120    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6418b389-728b-4040-9b73-50cf013c00f3-workload-certs" pod "6418b389-728b-4040-9b73-50cf013c00f3" (UID: "6418b389-728b-4040-9b73-50cf013c00f3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:42:57.817347 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:57.817280    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6418b389-728b-4040-9b73-50cf013c00f3-istiod-ca-cert" pod "6418b389-728b-4040-9b73-50cf013c00f3" (UID: "6418b389-728b-4040-9b73-50cf013c00f3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 17:42:57.817347 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:57.817326    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6418b389-728b-4040-9b73-50cf013c00f3-istio-data" pod "6418b389-728b-4040-9b73-50cf013c00f3" (UID: "6418b389-728b-4040-9b73-50cf013c00f3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:42:57.817693 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:57.817659    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6418b389-728b-4040-9b73-50cf013c00f3-credential-socket" pod "6418b389-728b-4040-9b73-50cf013c00f3" (UID: "6418b389-728b-4040-9b73-50cf013c00f3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:42:57.817827 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:57.817797    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6418b389-728b-4040-9b73-50cf013c00f3-workload-socket" pod "6418b389-728b-4040-9b73-50cf013c00f3" (UID: "6418b389-728b-4040-9b73-50cf013c00f3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:42:57.823838 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:57.823780    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6418b389-728b-4040-9b73-50cf013c00f3-istio-token" pod "6418b389-728b-4040-9b73-50cf013c00f3" (UID: "6418b389-728b-4040-9b73-50cf013c00f3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 17:42:57.824629 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:57.824604    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6418b389-728b-4040-9b73-50cf013c00f3-istio-envoy" pod "6418b389-728b-4040-9b73-50cf013c00f3" (UID: "6418b389-728b-4040-9b73-50cf013c00f3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:42:57.825347 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:57.825231    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6418b389-728b-4040-9b73-50cf013c00f3-istio-podinfo" pod "6418b389-728b-4040-9b73-50cf013c00f3" (UID: "6418b389-728b-4040-9b73-50cf013c00f3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 17:42:57.825347 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:57.825324    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6418b389-728b-4040-9b73-50cf013c00f3-kube-api-access-tlzgw" pod "6418b389-728b-4040-9b73-50cf013c00f3" (UID: "6418b389-728b-4040-9b73-50cf013c00f3"). InnerVolumeSpecName "kube-api-access-tlzgw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 17:42:57.952516 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:57.902149    4276 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tlzgw\" (UniqueName: \"kubernetes.io/projected/6418b389-728b-4040-9b73-50cf013c00f3-kube-api-access-tlzgw\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:42:57.952516 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:57.902191    4276 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6418b389-728b-4040-9b73-50cf013c00f3-istiod-ca-cert\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:42:57.952516 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:57.902207    4276 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6418b389-728b-4040-9b73-50cf013c00f3-workload-certs\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:42:57.952516 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:57.902221    4276 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6418b389-728b-4040-9b73-50cf013c00f3-credential-socket\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:42:57.952516 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:57.902235    4276 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6418b389-728b-4040-9b73-50cf013c00f3-istio-podinfo\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:42:57.952516 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:57.902247    4276 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6418b389-728b-4040-9b73-50cf013c00f3-istio-data\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:42:57.952516 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:57.902259    4276 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6418b389-728b-4040-9b73-50cf013c00f3-istio-token\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:42:57.952516 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:57.902270    4276 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6418b389-728b-4040-9b73-50cf013c00f3-workload-socket\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:42:57.952516 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:57.902282    4276 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6418b389-728b-4040-9b73-50cf013c00f3-istio-envoy\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:42:58.069511 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:58.019528    4276 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-1c0bfd-6569c7d6f-l29ff"]
Oct 31 17:42:58.069511 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:58.019770    4276 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-1c0bfd-6569c7d6f-l29ff" podUID="95755676-9f02-4b03-a7e5-f6c8ab4fa965" containerName="maas-api" containerID="cri-o://b1508c7bbf6311fa84ae2faa759301a8f5a726bee2c9b402767cd3fd5ef15bf1" gracePeriod=30
Oct 31 17:42:58.069511 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:58.044708    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:42:58.163928 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:58.163650    4276 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-f8xx7"]
Oct 31 17:42:58.182659 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:58.182411    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:42:58.183973 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:58.183949    4276 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-f8xx7"]
Oct 31 17:42:58.203127 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:58.203039    4276 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-fgvsp"]
Oct 31 17:42:58.210427 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:58.209389    4276 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-fgvsp"]
Oct 31 17:42:58.297504 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:58.297472    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:42:58.316245 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:58.316224    4276 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-9lts8"
Oct 31 17:42:58.386468 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:58.381848    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/79a9c2b2-8458-4a74-afde-ee00374beb5b-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/79a9c2b2-8458-4a74-afde-ee00374beb5b-istio-envoy\") pod \"79a9c2b2-8458-4a74-afde-ee00374beb5b\" (UID: \"79a9c2b2-8458-4a74-afde-ee00374beb5b\") "
Oct 31 17:42:58.386468 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:58.381898    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/79a9c2b2-8458-4a74-afde-ee00374beb5b-istio-token\" (UniqueName: \"kubernetes.io/projected/79a9c2b2-8458-4a74-afde-ee00374beb5b-istio-token\") pod \"79a9c2b2-8458-4a74-afde-ee00374beb5b\" (UID: \"79a9c2b2-8458-4a74-afde-ee00374beb5b\") "
Oct 31 17:42:58.386468 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:58.381963    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/79a9c2b2-8458-4a74-afde-ee00374beb5b-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/79a9c2b2-8458-4a74-afde-ee00374beb5b-istiod-ca-cert\") pod \"79a9c2b2-8458-4a74-afde-ee00374beb5b\" (UID: \"79a9c2b2-8458-4a74-afde-ee00374beb5b\") "
Oct 31 17:42:58.386468 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:58.382036    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/79a9c2b2-8458-4a74-afde-ee00374beb5b-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/79a9c2b2-8458-4a74-afde-ee00374beb5b-workload-socket\") pod \"79a9c2b2-8458-4a74-afde-ee00374beb5b\" (UID: \"79a9c2b2-8458-4a74-afde-ee00374beb5b\") "
Oct 31 17:42:58.386468 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:58.382078    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/79a9c2b2-8458-4a74-afde-ee00374beb5b-kube-api-access-5572l\" (UniqueName: \"kubernetes.io/projected/79a9c2b2-8458-4a74-afde-ee00374beb5b-kube-api-access-5572l\") pod \"79a9c2b2-8458-4a74-afde-ee00374beb5b\" (UID: \"79a9c2b2-8458-4a74-afde-ee00374beb5b\") "
Oct 31 17:42:58.386468 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:58.382128    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/79a9c2b2-8458-4a74-afde-ee00374beb5b-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/79a9c2b2-8458-4a74-afde-ee00374beb5b-credential-socket\") pod \"79a9c2b2-8458-4a74-afde-ee00374beb5b\" (UID: \"79a9c2b2-8458-4a74-afde-ee00374beb5b\") "
Oct 31 17:42:58.386468 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:58.382154    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/79a9c2b2-8458-4a74-afde-ee00374beb5b-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/79a9c2b2-8458-4a74-afde-ee00374beb5b-istio-podinfo\") pod \"79a9c2b2-8458-4a74-afde-ee00374beb5b\" (UID: \"79a9c2b2-8458-4a74-afde-ee00374beb5b\") "
Oct 31 17:42:58.386468 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:58.382197    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/79a9c2b2-8458-4a74-afde-ee00374beb5b-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/79a9c2b2-8458-4a74-afde-ee00374beb5b-workload-certs\") pod \"79a9c2b2-8458-4a74-afde-ee00374beb5b\" (UID: \"79a9c2b2-8458-4a74-afde-ee00374beb5b\") "
Oct 31 17:42:58.386468 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:58.382250    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/79a9c2b2-8458-4a74-afde-ee00374beb5b-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/79a9c2b2-8458-4a74-afde-ee00374beb5b-istio-data\") pod \"79a9c2b2-8458-4a74-afde-ee00374beb5b\" (UID: \"79a9c2b2-8458-4a74-afde-ee00374beb5b\") "
Oct 31 17:42:58.424533 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:58.408758    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/79a9c2b2-8458-4a74-afde-ee00374beb5b-workload-certs" pod "79a9c2b2-8458-4a74-afde-ee00374beb5b" (UID: "79a9c2b2-8458-4a74-afde-ee00374beb5b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:42:58.424533 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:58.418103    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/79a9c2b2-8458-4a74-afde-ee00374beb5b-credential-socket" pod "79a9c2b2-8458-4a74-afde-ee00374beb5b" (UID: "79a9c2b2-8458-4a74-afde-ee00374beb5b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:42:58.424533 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:58.418838    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/79a9c2b2-8458-4a74-afde-ee00374beb5b-istiod-ca-cert" pod "79a9c2b2-8458-4a74-afde-ee00374beb5b" (UID: "79a9c2b2-8458-4a74-afde-ee00374beb5b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 17:42:58.424533 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:58.418898    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/79a9c2b2-8458-4a74-afde-ee00374beb5b-istio-data" pod "79a9c2b2-8458-4a74-afde-ee00374beb5b" (UID: "79a9c2b2-8458-4a74-afde-ee00374beb5b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:42:58.424533 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:58.422578    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/79a9c2b2-8458-4a74-afde-ee00374beb5b-workload-socket" pod "79a9c2b2-8458-4a74-afde-ee00374beb5b" (UID: "79a9c2b2-8458-4a74-afde-ee00374beb5b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:42:58.446511 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:58.441782    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/79a9c2b2-8458-4a74-afde-ee00374beb5b-istio-podinfo" pod "79a9c2b2-8458-4a74-afde-ee00374beb5b" (UID: "79a9c2b2-8458-4a74-afde-ee00374beb5b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 17:42:58.447622 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:58.447590    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/79a9c2b2-8458-4a74-afde-ee00374beb5b-istio-envoy" pod "79a9c2b2-8458-4a74-afde-ee00374beb5b" (UID: "79a9c2b2-8458-4a74-afde-ee00374beb5b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:42:58.448358 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:58.448318    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/79a9c2b2-8458-4a74-afde-ee00374beb5b-istio-token" pod "79a9c2b2-8458-4a74-afde-ee00374beb5b" (UID: "79a9c2b2-8458-4a74-afde-ee00374beb5b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 17:42:58.449830 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:58.449805    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/79a9c2b2-8458-4a74-afde-ee00374beb5b-kube-api-access-5572l" pod "79a9c2b2-8458-4a74-afde-ee00374beb5b" (UID: "79a9c2b2-8458-4a74-afde-ee00374beb5b"). InnerVolumeSpecName "kube-api-access-5572l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 17:42:58.455382 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:58.455307    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:42:58.485512 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:58.485476    4276 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/79a9c2b2-8458-4a74-afde-ee00374beb5b-credential-socket\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:42:58.485512 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:58.485513    4276 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/79a9c2b2-8458-4a74-afde-ee00374beb5b-istio-podinfo\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:42:58.485780 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:58.485532    4276 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/79a9c2b2-8458-4a74-afde-ee00374beb5b-workload-certs\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:42:58.485780 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:58.485546    4276 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/79a9c2b2-8458-4a74-afde-ee00374beb5b-istio-data\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:42:58.485780 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:58.485559    4276 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/79a9c2b2-8458-4a74-afde-ee00374beb5b-istio-envoy\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:42:58.485780 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:58.485572    4276 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/79a9c2b2-8458-4a74-afde-ee00374beb5b-istio-token\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:42:58.485780 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:58.485584    4276 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/79a9c2b2-8458-4a74-afde-ee00374beb5b-istiod-ca-cert\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:42:58.485780 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:58.485597    4276 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/79a9c2b2-8458-4a74-afde-ee00374beb5b-workload-socket\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:42:58.485780 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:58.485611    4276 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5572l\" (UniqueName: \"kubernetes.io/projected/79a9c2b2-8458-4a74-afde-ee00374beb5b-kube-api-access-5572l\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:42:58.709521 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:58.708804    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:42:58.801548 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:58.793039    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-9lts8" event={"ID":"79a9c2b2-8458-4a74-afde-ee00374beb5b","Type":"ContainerDied","Data":"2025f3ff521fd539691fccf5d10485bb5ef879f003b99cf51f6a111b6505f81e"}
Oct 31 17:42:58.801548 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:58.793111    4276 scope.go:122] "RemoveContainer" containerID="086d73b8f16757218462e0350e934c0a704569110746210e0031445ea87e9d90"
Oct 31 17:42:58.801548 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:58.793261    4276 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-9lts8"
Oct 31 17:42:58.860371 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:58.860239    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:42:58.995346 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:58.995268    4276 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-9lts8"]
Oct 31 17:42:59.008554 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:59.008483    4276 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-0e59ca-openshift-default-7cd8df4957-9lts8"]
Oct 31 17:42:59.062289 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:59.060720    4276 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-d80080-869f895d55-s62hv"]
Oct 31 17:42:59.062289 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:59.060966    4276 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-d80080-869f895d55-s62hv" podUID="175639e5-138e-428e-bb53-7fe484543496" containerName="maas-api" containerID="cri-o://9f14cf231e12847b2388f04f42d866775667f02335e3fc4c7e81924a5b3a16d9" gracePeriod=30
Oct 31 17:42:59.177076 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:59.176937    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:42:59.332481 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:59.326873    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:42:59.487405 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:59.487372    4276 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6418b389-728b-4040-9b73-50cf013c00f3" path="/var/lib/kubelet/pods/6418b389-728b-4040-9b73-50cf013c00f3/volumes"
Oct 31 17:42:59.488268 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:59.488249    4276 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="79a9c2b2-8458-4a74-afde-ee00374beb5b" path="/var/lib/kubelet/pods/79a9c2b2-8458-4a74-afde-ee00374beb5b/volumes"
Oct 31 17:42:59.490534 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:59.490515    4276 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d46713a5-7bba-4dfb-b77f-8fae0b39271e" path="/var/lib/kubelet/pods/d46713a5-7bba-4dfb-b77f-8fae0b39271e/volumes"
Oct 31 17:42:59.491146 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:59.491129    4276 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f7894ac3-7f53-4a3f-8d20-17359e24f69e" path="/var/lib/kubelet/pods/f7894ac3-7f53-4a3f-8d20-17359e24f69e/volumes"
Oct 31 17:42:59.491532 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:59.488763    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:42:59.607262 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:59.607095    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:42:59.834462 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:59.834030    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-d80080-59b65f4495-h6gz8" podUID="6e4109e9-1a59-48c7-9143-e14a01758994" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 31 17:42:59.843462 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:59.841493    4276 generic.go:356] "Generic (PLEG): container finished" podID="95755676-9f02-4b03-a7e5-f6c8ab4fa965" containerID="b1508c7bbf6311fa84ae2faa759301a8f5a726bee2c9b402767cd3fd5ef15bf1" exitCode=0
Oct 31 17:42:59.843462 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:59.841633    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-1c0bfd-6569c7d6f-l29ff" event={"ID":"95755676-9f02-4b03-a7e5-f6c8ab4fa965","Type":"ContainerDied","Data":"b1508c7bbf6311fa84ae2faa759301a8f5a726bee2c9b402767cd3fd5ef15bf1"}
Oct 31 17:42:59.845712 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:59.845534    4276 generic.go:356] "Generic (PLEG): container finished" podID="175639e5-138e-428e-bb53-7fe484543496" containerID="9f14cf231e12847b2388f04f42d866775667f02335e3fc4c7e81924a5b3a16d9" exitCode=0
Oct 31 17:42:59.845712 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:59.845639    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-d80080-869f895d55-s62hv" event={"ID":"175639e5-138e-428e-bb53-7fe484543496","Type":"ContainerDied","Data":"9f14cf231e12847b2388f04f42d866775667f02335e3fc4c7e81924a5b3a16d9"}
Oct 31 17:42:59.892903 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:59.892625    4276 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-d80080-869f895d55-s62hv"
Oct 31 17:42:59.970816 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:59.970787    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/175639e5-138e-428e-bb53-7fe484543496-kube-api-access-dw6tr\" (UniqueName: \"kubernetes.io/projected/175639e5-138e-428e-bb53-7fe484543496-kube-api-access-dw6tr\") pod \"175639e5-138e-428e-bb53-7fe484543496\" (UID: \"175639e5-138e-428e-bb53-7fe484543496\") "
Oct 31 17:42:59.970973 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:59.970889    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/175639e5-138e-428e-bb53-7fe484543496-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/175639e5-138e-428e-bb53-7fe484543496-maas-api-tls\") pod \"175639e5-138e-428e-bb53-7fe484543496\" (UID: \"175639e5-138e-428e-bb53-7fe484543496\") "
Oct 31 17:42:59.971071 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:59.971058    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/175639e5-138e-428e-bb53-7fe484543496-metrics-certs\" (UniqueName: \"kubernetes.io/secret/175639e5-138e-428e-bb53-7fe484543496-metrics-certs\") pod \"175639e5-138e-428e-bb53-7fe484543496\" (UID: \"175639e5-138e-428e-bb53-7fe484543496\") "
Oct 31 17:42:59.971147 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:59.971130    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/175639e5-138e-428e-bb53-7fe484543496-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/175639e5-138e-428e-bb53-7fe484543496-openshift-service-ca\") pod \"175639e5-138e-428e-bb53-7fe484543496\" (UID: \"175639e5-138e-428e-bb53-7fe484543496\") "
Oct 31 17:42:59.971568 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:59.971532    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/175639e5-138e-428e-bb53-7fe484543496-openshift-service-ca" pod "175639e5-138e-428e-bb53-7fe484543496" (UID: "175639e5-138e-428e-bb53-7fe484543496"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 17:42:59.972952 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:59.972925    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/175639e5-138e-428e-bb53-7fe484543496-metrics-certs" pod "175639e5-138e-428e-bb53-7fe484543496" (UID: "175639e5-138e-428e-bb53-7fe484543496"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 17:42:59.972952 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:59.972934    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/175639e5-138e-428e-bb53-7fe484543496-kube-api-access-dw6tr" pod "175639e5-138e-428e-bb53-7fe484543496" (UID: "175639e5-138e-428e-bb53-7fe484543496"). InnerVolumeSpecName "kube-api-access-dw6tr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 17:42:59.973097 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:59.972981    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/175639e5-138e-428e-bb53-7fe484543496-maas-api-tls" pod "175639e5-138e-428e-bb53-7fe484543496" (UID: "175639e5-138e-428e-bb53-7fe484543496"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 17:42:59.991621 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:42:59.991601    4276 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-1c0bfd-6569c7d6f-l29ff"
Oct 31 17:43:00.016686 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:00.016662    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:43:00.072853 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:00.072822    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/95755676-9f02-4b03-a7e5-f6c8ab4fa965-metrics-certs\" (UniqueName: \"kubernetes.io/secret/95755676-9f02-4b03-a7e5-f6c8ab4fa965-metrics-certs\") pod \"95755676-9f02-4b03-a7e5-f6c8ab4fa965\" (UID: \"95755676-9f02-4b03-a7e5-f6c8ab4fa965\") "
Oct 31 17:43:00.072967 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:00.072901    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/95755676-9f02-4b03-a7e5-f6c8ab4fa965-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/95755676-9f02-4b03-a7e5-f6c8ab4fa965-maas-api-tls\") pod \"95755676-9f02-4b03-a7e5-f6c8ab4fa965\" (UID: \"95755676-9f02-4b03-a7e5-f6c8ab4fa965\") "
Oct 31 17:43:00.073017 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:00.072970    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/95755676-9f02-4b03-a7e5-f6c8ab4fa965-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/95755676-9f02-4b03-a7e5-f6c8ab4fa965-openshift-service-ca\") pod \"95755676-9f02-4b03-a7e5-f6c8ab4fa965\" (UID: \"95755676-9f02-4b03-a7e5-f6c8ab4fa965\") "
Oct 31 17:43:00.073143 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:00.073126    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/95755676-9f02-4b03-a7e5-f6c8ab4fa965-kube-api-access-vsfjr\" (UniqueName: \"kubernetes.io/projected/95755676-9f02-4b03-a7e5-f6c8ab4fa965-kube-api-access-vsfjr\") pod \"95755676-9f02-4b03-a7e5-f6c8ab4fa965\" (UID: \"95755676-9f02-4b03-a7e5-f6c8ab4fa965\") "
Oct 31 17:43:00.073388 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:00.073357    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/95755676-9f02-4b03-a7e5-f6c8ab4fa965-openshift-service-ca" pod "95755676-9f02-4b03-a7e5-f6c8ab4fa965" (UID: "95755676-9f02-4b03-a7e5-f6c8ab4fa965"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 17:43:00.073747 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:00.073723    4276 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/175639e5-138e-428e-bb53-7fe484543496-metrics-certs\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:43:00.073859 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:00.073753    4276 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/175639e5-138e-428e-bb53-7fe484543496-openshift-service-ca\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:43:00.073859 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:00.073770    4276 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dw6tr\" (UniqueName: \"kubernetes.io/projected/175639e5-138e-428e-bb53-7fe484543496-kube-api-access-dw6tr\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:43:00.073859 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:00.073786    4276 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/175639e5-138e-428e-bb53-7fe484543496-maas-api-tls\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:43:00.073859 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:00.073799    4276 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/95755676-9f02-4b03-a7e5-f6c8ab4fa965-openshift-service-ca\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:43:00.075094 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:00.075068    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/95755676-9f02-4b03-a7e5-f6c8ab4fa965-maas-api-tls" pod "95755676-9f02-4b03-a7e5-f6c8ab4fa965" (UID: "95755676-9f02-4b03-a7e5-f6c8ab4fa965"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 17:43:00.075189 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:00.075112    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/95755676-9f02-4b03-a7e5-f6c8ab4fa965-metrics-certs" pod "95755676-9f02-4b03-a7e5-f6c8ab4fa965" (UID: "95755676-9f02-4b03-a7e5-f6c8ab4fa965"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 17:43:00.075189 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:00.075131    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/95755676-9f02-4b03-a7e5-f6c8ab4fa965-kube-api-access-vsfjr" pod "95755676-9f02-4b03-a7e5-f6c8ab4fa965" (UID: "95755676-9f02-4b03-a7e5-f6c8ab4fa965"). InnerVolumeSpecName "kube-api-access-vsfjr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 17:43:00.137372 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:00.137306    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:43:00.175006 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:00.174981    4276 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/95755676-9f02-4b03-a7e5-f6c8ab4fa965-metrics-certs\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:43:00.175006 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:00.175003    4276 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/95755676-9f02-4b03-a7e5-f6c8ab4fa965-maas-api-tls\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:43:00.175167 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:00.175012    4276 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vsfjr\" (UniqueName: \"kubernetes.io/projected/95755676-9f02-4b03-a7e5-f6c8ab4fa965-kube-api-access-vsfjr\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:43:00.258235 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:00.258196    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:43:00.803557 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:00.803528    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:43:00.862597 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:00.862568    4276 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-1c0bfd-6569c7d6f-l29ff"
Oct 31 17:43:00.862597 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:00.862586    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-1c0bfd-6569c7d6f-l29ff" event={"ID":"95755676-9f02-4b03-a7e5-f6c8ab4fa965","Type":"ContainerDied","Data":"073e0d34c64ac886838643937844ea2d1afcb25d2319af383326e4e0c00b22f5"}
Oct 31 17:43:00.862824 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:00.862719    4276 scope.go:122] "RemoveContainer" containerID="b1508c7bbf6311fa84ae2faa759301a8f5a726bee2c9b402767cd3fd5ef15bf1"
Oct 31 17:43:00.864171 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:00.864114    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-d80080-869f895d55-s62hv" event={"ID":"175639e5-138e-428e-bb53-7fe484543496","Type":"ContainerDied","Data":"6c2a417b1d597e443b6fd947fbcb2525e4a016d06869c5062e2b501da39dbe7c"}
Oct 31 17:43:00.864171 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:00.864160    4276 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-d80080-869f895d55-s62hv"
Oct 31 17:43:00.872551 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:00.872531    4276 scope.go:122] "RemoveContainer" containerID="9f14cf231e12847b2388f04f42d866775667f02335e3fc4c7e81924a5b3a16d9"
Oct 31 17:43:00.921225 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:00.921185    4276 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-d80080-869f895d55-s62hv"]
Oct 31 17:43:00.934995 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:00.934971    4276 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-d80080-869f895d55-s62hv"]
Oct 31 17:43:00.960479 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:00.960457    4276 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-1c0bfd-6569c7d6f-l29ff"]
Oct 31 17:43:00.962756 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:00.962739    4276 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-1c0bfd-6569c7d6f-l29ff"]
Oct 31 17:43:01.208316 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:01.208214    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:43:01.319370 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:01.319336    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:43:01.367251 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:01.367212    4276 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="175639e5-138e-428e-bb53-7fe484543496" path="/var/lib/kubelet/pods/175639e5-138e-428e-bb53-7fe484543496/volumes"
Oct 31 17:43:01.367812 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:01.367789    4276 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="95755676-9f02-4b03-a7e5-f6c8ab4fa965" path="/var/lib/kubelet/pods/95755676-9f02-4b03-a7e5-f6c8ab4fa965/volumes"
Oct 31 17:43:02.188957 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:02.188922    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:43:02.666388 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:02.666341    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-d80080-869f895d55-s62hv" podUID="175639e5-138e-428e-bb53-7fe484543496" containerName="maas-api" probeResult="failure" output="Get \"https://10.133.0.41:8443/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 17:43:02.879621 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:02.879591    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:43:03.005987 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:03.005906    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:43:03.617169 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:03.617139    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:43:04.088418 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:04.088385    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:43:05.083103 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:05.078699    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:43:05.194722 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:05.194692    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:43:06.067624 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:06.067584    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:43:06.195838 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:06.195803    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:43:06.337691 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:06.337599    4276 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-zsl5x"]
Oct 31 17:43:06.337946 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:06.337914    4276 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-zsl5x" podUID="b2b7f15e-756a-40e6-90af-961551719a7d" containerName="istio-proxy" containerID="cri-o://d8d93d1ddbaec4597147921eac145f6223bd6f3d5d1e15e0f4f400c39bf468f1" gracePeriod=30
Oct 31 17:43:06.350940 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:06.350899    4276 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-94vcs"]
Oct 31 17:43:06.351118 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:06.351093    4276 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-94vcs" podUID="4c3cb076-d13d-4212-8fbb-86c851b14fe7" containerName="istio-proxy" containerID="cri-o://f55bc0bfbf838cd79b47049ad0a717173d0a9bf489fa6d669724fff4c02d1776" gracePeriod=30
Oct 31 17:43:06.353741 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:06.353719    4276 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-2v74j"]
Oct 31 17:43:06.353947 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:06.353917    4276 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-2v74j" podUID="e948a32e-25a4-4d29-b9dd-01a0539d337c" containerName="istio-proxy" containerID="cri-o://06a63ac7ce2591fda4bf36eab5086dc1977b339ae31f3a5f73921a0d03fe94c2" gracePeriod=30
Oct 31 17:43:06.356176 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:06.356121    4276 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-dx9cd"]
Oct 31 17:43:06.356324 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:06.356297    4276 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-dx9cd" podUID="3716260a-fa38-4064-991c-48322b1b48fa" containerName="istio-proxy" containerID="cri-o://122ad55efb2bd53bf8bad3bce430e571a33aa4bfe0469d290e9c49dc7e3b99a5" gracePeriod=30
Oct 31 17:43:06.414228 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:06.414201    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:43:07.968124 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:07.968087    4276 generic.go:356] "Generic (PLEG): container finished" podID="6e4109e9-1a59-48c7-9143-e14a01758994" containerID="4a0dc01c7432bd39e6e6f75d3eac48c4c6452840954242f79abbd2bfb8d4600e" exitCode=0
Oct 31 17:43:07.969227 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:07.968688    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-d80080-59b65f4495-h6gz8" event={"ID":"6e4109e9-1a59-48c7-9143-e14a01758994","Type":"ContainerDied","Data":"4a0dc01c7432bd39e6e6f75d3eac48c4c6452840954242f79abbd2bfb8d4600e"}
Oct 31 17:43:08.226323 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:08.211939    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:43:08.328155 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:08.319148    4276 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-d80080-59b65f4495-h6gz8"
Oct 31 17:43:08.338463 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:08.334867    4276 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-8bf7k"]
Oct 31 17:43:08.338463 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:08.335127    4276 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-8bf7k" podUID="28cbdb7c-1bc2-49b4-834c-de871be33524" containerName="istio-proxy" containerID="cri-o://1b62347ed05ecd31efa2e9cb7a621deeb263f006afe6411c319be80c5cd86a75" gracePeriod=30
Oct 31 17:43:08.348921 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:08.348879    4276 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-m7vsj"]
Oct 31 17:43:08.349722 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:08.349677    4276 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-m7vsj" podUID="f675ff71-27f9-44e8-aa32-89f4332d4c3e" containerName="istio-proxy" containerID="cri-o://1019d46050d0bb7e6f4da0f693b5f99640f2050d06769321e7a4b15b04524aa5" gracePeriod=30
Oct 31 17:43:08.356745 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:08.356722    4276 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-ghwks"]
Oct 31 17:43:08.357047 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:08.357026    4276 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-ghwks" podUID="33671198-f448-4f99-9bc9-6a1ed6980c5b" containerName="istio-proxy" containerID="cri-o://09b8ee7e51fe351cc0bcc6f9d3dbcf11853e736da058d8d30cb89388f02fbd0c" gracePeriod=30
Oct 31 17:43:08.368388 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:08.368101    4276 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-qlhn8"]
Oct 31 17:43:08.368388 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:08.368255    4276 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-qlhn8" podUID="c3c40ebe-5cac-4981-913d-18f12d0bf439" containerName="istio-proxy" containerID="cri-o://395370f5dee6ae088f35afe2eeb8f02c629d745ac4ddf5cacb9129b95407042a" gracePeriod=30
Oct 31 17:43:08.401641 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:08.401597    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:43:08.407101 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:08.407072    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6e4109e9-1a59-48c7-9143-e14a01758994-kube-api-access-jw7mz\" (UniqueName: \"kubernetes.io/projected/6e4109e9-1a59-48c7-9143-e14a01758994-kube-api-access-jw7mz\") pod \"6e4109e9-1a59-48c7-9143-e14a01758994\" (UID: \"6e4109e9-1a59-48c7-9143-e14a01758994\") "
Oct 31 17:43:08.407339 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:08.407321    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6e4109e9-1a59-48c7-9143-e14a01758994-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6e4109e9-1a59-48c7-9143-e14a01758994-plugins-config-volume\") pod \"6e4109e9-1a59-48c7-9143-e14a01758994\" (UID: \"6e4109e9-1a59-48c7-9143-e14a01758994\") "
Oct 31 17:43:08.407774 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:08.407749    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6e4109e9-1a59-48c7-9143-e14a01758994-plugins-config-volume" pod "6e4109e9-1a59-48c7-9143-e14a01758994" (UID: "6e4109e9-1a59-48c7-9143-e14a01758994"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 17:43:08.407932 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:08.407905    4276 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6e4109e9-1a59-48c7-9143-e14a01758994-plugins-config-volume\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:43:08.409563 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:08.409538    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6e4109e9-1a59-48c7-9143-e14a01758994-kube-api-access-jw7mz" pod "6e4109e9-1a59-48c7-9143-e14a01758994" (UID: "6e4109e9-1a59-48c7-9143-e14a01758994"). InnerVolumeSpecName "kube-api-access-jw7mz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 17:43:08.509806 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:08.509722    4276 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jw7mz\" (UniqueName: \"kubernetes.io/projected/6e4109e9-1a59-48c7-9143-e14a01758994-kube-api-access-jw7mz\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:43:08.553012 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:08.552981    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:43:08.800574 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:08.800541    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:43:08.964173 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:08.964144    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:43:08.975541 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:08.975507    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-d80080-59b65f4495-h6gz8" event={"ID":"6e4109e9-1a59-48c7-9143-e14a01758994","Type":"ContainerDied","Data":"eeb05c6ad1920207575b6847a78388cb26783dda8e423046763f70217f81ee85"}
Oct 31 17:43:08.975541 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:08.975529    4276 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-d80080-59b65f4495-h6gz8"
Oct 31 17:43:08.976012 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:08.975606    4276 scope.go:122] "RemoveContainer" containerID="4a0dc01c7432bd39e6e6f75d3eac48c4c6452840954242f79abbd2bfb8d4600e"
Oct 31 17:43:09.042264 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:09.042240    4276 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w6-d80080-59b65f4495-h6gz8"]
Oct 31 17:43:09.058007 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:09.057947    4276 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w6-d80080-59b65f4495-h6gz8"]
Oct 31 17:43:09.165712 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:09.165675    4276 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-kfkwb"]
Oct 31 17:43:09.165900 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:09.165874    4276 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-kfkwb" podUID="4b04b1b0-d2cc-4304-92e6-90e48105f442" containerName="istio-proxy" containerID="cri-o://a819b54f19dd2fb68538d2c781d08d3ab2856ca18c49b5f01808e5bc0c355a28" gracePeriod=30
Oct 31 17:43:09.181119 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:09.181085    4276 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-wm2s9"]
Oct 31 17:43:09.181333 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:09.181308    4276 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-wm2s9" podUID="34243752-91ce-4e92-b0a6-72490b1f7c9e" containerName="istio-proxy" containerID="cri-o://89e2d0c4c006a4779f619c4ee4c592e212bcd6cf3946b5efe9b95513bc818853" gracePeriod=30
Oct 31 17:43:09.212749 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:09.212718    4276 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-m99sg"]
Oct 31 17:43:09.212970 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:09.212946    4276 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-m99sg" podUID="56c4ff3e-b2f2-4dce-938d-48672451d1fe" containerName="istio-proxy" containerID="cri-o://4f62c3ff869ee3e1962a35435aba4b5f419648311c604b0634911e8968b4c93f" gracePeriod=30
Oct 31 17:43:09.239215 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:09.239174    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:43:09.367673 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:09.367591    4276 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6e4109e9-1a59-48c7-9143-e14a01758994" path="/var/lib/kubelet/pods/6e4109e9-1a59-48c7-9143-e14a01758994/volumes"
Oct 31 17:43:09.975519 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:09.975484    4276 patch_prober.go:29] interesting pod/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-qlhn8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 31 17:43:09.975700 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:09.975540    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-qlhn8" podUID="c3c40ebe-5cac-4981-913d-18f12d0bf439" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 17:43:10.270593 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:10.270513    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:43:10.318420 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:10.318385    4276 patch_prober.go:29] interesting pod/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-2v74j container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 31 17:43:10.318613 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:10.318467    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-2v74j" podUID="e948a32e-25a4-4d29-b9dd-01a0539d337c" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 17:43:10.358476 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:10.358433    4276 patch_prober.go:29] interesting pod/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-94vcs container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 31 17:43:10.358626 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:10.358514    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-94vcs" podUID="4c3cb076-d13d-4212-8fbb-86c851b14fe7" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 17:43:10.398023 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:10.397996    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:43:10.585280 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:10.585247    4276 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-g646p"]
Oct 31 17:43:10.585532 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:10.585454    4276 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-g646p" podUID="14900971-460e-4126-9089-c4b110d539ab" containerName="istio-proxy" containerID="cri-o://661fccd383b68b0099aeec974f6cc1dd943864f8e2e0df4870d8bcd831a23a1a" gracePeriod=30
Oct 31 17:43:10.600661 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:10.600628    4276 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-cfvtd"]
Oct 31 17:43:10.600868 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:10.600845    4276 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-cfvtd" podUID="bc5595c7-f428-4092-ac29-a30af2e01065" containerName="istio-proxy" containerID="cri-o://94875b822b89638cdb029172564865300c6eabb87124433d1493ab36ff0356d6" gracePeriod=30
Oct 31 17:43:10.602457 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:10.602414    4276 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-25kdb"]
Oct 31 17:43:10.602626 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:10.602605    4276 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-25kdb" podUID="6ec6accb-4b2a-45a8-bab2-fe65f2f5df2d" containerName="istio-proxy" containerID="cri-o://a1287b2dab22e7f754ce2563530796e068f02327c3f3a8993b891d470489e006" gracePeriod=30
Oct 31 17:43:10.615139 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:10.614177    4276 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-hgtdx"]
Oct 31 17:43:10.615139 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:10.614362    4276 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-hgtdx" podUID="a59e3d47-2d57-4bf7-9a6c-5584d2d58d63" containerName="istio-proxy" containerID="cri-o://c11a9fcac7492f02a86e3e250a3a1b0c7b66682168c63268a347b1c9db01093e" gracePeriod=30
Oct 31 17:43:10.633793 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:10.633761    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:43:11.051500 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:11.051464    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:43:11.642367 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:11.642339    4276 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-zsl5x"
Oct 31 17:43:11.697181 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:11.697152    4276 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-2v74j"
Oct 31 17:43:11.705955 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:11.705621    4276 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-94vcs"
Oct 31 17:43:11.729376 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:11.729351    4276 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-dx9cd"
Oct 31 17:43:11.768746 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:11.768713    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4c3cb076-d13d-4212-8fbb-86c851b14fe7-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c3cb076-d13d-4212-8fbb-86c851b14fe7-workload-socket\") pod \"4c3cb076-d13d-4212-8fbb-86c851b14fe7\" (UID: \"4c3cb076-d13d-4212-8fbb-86c851b14fe7\") "
Oct 31 17:43:11.768894 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:11.768796    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4c3cb076-d13d-4212-8fbb-86c851b14fe7-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4c3cb076-d13d-4212-8fbb-86c851b14fe7-istio-envoy\") pod \"4c3cb076-d13d-4212-8fbb-86c851b14fe7\" (UID: \"4c3cb076-d13d-4212-8fbb-86c851b14fe7\") "
Oct 31 17:43:11.768894 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:11.768835    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e948a32e-25a4-4d29-b9dd-01a0539d337c-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e948a32e-25a4-4d29-b9dd-01a0539d337c-istio-data\") pod \"e948a32e-25a4-4d29-b9dd-01a0539d337c\" (UID: \"e948a32e-25a4-4d29-b9dd-01a0539d337c\") "
Oct 31 17:43:11.768894 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:11.768862    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b2b7f15e-756a-40e6-90af-961551719a7d-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b2b7f15e-756a-40e6-90af-961551719a7d-workload-certs\") pod \"b2b7f15e-756a-40e6-90af-961551719a7d\" (UID: \"b2b7f15e-756a-40e6-90af-961551719a7d\") "
Oct 31 17:43:11.769112 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:11.768902    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e948a32e-25a4-4d29-b9dd-01a0539d337c-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e948a32e-25a4-4d29-b9dd-01a0539d337c-istio-envoy\") pod \"e948a32e-25a4-4d29-b9dd-01a0539d337c\" (UID: \"e948a32e-25a4-4d29-b9dd-01a0539d337c\") "
Oct 31 17:43:11.769112 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:11.768944    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4c3cb076-d13d-4212-8fbb-86c851b14fe7-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4c3cb076-d13d-4212-8fbb-86c851b14fe7-workload-certs\") pod \"4c3cb076-d13d-4212-8fbb-86c851b14fe7\" (UID: \"4c3cb076-d13d-4212-8fbb-86c851b14fe7\") "
Oct 31 17:43:11.769112 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:11.768979    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3716260a-fa38-4064-991c-48322b1b48fa-kube-api-access-dnl6c\" (UniqueName: \"kubernetes.io/projected/3716260a-fa38-4064-991c-48322b1b48fa-kube-api-access-dnl6c\") pod \"3716260a-fa38-4064-991c-48322b1b48fa\" (UID: \"3716260a-fa38-4064-991c-48322b1b48fa\") "
Oct 31 17:43:11.769112 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:11.769003    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c3cb076-d13d-4212-8fbb-86c851b14fe7-workload-socket" pod "4c3cb076-d13d-4212-8fbb-86c851b14fe7" (UID: "4c3cb076-d13d-4212-8fbb-86c851b14fe7"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:43:11.769112 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:11.769010    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/3716260a-fa38-4064-991c-48322b1b48fa-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3716260a-fa38-4064-991c-48322b1b48fa-istio-podinfo\") pod \"3716260a-fa38-4064-991c-48322b1b48fa\" (UID: \"3716260a-fa38-4064-991c-48322b1b48fa\") "
Oct 31 17:43:11.769112 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:11.769059    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e948a32e-25a4-4d29-b9dd-01a0539d337c-istio-data" pod "e948a32e-25a4-4d29-b9dd-01a0539d337c" (UID: "e948a32e-25a4-4d29-b9dd-01a0539d337c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:43:11.769112 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:11.769091    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3716260a-fa38-4064-991c-48322b1b48fa-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3716260a-fa38-4064-991c-48322b1b48fa-workload-socket\") pod \"3716260a-fa38-4064-991c-48322b1b48fa\" (UID: \"3716260a-fa38-4064-991c-48322b1b48fa\") "
Oct 31 17:43:11.769512 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:11.769195    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b2b7f15e-756a-40e6-90af-961551719a7d-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2b7f15e-756a-40e6-90af-961551719a7d-workload-socket\") pod \"b2b7f15e-756a-40e6-90af-961551719a7d\" (UID: \"b2b7f15e-756a-40e6-90af-961551719a7d\") "
Oct 31 17:43:11.769512 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:11.769226    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3716260a-fa38-4064-991c-48322b1b48fa-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3716260a-fa38-4064-991c-48322b1b48fa-workload-certs\") pod \"3716260a-fa38-4064-991c-48322b1b48fa\" (UID: \"3716260a-fa38-4064-991c-48322b1b48fa\") "
Oct 31 17:43:11.769512 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:11.769253    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3716260a-fa38-4064-991c-48322b1b48fa-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3716260a-fa38-4064-991c-48322b1b48fa-istiod-ca-cert\") pod \"3716260a-fa38-4064-991c-48322b1b48fa\" (UID: \"3716260a-fa38-4064-991c-48322b1b48fa\") "
Oct 31 17:43:11.769512 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:11.769277    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c3cb076-d13d-4212-8fbb-86c851b14fe7-workload-certs" pod "4c3cb076-d13d-4212-8fbb-86c851b14fe7" (UID: "4c3cb076-d13d-4212-8fbb-86c851b14fe7"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:43:11.769512 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:11.769304    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4c3cb076-d13d-4212-8fbb-86c851b14fe7-istio-token\" (UniqueName: \"kubernetes.io/projected/4c3cb076-d13d-4212-8fbb-86c851b14fe7-istio-token\") pod \"4c3cb076-d13d-4212-8fbb-86c851b14fe7\" (UID: \"4c3cb076-d13d-4212-8fbb-86c851b14fe7\") "
Oct 31 17:43:11.769512 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:11.769362    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b2b7f15e-756a-40e6-90af-961551719a7d-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2b7f15e-756a-40e6-90af-961551719a7d-credential-socket\") pod \"b2b7f15e-756a-40e6-90af-961551719a7d\" (UID: \"b2b7f15e-756a-40e6-90af-961551719a7d\") "
Oct 31 17:43:11.769512 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:11.769415    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/e948a32e-25a4-4d29-b9dd-01a0539d337c-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e948a32e-25a4-4d29-b9dd-01a0539d337c-istio-podinfo\") pod \"e948a32e-25a4-4d29-b9dd-01a0539d337c\" (UID: \"e948a32e-25a4-4d29-b9dd-01a0539d337c\") "
Oct 31 17:43:11.769512 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:11.769497    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3716260a-fa38-4064-991c-48322b1b48fa-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3716260a-fa38-4064-991c-48322b1b48fa-istio-data\") pod \"3716260a-fa38-4064-991c-48322b1b48fa\" (UID: \"3716260a-fa38-4064-991c-48322b1b48fa\") "
Oct 31 17:43:11.769512 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:11.769506    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b2b7f15e-756a-40e6-90af-961551719a7d-workload-certs" pod "b2b7f15e-756a-40e6-90af-961551719a7d" (UID: "b2b7f15e-756a-40e6-90af-961551719a7d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:43:11.769997 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:11.769550    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4c3cb076-d13d-4212-8fbb-86c851b14fe7-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c3cb076-d13d-4212-8fbb-86c851b14fe7-credential-socket\") pod \"4c3cb076-d13d-4212-8fbb-86c851b14fe7\" (UID: \"4c3cb076-d13d-4212-8fbb-86c851b14fe7\") "
Oct 31 17:43:11.769997 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:11.769592    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3716260a-fa38-4064-991c-48322b1b48fa-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3716260a-fa38-4064-991c-48322b1b48fa-istio-envoy\") pod \"3716260a-fa38-4064-991c-48322b1b48fa\" (UID: \"3716260a-fa38-4064-991c-48322b1b48fa\") "
Oct 31 17:43:11.769997 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:11.769632    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b2b7f15e-756a-40e6-90af-961551719a7d-istio-token\" (UniqueName: \"kubernetes.io/projected/b2b7f15e-756a-40e6-90af-961551719a7d-istio-token\") pod \"b2b7f15e-756a-40e6-90af-961551719a7d\" (UID: \"b2b7f15e-756a-40e6-90af-961551719a7d\") "
Oct 31 17:43:11.769997 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:11.769666    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4c3cb076-d13d-4212-8fbb-86c851b14fe7-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4c3cb076-d13d-4212-8fbb-86c851b14fe7-istiod-ca-cert\") pod \"4c3cb076-d13d-4212-8fbb-86c851b14fe7\" (UID: \"4c3cb076-d13d-4212-8fbb-86c851b14fe7\") "
Oct 31 17:43:11.769997 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:11.769702    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e948a32e-25a4-4d29-b9dd-01a0539d337c-istio-token\" (UniqueName: \"kubernetes.io/projected/e948a32e-25a4-4d29-b9dd-01a0539d337c-istio-token\") pod \"e948a32e-25a4-4d29-b9dd-01a0539d337c\" (UID: \"e948a32e-25a4-4d29-b9dd-01a0539d337c\") "
Oct 31 17:43:11.769997 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:11.769753    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b2b7f15e-756a-40e6-90af-961551719a7d-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b2b7f15e-756a-40e6-90af-961551719a7d-istio-data\") pod \"b2b7f15e-756a-40e6-90af-961551719a7d\" (UID: \"b2b7f15e-756a-40e6-90af-961551719a7d\") "
Oct 31 17:43:11.769997 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:11.769769    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b2b7f15e-756a-40e6-90af-961551719a7d-credential-socket" pod "b2b7f15e-756a-40e6-90af-961551719a7d" (UID: "b2b7f15e-756a-40e6-90af-961551719a7d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:43:11.769997 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:11.769782    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e948a32e-25a4-4d29-b9dd-01a0539d337c-kube-api-access-c2x4b\" (UniqueName: \"kubernetes.io/projected/e948a32e-25a4-4d29-b9dd-01a0539d337c-kube-api-access-c2x4b\") pod \"e948a32e-25a4-4d29-b9dd-01a0539d337c\" (UID: \"e948a32e-25a4-4d29-b9dd-01a0539d337c\") "
Oct 31 17:43:11.769997 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:11.769822    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e948a32e-25a4-4d29-b9dd-01a0539d337c-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e948a32e-25a4-4d29-b9dd-01a0539d337c-credential-socket\") pod \"e948a32e-25a4-4d29-b9dd-01a0539d337c\" (UID: \"e948a32e-25a4-4d29-b9dd-01a0539d337c\") "
Oct 31 17:43:11.769997 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:11.769835    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3716260a-fa38-4064-991c-48322b1b48fa-istiod-ca-cert" pod "3716260a-fa38-4064-991c-48322b1b48fa" (UID: "3716260a-fa38-4064-991c-48322b1b48fa"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 17:43:11.769997 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:11.769864    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e948a32e-25a4-4d29-b9dd-01a0539d337c-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e948a32e-25a4-4d29-b9dd-01a0539d337c-istiod-ca-cert\") pod \"e948a32e-25a4-4d29-b9dd-01a0539d337c\" (UID: \"e948a32e-25a4-4d29-b9dd-01a0539d337c\") "
Oct 31 17:43:11.769997 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:11.769952    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b2b7f15e-756a-40e6-90af-961551719a7d-kube-api-access-wxzcv\" (UniqueName: \"kubernetes.io/projected/b2b7f15e-756a-40e6-90af-961551719a7d-kube-api-access-wxzcv\") pod \"b2b7f15e-756a-40e6-90af-961551719a7d\" (UID: \"b2b7f15e-756a-40e6-90af-961551719a7d\") "
Oct 31 17:43:11.769997 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:11.769994    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4c3cb076-d13d-4212-8fbb-86c851b14fe7-kube-api-access-lnk4k\" (UniqueName: \"kubernetes.io/projected/4c3cb076-d13d-4212-8fbb-86c851b14fe7-kube-api-access-lnk4k\") pod \"4c3cb076-d13d-4212-8fbb-86c851b14fe7\" (UID: \"4c3cb076-d13d-4212-8fbb-86c851b14fe7\") "
Oct 31 17:43:11.770698 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:11.770036    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b2b7f15e-756a-40e6-90af-961551719a7d-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b2b7f15e-756a-40e6-90af-961551719a7d-istio-envoy\") pod \"b2b7f15e-756a-40e6-90af-961551719a7d\" (UID: \"b2b7f15e-756a-40e6-90af-961551719a7d\") "
Oct 31 17:43:11.770698 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:11.770070    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e948a32e-25a4-4d29-b9dd-01a0539d337c-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e948a32e-25a4-4d29-b9dd-01a0539d337c-workload-socket\") pod \"e948a32e-25a4-4d29-b9dd-01a0539d337c\" (UID: \"e948a32e-25a4-4d29-b9dd-01a0539d337c\") "
Oct 31 17:43:11.770698 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:11.770078    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3716260a-fa38-4064-991c-48322b1b48fa-workload-socket" pod "3716260a-fa38-4064-991c-48322b1b48fa" (UID: "3716260a-fa38-4064-991c-48322b1b48fa"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:43:11.770698 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:11.770105    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e948a32e-25a4-4d29-b9dd-01a0539d337c-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e948a32e-25a4-4d29-b9dd-01a0539d337c-workload-certs\") pod \"e948a32e-25a4-4d29-b9dd-01a0539d337c\" (UID: \"e948a32e-25a4-4d29-b9dd-01a0539d337c\") "
Oct 31 17:43:11.770698 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:11.770146    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/4c3cb076-d13d-4212-8fbb-86c851b14fe7-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4c3cb076-d13d-4212-8fbb-86c851b14fe7-istio-podinfo\") pod \"4c3cb076-d13d-4212-8fbb-86c851b14fe7\" (UID: \"4c3cb076-d13d-4212-8fbb-86c851b14fe7\") "
Oct 31 17:43:11.770698 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:11.770190    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/b2b7f15e-756a-40e6-90af-961551719a7d-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b2b7f15e-756a-40e6-90af-961551719a7d-istio-podinfo\") pod \"b2b7f15e-756a-40e6-90af-961551719a7d\" (UID: \"b2b7f15e-756a-40e6-90af-961551719a7d\") "
Oct 31 17:43:11.770698 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:11.770247    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b2b7f15e-756a-40e6-90af-961551719a7d-workload-socket" pod "b2b7f15e-756a-40e6-90af-961551719a7d" (UID: "b2b7f15e-756a-40e6-90af-961551719a7d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:43:11.770698 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:11.770256    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b2b7f15e-756a-40e6-90af-961551719a7d-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b2b7f15e-756a-40e6-90af-961551719a7d-istiod-ca-cert\") pod \"b2b7f15e-756a-40e6-90af-961551719a7d\" (UID: \"b2b7f15e-756a-40e6-90af-961551719a7d\") "
Oct 31 17:43:11.770698 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:11.770298    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4c3cb076-d13d-4212-8fbb-86c851b14fe7-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4c3cb076-d13d-4212-8fbb-86c851b14fe7-istio-data\") pod \"4c3cb076-d13d-4212-8fbb-86c851b14fe7\" (UID: \"4c3cb076-d13d-4212-8fbb-86c851b14fe7\") "
Oct 31 17:43:11.770698 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:11.770334    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3716260a-fa38-4064-991c-48322b1b48fa-istio-token\" (UniqueName: \"kubernetes.io/projected/3716260a-fa38-4064-991c-48322b1b48fa-istio-token\") pod \"3716260a-fa38-4064-991c-48322b1b48fa\" (UID: \"3716260a-fa38-4064-991c-48322b1b48fa\") "
Oct 31 17:43:11.770698 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:11.770379    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3716260a-fa38-4064-991c-48322b1b48fa-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3716260a-fa38-4064-991c-48322b1b48fa-credential-socket\") pod \"3716260a-fa38-4064-991c-48322b1b48fa\" (UID: \"3716260a-fa38-4064-991c-48322b1b48fa\") "
Oct 31 17:43:11.770698 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:11.770408    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3716260a-fa38-4064-991c-48322b1b48fa-workload-certs" pod "3716260a-fa38-4064-991c-48322b1b48fa" (UID: "3716260a-fa38-4064-991c-48322b1b48fa"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:43:11.771317 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:11.770753    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e948a32e-25a4-4d29-b9dd-01a0539d337c-istiod-ca-cert" pod "e948a32e-25a4-4d29-b9dd-01a0539d337c" (UID: "e948a32e-25a4-4d29-b9dd-01a0539d337c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 17:43:11.771317 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:11.770980    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3716260a-fa38-4064-991c-48322b1b48fa-credential-socket" pod "3716260a-fa38-4064-991c-48322b1b48fa" (UID: "3716260a-fa38-4064-991c-48322b1b48fa"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:43:11.771317 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:11.771177    4276 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2b7f15e-756a-40e6-90af-961551719a7d-workload-socket\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:43:11.771317 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:11.771198    4276 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3716260a-fa38-4064-991c-48322b1b48fa-workload-certs\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:43:11.771317 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:11.771213    4276 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3716260a-fa38-4064-991c-48322b1b48fa-istiod-ca-cert\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:43:11.771317 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:11.771226    4276 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2b7f15e-756a-40e6-90af-961551719a7d-credential-socket\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:43:11.771317 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:11.771240    4276 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e948a32e-25a4-4d29-b9dd-01a0539d337c-istiod-ca-cert\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:43:11.771317 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:11.771254    4276 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3716260a-fa38-4064-991c-48322b1b48fa-credential-socket\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:43:11.771317 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:11.771267    4276 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c3cb076-d13d-4212-8fbb-86c851b14fe7-workload-socket\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:43:11.771317 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:11.771280    4276 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e948a32e-25a4-4d29-b9dd-01a0539d337c-istio-data\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:43:11.771317 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:11.771294    4276 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b2b7f15e-756a-40e6-90af-961551719a7d-workload-certs\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:43:11.771317 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:11.771308    4276 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4c3cb076-d13d-4212-8fbb-86c851b14fe7-workload-certs\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:43:11.772724 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:11.771784    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b2b7f15e-756a-40e6-90af-961551719a7d-istio-data" pod "b2b7f15e-756a-40e6-90af-961551719a7d" (UID: "b2b7f15e-756a-40e6-90af-961551719a7d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:43:11.772724 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:11.771881    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c3cb076-d13d-4212-8fbb-86c851b14fe7-credential-socket" pod "4c3cb076-d13d-4212-8fbb-86c851b14fe7" (UID: "4c3cb076-d13d-4212-8fbb-86c851b14fe7"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:43:11.772724 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:11.772127    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e948a32e-25a4-4d29-b9dd-01a0539d337c-credential-socket" pod "e948a32e-25a4-4d29-b9dd-01a0539d337c" (UID: "e948a32e-25a4-4d29-b9dd-01a0539d337c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:43:11.772724 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:11.772173    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e948a32e-25a4-4d29-b9dd-01a0539d337c-workload-socket" pod "e948a32e-25a4-4d29-b9dd-01a0539d337c" (UID: "e948a32e-25a4-4d29-b9dd-01a0539d337c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:43:11.772724 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:11.772238    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c3cb076-d13d-4212-8fbb-86c851b14fe7-istio-envoy" pod "4c3cb076-d13d-4212-8fbb-86c851b14fe7" (UID: "4c3cb076-d13d-4212-8fbb-86c851b14fe7"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:43:11.773375 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:11.773049    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c3cb076-d13d-4212-8fbb-86c851b14fe7-istio-data" pod "4c3cb076-d13d-4212-8fbb-86c851b14fe7" (UID: "4c3cb076-d13d-4212-8fbb-86c851b14fe7"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:43:11.773375 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:11.773173    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4c3cb076-d13d-4212-8fbb-86c851b14fe7-istiod-ca-cert" pod "4c3cb076-d13d-4212-8fbb-86c851b14fe7" (UID: "4c3cb076-d13d-4212-8fbb-86c851b14fe7"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 17:43:11.773375 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:11.773223    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3716260a-fa38-4064-991c-48322b1b48fa-istio-data" pod "3716260a-fa38-4064-991c-48322b1b48fa" (UID: "3716260a-fa38-4064-991c-48322b1b48fa"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:43:11.773375 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:11.773324    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4c3cb076-d13d-4212-8fbb-86c851b14fe7-istio-token" pod "4c3cb076-d13d-4212-8fbb-86c851b14fe7" (UID: "4c3cb076-d13d-4212-8fbb-86c851b14fe7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 17:43:11.773375 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:11.773335    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b2b7f15e-756a-40e6-90af-961551719a7d-istiod-ca-cert" pod "b2b7f15e-756a-40e6-90af-961551719a7d" (UID: "b2b7f15e-756a-40e6-90af-961551719a7d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 17:43:11.773886 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:11.773665    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e948a32e-25a4-4d29-b9dd-01a0539d337c-workload-certs" pod "e948a32e-25a4-4d29-b9dd-01a0539d337c" (UID: "e948a32e-25a4-4d29-b9dd-01a0539d337c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:43:11.774903 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:11.774868    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b2b7f15e-756a-40e6-90af-961551719a7d-istio-podinfo" pod "b2b7f15e-756a-40e6-90af-961551719a7d" (UID: "b2b7f15e-756a-40e6-90af-961551719a7d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 17:43:11.775626 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:11.775603    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3716260a-fa38-4064-991c-48322b1b48fa-istio-podinfo" pod "3716260a-fa38-4064-991c-48322b1b48fa" (UID: "3716260a-fa38-4064-991c-48322b1b48fa"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 17:43:11.775733 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:11.775682    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3716260a-fa38-4064-991c-48322b1b48fa-istio-token" pod "3716260a-fa38-4064-991c-48322b1b48fa" (UID: "3716260a-fa38-4064-991c-48322b1b48fa"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 17:43:11.776145 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:11.776116    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b2b7f15e-756a-40e6-90af-961551719a7d-istio-token" pod "b2b7f15e-756a-40e6-90af-961551719a7d" (UID: "b2b7f15e-756a-40e6-90af-961551719a7d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 17:43:11.776244 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:11.776201    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e948a32e-25a4-4d29-b9dd-01a0539d337c-istio-podinfo" pod "e948a32e-25a4-4d29-b9dd-01a0539d337c" (UID: "e948a32e-25a4-4d29-b9dd-01a0539d337c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 17:43:11.776310 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:11.776287    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3716260a-fa38-4064-991c-48322b1b48fa-kube-api-access-dnl6c" pod "3716260a-fa38-4064-991c-48322b1b48fa" (UID: "3716260a-fa38-4064-991c-48322b1b48fa"). InnerVolumeSpecName "kube-api-access-dnl6c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 17:43:11.776611 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:11.776580    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4c3cb076-d13d-4212-8fbb-86c851b14fe7-istio-podinfo" pod "4c3cb076-d13d-4212-8fbb-86c851b14fe7" (UID: "4c3cb076-d13d-4212-8fbb-86c851b14fe7"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 17:43:11.777563 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:11.777513    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e948a32e-25a4-4d29-b9dd-01a0539d337c-istio-envoy" pod "e948a32e-25a4-4d29-b9dd-01a0539d337c" (UID: "e948a32e-25a4-4d29-b9dd-01a0539d337c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:43:11.777736 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:11.777713    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4c3cb076-d13d-4212-8fbb-86c851b14fe7-kube-api-access-lnk4k" pod "4c3cb076-d13d-4212-8fbb-86c851b14fe7" (UID: "4c3cb076-d13d-4212-8fbb-86c851b14fe7"). InnerVolumeSpecName "kube-api-access-lnk4k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 17:43:11.777736 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:11.777722    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3716260a-fa38-4064-991c-48322b1b48fa-istio-envoy" pod "3716260a-fa38-4064-991c-48322b1b48fa" (UID: "3716260a-fa38-4064-991c-48322b1b48fa"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:43:11.777871 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:11.777737    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e948a32e-25a4-4d29-b9dd-01a0539d337c-istio-token" pod "e948a32e-25a4-4d29-b9dd-01a0539d337c" (UID: "e948a32e-25a4-4d29-b9dd-01a0539d337c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 17:43:11.777871 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:11.777812    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b2b7f15e-756a-40e6-90af-961551719a7d-kube-api-access-wxzcv" pod "b2b7f15e-756a-40e6-90af-961551719a7d" (UID: "b2b7f15e-756a-40e6-90af-961551719a7d"). InnerVolumeSpecName "kube-api-access-wxzcv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 17:43:11.777871 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:11.777830    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b2b7f15e-756a-40e6-90af-961551719a7d-istio-envoy" pod "b2b7f15e-756a-40e6-90af-961551719a7d" (UID: "b2b7f15e-756a-40e6-90af-961551719a7d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:43:11.779415 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:11.779390    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e948a32e-25a4-4d29-b9dd-01a0539d337c-kube-api-access-c2x4b" pod "e948a32e-25a4-4d29-b9dd-01a0539d337c" (UID: "e948a32e-25a4-4d29-b9dd-01a0539d337c"). InnerVolumeSpecName "kube-api-access-c2x4b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 17:43:11.872317 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:11.872247    4276 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lnk4k\" (UniqueName: \"kubernetes.io/projected/4c3cb076-d13d-4212-8fbb-86c851b14fe7-kube-api-access-lnk4k\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:43:11.872317 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:11.872275    4276 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b2b7f15e-756a-40e6-90af-961551719a7d-istio-envoy\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:43:11.872317 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:11.872285    4276 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e948a32e-25a4-4d29-b9dd-01a0539d337c-workload-socket\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:43:11.872317 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:11.872294    4276 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e948a32e-25a4-4d29-b9dd-01a0539d337c-workload-certs\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:43:11.872317 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:11.872303    4276 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4c3cb076-d13d-4212-8fbb-86c851b14fe7-istio-podinfo\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:43:11.872317 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:11.872311    4276 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b2b7f15e-756a-40e6-90af-961551719a7d-istio-podinfo\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:43:11.872317 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:11.872319    4276 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b2b7f15e-756a-40e6-90af-961551719a7d-istiod-ca-cert\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:43:11.872664 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:11.872327    4276 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4c3cb076-d13d-4212-8fbb-86c851b14fe7-istio-data\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:43:11.872664 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:11.872334    4276 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3716260a-fa38-4064-991c-48322b1b48fa-istio-token\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:43:11.872664 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:11.872343    4276 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4c3cb076-d13d-4212-8fbb-86c851b14fe7-istio-envoy\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:43:11.872664 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:11.872350    4276 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e948a32e-25a4-4d29-b9dd-01a0539d337c-istio-envoy\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:43:11.872664 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:11.872357    4276 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dnl6c\" (UniqueName: \"kubernetes.io/projected/3716260a-fa38-4064-991c-48322b1b48fa-kube-api-access-dnl6c\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:43:11.872664 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:11.872365    4276 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3716260a-fa38-4064-991c-48322b1b48fa-istio-podinfo\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:43:11.872664 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:11.872373    4276 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3716260a-fa38-4064-991c-48322b1b48fa-workload-socket\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:43:11.872664 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:11.872381    4276 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4c3cb076-d13d-4212-8fbb-86c851b14fe7-istio-token\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:43:11.872664 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:11.872388    4276 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e948a32e-25a4-4d29-b9dd-01a0539d337c-istio-podinfo\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:43:11.872664 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:11.872395    4276 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3716260a-fa38-4064-991c-48322b1b48fa-istio-data\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:43:11.872664 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:11.872403    4276 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c3cb076-d13d-4212-8fbb-86c851b14fe7-credential-socket\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:43:11.872664 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:11.872411    4276 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3716260a-fa38-4064-991c-48322b1b48fa-istio-envoy\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:43:11.872664 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:11.872418    4276 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b2b7f15e-756a-40e6-90af-961551719a7d-istio-token\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:43:11.872664 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:11.872426    4276 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4c3cb076-d13d-4212-8fbb-86c851b14fe7-istiod-ca-cert\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:43:11.872664 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:11.872433    4276 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e948a32e-25a4-4d29-b9dd-01a0539d337c-istio-token\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:43:11.872664 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:11.872455    4276 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b2b7f15e-756a-40e6-90af-961551719a7d-istio-data\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:43:11.872664 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:11.872464    4276 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c2x4b\" (UniqueName: \"kubernetes.io/projected/e948a32e-25a4-4d29-b9dd-01a0539d337c-kube-api-access-c2x4b\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:43:11.872664 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:11.872471    4276 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e948a32e-25a4-4d29-b9dd-01a0539d337c-credential-socket\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:43:11.872664 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:11.872479    4276 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wxzcv\" (UniqueName: \"kubernetes.io/projected/b2b7f15e-756a-40e6-90af-961551719a7d-kube-api-access-wxzcv\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:43:12.005399 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:12.005366    4276 generic.go:356] "Generic (PLEG): container finished" podID="4c3cb076-d13d-4212-8fbb-86c851b14fe7" containerID="f55bc0bfbf838cd79b47049ad0a717173d0a9bf489fa6d669724fff4c02d1776" exitCode=0
Oct 31 17:43:12.005620 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:12.005413    4276 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-94vcs"
Oct 31 17:43:12.005620 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:12.005460    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-94vcs" event={"ID":"4c3cb076-d13d-4212-8fbb-86c851b14fe7","Type":"ContainerDied","Data":"f55bc0bfbf838cd79b47049ad0a717173d0a9bf489fa6d669724fff4c02d1776"}
Oct 31 17:43:12.005620 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:12.005517    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-94vcs" event={"ID":"4c3cb076-d13d-4212-8fbb-86c851b14fe7","Type":"ContainerDied","Data":"b4f6b85db1f316c047778c680a50778a4224607a471d957434b3ba5c1b189518"}
Oct 31 17:43:12.005620 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:12.005543    4276 scope.go:122] "RemoveContainer" containerID="f55bc0bfbf838cd79b47049ad0a717173d0a9bf489fa6d669724fff4c02d1776"
Oct 31 17:43:12.007088 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:12.007066    4276 generic.go:356] "Generic (PLEG): container finished" podID="3716260a-fa38-4064-991c-48322b1b48fa" containerID="122ad55efb2bd53bf8bad3bce430e571a33aa4bfe0469d290e9c49dc7e3b99a5" exitCode=0
Oct 31 17:43:12.007187 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:12.007102    4276 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-dx9cd"
Oct 31 17:43:12.007187 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:12.007134    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-dx9cd" event={"ID":"3716260a-fa38-4064-991c-48322b1b48fa","Type":"ContainerDied","Data":"122ad55efb2bd53bf8bad3bce430e571a33aa4bfe0469d290e9c49dc7e3b99a5"}
Oct 31 17:43:12.007279 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:12.007192    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-dx9cd" event={"ID":"3716260a-fa38-4064-991c-48322b1b48fa","Type":"ContainerDied","Data":"d1130cce08a06c9fc72c9c1126d0154246c0a939b75b11809b8c6f9132738629"}
Oct 31 17:43:12.008733 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:12.008713    4276 generic.go:356] "Generic (PLEG): container finished" podID="e948a32e-25a4-4d29-b9dd-01a0539d337c" containerID="06a63ac7ce2591fda4bf36eab5086dc1977b339ae31f3a5f73921a0d03fe94c2" exitCode=0
Oct 31 17:43:12.008835 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:12.008758    4276 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-2v74j"
Oct 31 17:43:12.008835 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:12.008793    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-2v74j" event={"ID":"e948a32e-25a4-4d29-b9dd-01a0539d337c","Type":"ContainerDied","Data":"06a63ac7ce2591fda4bf36eab5086dc1977b339ae31f3a5f73921a0d03fe94c2"}
Oct 31 17:43:12.008926 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:12.008842    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-2v74j" event={"ID":"e948a32e-25a4-4d29-b9dd-01a0539d337c","Type":"ContainerDied","Data":"593776d33d32ef9e459bdf8d3e69ddc5c8559b337359c156a6a4d00e748a32da"}
Oct 31 17:43:12.010388 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:12.010364    4276 generic.go:356] "Generic (PLEG): container finished" podID="b2b7f15e-756a-40e6-90af-961551719a7d" containerID="d8d93d1ddbaec4597147921eac145f6223bd6f3d5d1e15e0f4f400c39bf468f1" exitCode=0
Oct 31 17:43:12.010508 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:12.010397    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-zsl5x" event={"ID":"b2b7f15e-756a-40e6-90af-961551719a7d","Type":"ContainerDied","Data":"d8d93d1ddbaec4597147921eac145f6223bd6f3d5d1e15e0f4f400c39bf468f1"}
Oct 31 17:43:12.010508 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:12.010410    4276 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-zsl5x"
Oct 31 17:43:12.010508 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:12.010432    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-zsl5x" event={"ID":"b2b7f15e-756a-40e6-90af-961551719a7d","Type":"ContainerDied","Data":"cb136a0a66bdd40f74a955e6da45153396d648385a1fd675d7132591b9698c44"}
Oct 31 17:43:12.015301 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:12.015274    4276 patch_prober.go:29] interesting pod/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-m99sg container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 31 17:43:12.015420 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:12.015322    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-m99sg" podUID="56c4ff3e-b2f2-4dce-938d-48672451d1fe" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 17:43:12.017553 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:12.017533    4276 scope.go:122] "RemoveContainer" containerID="f55bc0bfbf838cd79b47049ad0a717173d0a9bf489fa6d669724fff4c02d1776"
Oct 31 17:43:12.017779 ip-10-0-129-127 kubenswrapper[4276]: E1031 17:43:12.017758    4276 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f55bc0bfbf838cd79b47049ad0a717173d0a9bf489fa6d669724fff4c02d1776\": container with ID starting with f55bc0bfbf838cd79b47049ad0a717173d0a9bf489fa6d669724fff4c02d1776 not found: ID does not exist" containerID="f55bc0bfbf838cd79b47049ad0a717173d0a9bf489fa6d669724fff4c02d1776"
Oct 31 17:43:12.017867 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:12.017786    4276 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f55bc0bfbf838cd79b47049ad0a717173d0a9bf489fa6d669724fff4c02d1776"} err="failed to get container status \"f55bc0bfbf838cd79b47049ad0a717173d0a9bf489fa6d669724fff4c02d1776\": rpc error: code = NotFound desc = could not find container \"f55bc0bfbf838cd79b47049ad0a717173d0a9bf489fa6d669724fff4c02d1776\": container with ID starting with f55bc0bfbf838cd79b47049ad0a717173d0a9bf489fa6d669724fff4c02d1776 not found: ID does not exist"
Oct 31 17:43:12.017867 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:12.017807    4276 scope.go:122] "RemoveContainer" containerID="122ad55efb2bd53bf8bad3bce430e571a33aa4bfe0469d290e9c49dc7e3b99a5"
Oct 31 17:43:12.027191 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:12.027174    4276 scope.go:122] "RemoveContainer" containerID="122ad55efb2bd53bf8bad3bce430e571a33aa4bfe0469d290e9c49dc7e3b99a5"
Oct 31 17:43:12.027470 ip-10-0-129-127 kubenswrapper[4276]: E1031 17:43:12.027431    4276 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"122ad55efb2bd53bf8bad3bce430e571a33aa4bfe0469d290e9c49dc7e3b99a5\": container with ID starting with 122ad55efb2bd53bf8bad3bce430e571a33aa4bfe0469d290e9c49dc7e3b99a5 not found: ID does not exist" containerID="122ad55efb2bd53bf8bad3bce430e571a33aa4bfe0469d290e9c49dc7e3b99a5"
Oct 31 17:43:12.027528 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:12.027470    4276 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"122ad55efb2bd53bf8bad3bce430e571a33aa4bfe0469d290e9c49dc7e3b99a5"} err="failed to get container status \"122ad55efb2bd53bf8bad3bce430e571a33aa4bfe0469d290e9c49dc7e3b99a5\": rpc error: code = NotFound desc = could not find container \"122ad55efb2bd53bf8bad3bce430e571a33aa4bfe0469d290e9c49dc7e3b99a5\": container with ID starting with 122ad55efb2bd53bf8bad3bce430e571a33aa4bfe0469d290e9c49dc7e3b99a5 not found: ID does not exist"
Oct 31 17:43:12.027528 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:12.027485    4276 scope.go:122] "RemoveContainer" containerID="06a63ac7ce2591fda4bf36eab5086dc1977b339ae31f3a5f73921a0d03fe94c2"
Oct 31 17:43:12.035356 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:12.035341    4276 scope.go:122] "RemoveContainer" containerID="06a63ac7ce2591fda4bf36eab5086dc1977b339ae31f3a5f73921a0d03fe94c2"
Oct 31 17:43:12.035608 ip-10-0-129-127 kubenswrapper[4276]: E1031 17:43:12.035592    4276 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"06a63ac7ce2591fda4bf36eab5086dc1977b339ae31f3a5f73921a0d03fe94c2\": container with ID starting with 06a63ac7ce2591fda4bf36eab5086dc1977b339ae31f3a5f73921a0d03fe94c2 not found: ID does not exist" containerID="06a63ac7ce2591fda4bf36eab5086dc1977b339ae31f3a5f73921a0d03fe94c2"
Oct 31 17:43:12.035666 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:12.035615    4276 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"06a63ac7ce2591fda4bf36eab5086dc1977b339ae31f3a5f73921a0d03fe94c2"} err="failed to get container status \"06a63ac7ce2591fda4bf36eab5086dc1977b339ae31f3a5f73921a0d03fe94c2\": rpc error: code = NotFound desc = could not find container \"06a63ac7ce2591fda4bf36eab5086dc1977b339ae31f3a5f73921a0d03fe94c2\": container with ID starting with 06a63ac7ce2591fda4bf36eab5086dc1977b339ae31f3a5f73921a0d03fe94c2 not found: ID does not exist"
Oct 31 17:43:12.035666 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:12.035630    4276 scope.go:122] "RemoveContainer" containerID="d8d93d1ddbaec4597147921eac145f6223bd6f3d5d1e15e0f4f400c39bf468f1"
Oct 31 17:43:12.044153 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:12.044136    4276 scope.go:122] "RemoveContainer" containerID="d8d93d1ddbaec4597147921eac145f6223bd6f3d5d1e15e0f4f400c39bf468f1"
Oct 31 17:43:12.044419 ip-10-0-129-127 kubenswrapper[4276]: E1031 17:43:12.044400    4276 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d8d93d1ddbaec4597147921eac145f6223bd6f3d5d1e15e0f4f400c39bf468f1\": container with ID starting with d8d93d1ddbaec4597147921eac145f6223bd6f3d5d1e15e0f4f400c39bf468f1 not found: ID does not exist" containerID="d8d93d1ddbaec4597147921eac145f6223bd6f3d5d1e15e0f4f400c39bf468f1"
Oct 31 17:43:12.044528 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:12.044428    4276 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d8d93d1ddbaec4597147921eac145f6223bd6f3d5d1e15e0f4f400c39bf468f1"} err="failed to get container status \"d8d93d1ddbaec4597147921eac145f6223bd6f3d5d1e15e0f4f400c39bf468f1\": rpc error: code = NotFound desc = could not find container \"d8d93d1ddbaec4597147921eac145f6223bd6f3d5d1e15e0f4f400c39bf468f1\": container with ID starting with d8d93d1ddbaec4597147921eac145f6223bd6f3d5d1e15e0f4f400c39bf468f1 not found: ID does not exist"
Oct 31 17:43:12.083677 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:12.083649    4276 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-2v74j"]
Oct 31 17:43:12.106368 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:12.106346    4276 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-2v74j"]
Oct 31 17:43:12.146342 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:12.146279    4276 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-94vcs"]
Oct 31 17:43:12.155213 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:12.155176    4276 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-94vcs"]
Oct 31 17:43:12.197270 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:12.197238    4276 patch_prober.go:29] interesting pod/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-m7vsj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 31 17:43:12.197409 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:12.197301    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-m7vsj" podUID="f675ff71-27f9-44e8-aa32-89f4332d4c3e" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 17:43:12.216786 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:12.216757    4276 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-dx9cd"]
Oct 31 17:43:12.227671 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:12.227644    4276 patch_prober.go:29] interesting pod/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-8bf7k container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 31 17:43:12.227801 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:12.227693    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-8bf7k" podUID="28cbdb7c-1bc2-49b4-834c-de871be33524" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 17:43:12.229969 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:12.229945    4276 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-dx9cd"]
Oct 31 17:43:12.239249 ip-10-0-129-127 kubenswrapper[4276]: E1031 17:43:12.239223    4276 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tg5qk" containerName="extractor"
Oct 31 17:43:12.266324 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:12.266297    4276 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-zsl5x"]
Oct 31 17:43:12.271162 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:12.271137    4276 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-b78c31-openshift-default-774cb4f6c9-zsl5x"]
Oct 31 17:43:12.319959 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:12.319931    4276 patch_prober.go:29] interesting pod/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-kfkwb container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 31 17:43:12.320080 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:12.319993    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-kfkwb" podUID="4b04b1b0-d2cc-4304-92e6-90e48105f442" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 17:43:12.550125 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:12.550094    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:43:12.664966 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:12.664933    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:43:13.018741 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:13.018653    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:43:13.366520 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:13.366489    4276 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3716260a-fa38-4064-991c-48322b1b48fa" path="/var/lib/kubelet/pods/3716260a-fa38-4064-991c-48322b1b48fa/volumes"
Oct 31 17:43:13.367213 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:13.367186    4276 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4c3cb076-d13d-4212-8fbb-86c851b14fe7" path="/var/lib/kubelet/pods/4c3cb076-d13d-4212-8fbb-86c851b14fe7/volumes"
Oct 31 17:43:13.367676 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:13.367662    4276 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b2b7f15e-756a-40e6-90af-961551719a7d" path="/var/lib/kubelet/pods/b2b7f15e-756a-40e6-90af-961551719a7d/volumes"
Oct 31 17:43:13.368356 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:13.368342    4276 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e948a32e-25a4-4d29-b9dd-01a0539d337c" path="/var/lib/kubelet/pods/e948a32e-25a4-4d29-b9dd-01a0539d337c/volumes"
Oct 31 17:43:13.637931 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:13.637652    4276 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-m7vsj"
Oct 31 17:43:13.708466 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:13.691390    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f675ff71-27f9-44e8-aa32-89f4332d4c3e-istio-token\" (UniqueName: \"kubernetes.io/projected/f675ff71-27f9-44e8-aa32-89f4332d4c3e-istio-token\") pod \"f675ff71-27f9-44e8-aa32-89f4332d4c3e\" (UID: \"f675ff71-27f9-44e8-aa32-89f4332d4c3e\") "
Oct 31 17:43:13.708466 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:13.698730    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f675ff71-27f9-44e8-aa32-89f4332d4c3e-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f675ff71-27f9-44e8-aa32-89f4332d4c3e-istio-data\") pod \"f675ff71-27f9-44e8-aa32-89f4332d4c3e\" (UID: \"f675ff71-27f9-44e8-aa32-89f4332d4c3e\") "
Oct 31 17:43:13.708466 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:13.698813    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/f675ff71-27f9-44e8-aa32-89f4332d4c3e-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f675ff71-27f9-44e8-aa32-89f4332d4c3e-istio-podinfo\") pod \"f675ff71-27f9-44e8-aa32-89f4332d4c3e\" (UID: \"f675ff71-27f9-44e8-aa32-89f4332d4c3e\") "
Oct 31 17:43:13.708466 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:13.698847    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f675ff71-27f9-44e8-aa32-89f4332d4c3e-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f675ff71-27f9-44e8-aa32-89f4332d4c3e-credential-socket\") pod \"f675ff71-27f9-44e8-aa32-89f4332d4c3e\" (UID: \"f675ff71-27f9-44e8-aa32-89f4332d4c3e\") "
Oct 31 17:43:13.708466 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:13.698903    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f675ff71-27f9-44e8-aa32-89f4332d4c3e-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f675ff71-27f9-44e8-aa32-89f4332d4c3e-istiod-ca-cert\") pod \"f675ff71-27f9-44e8-aa32-89f4332d4c3e\" (UID: \"f675ff71-27f9-44e8-aa32-89f4332d4c3e\") "
Oct 31 17:43:13.708466 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:13.699357    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f675ff71-27f9-44e8-aa32-89f4332d4c3e-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f675ff71-27f9-44e8-aa32-89f4332d4c3e-workload-certs\") pod \"f675ff71-27f9-44e8-aa32-89f4332d4c3e\" (UID: \"f675ff71-27f9-44e8-aa32-89f4332d4c3e\") "
Oct 31 17:43:13.708466 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:13.699469    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f675ff71-27f9-44e8-aa32-89f4332d4c3e-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f675ff71-27f9-44e8-aa32-89f4332d4c3e-istio-envoy\") pod \"f675ff71-27f9-44e8-aa32-89f4332d4c3e\" (UID: \"f675ff71-27f9-44e8-aa32-89f4332d4c3e\") "
Oct 31 17:43:13.708466 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:13.699539    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f675ff71-27f9-44e8-aa32-89f4332d4c3e-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f675ff71-27f9-44e8-aa32-89f4332d4c3e-workload-socket\") pod \"f675ff71-27f9-44e8-aa32-89f4332d4c3e\" (UID: \"f675ff71-27f9-44e8-aa32-89f4332d4c3e\") "
Oct 31 17:43:13.708466 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:13.699650    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f675ff71-27f9-44e8-aa32-89f4332d4c3e-kube-api-access-7hrd5\" (UniqueName: \"kubernetes.io/projected/f675ff71-27f9-44e8-aa32-89f4332d4c3e-kube-api-access-7hrd5\") pod \"f675ff71-27f9-44e8-aa32-89f4332d4c3e\" (UID: \"f675ff71-27f9-44e8-aa32-89f4332d4c3e\") "
Oct 31 17:43:13.708466 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:13.700126    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f675ff71-27f9-44e8-aa32-89f4332d4c3e-workload-certs" pod "f675ff71-27f9-44e8-aa32-89f4332d4c3e" (UID: "f675ff71-27f9-44e8-aa32-89f4332d4c3e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:43:13.708466 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:13.700228    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f675ff71-27f9-44e8-aa32-89f4332d4c3e-istiod-ca-cert" pod "f675ff71-27f9-44e8-aa32-89f4332d4c3e" (UID: "f675ff71-27f9-44e8-aa32-89f4332d4c3e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 17:43:13.708466 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:13.700497    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f675ff71-27f9-44e8-aa32-89f4332d4c3e-istio-data" pod "f675ff71-27f9-44e8-aa32-89f4332d4c3e" (UID: "f675ff71-27f9-44e8-aa32-89f4332d4c3e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:43:13.708466 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:13.700567    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f675ff71-27f9-44e8-aa32-89f4332d4c3e-credential-socket" pod "f675ff71-27f9-44e8-aa32-89f4332d4c3e" (UID: "f675ff71-27f9-44e8-aa32-89f4332d4c3e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:43:13.708466 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:13.700631    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f675ff71-27f9-44e8-aa32-89f4332d4c3e-workload-socket" pod "f675ff71-27f9-44e8-aa32-89f4332d4c3e" (UID: "f675ff71-27f9-44e8-aa32-89f4332d4c3e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:43:13.708466 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:13.704250    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f675ff71-27f9-44e8-aa32-89f4332d4c3e-istio-token" pod "f675ff71-27f9-44e8-aa32-89f4332d4c3e" (UID: "f675ff71-27f9-44e8-aa32-89f4332d4c3e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 17:43:13.708466 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:13.706768    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f675ff71-27f9-44e8-aa32-89f4332d4c3e-istio-podinfo" pod "f675ff71-27f9-44e8-aa32-89f4332d4c3e" (UID: "f675ff71-27f9-44e8-aa32-89f4332d4c3e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 17:43:13.708466 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:13.706944    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f675ff71-27f9-44e8-aa32-89f4332d4c3e-istio-envoy" pod "f675ff71-27f9-44e8-aa32-89f4332d4c3e" (UID: "f675ff71-27f9-44e8-aa32-89f4332d4c3e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:43:13.709412 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:13.709138    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f675ff71-27f9-44e8-aa32-89f4332d4c3e-kube-api-access-7hrd5" pod "f675ff71-27f9-44e8-aa32-89f4332d4c3e" (UID: "f675ff71-27f9-44e8-aa32-89f4332d4c3e"). InnerVolumeSpecName "kube-api-access-7hrd5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 17:43:13.716941 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:13.716913    4276 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-ghwks"
Oct 31 17:43:13.721709 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:13.721684    4276 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-qlhn8"
Oct 31 17:43:13.733725 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:13.725857    4276 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-8bf7k"
Oct 31 17:43:13.801195 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:13.801166    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/33671198-f448-4f99-9bc9-6a1ed6980c5b-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/33671198-f448-4f99-9bc9-6a1ed6980c5b-credential-socket\") pod \"33671198-f448-4f99-9bc9-6a1ed6980c5b\" (UID: \"33671198-f448-4f99-9bc9-6a1ed6980c5b\") "
Oct 31 17:43:13.801355 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:13.801210    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/28cbdb7c-1bc2-49b4-834c-de871be33524-istio-token\" (UniqueName: \"kubernetes.io/projected/28cbdb7c-1bc2-49b4-834c-de871be33524-istio-token\") pod \"28cbdb7c-1bc2-49b4-834c-de871be33524\" (UID: \"28cbdb7c-1bc2-49b4-834c-de871be33524\") "
Oct 31 17:43:13.801355 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:13.801239    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/28cbdb7c-1bc2-49b4-834c-de871be33524-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/28cbdb7c-1bc2-49b4-834c-de871be33524-credential-socket\") pod \"28cbdb7c-1bc2-49b4-834c-de871be33524\" (UID: \"28cbdb7c-1bc2-49b4-834c-de871be33524\") "
Oct 31 17:43:13.801355 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:13.801295    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/33671198-f448-4f99-9bc9-6a1ed6980c5b-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/33671198-f448-4f99-9bc9-6a1ed6980c5b-workload-certs\") pod \"33671198-f448-4f99-9bc9-6a1ed6980c5b\" (UID: \"33671198-f448-4f99-9bc9-6a1ed6980c5b\") "
Oct 31 17:43:13.801355 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:13.801324    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/33671198-f448-4f99-9bc9-6a1ed6980c5b-kube-api-access-7tmrh\" (UniqueName: \"kubernetes.io/projected/33671198-f448-4f99-9bc9-6a1ed6980c5b-kube-api-access-7tmrh\") pod \"33671198-f448-4f99-9bc9-6a1ed6980c5b\" (UID: \"33671198-f448-4f99-9bc9-6a1ed6980c5b\") "
Oct 31 17:43:13.801355 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:13.801352    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c3c40ebe-5cac-4981-913d-18f12d0bf439-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c3c40ebe-5cac-4981-913d-18f12d0bf439-istio-envoy\") pod \"c3c40ebe-5cac-4981-913d-18f12d0bf439\" (UID: \"c3c40ebe-5cac-4981-913d-18f12d0bf439\") "
Oct 31 17:43:13.801633 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:13.801387    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/33671198-f448-4f99-9bc9-6a1ed6980c5b-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/33671198-f448-4f99-9bc9-6a1ed6980c5b-istio-envoy\") pod \"33671198-f448-4f99-9bc9-6a1ed6980c5b\" (UID: \"33671198-f448-4f99-9bc9-6a1ed6980c5b\") "
Oct 31 17:43:13.801633 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:13.801432    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/33671198-f448-4f99-9bc9-6a1ed6980c5b-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/33671198-f448-4f99-9bc9-6a1ed6980c5b-istiod-ca-cert\") pod \"33671198-f448-4f99-9bc9-6a1ed6980c5b\" (UID: \"33671198-f448-4f99-9bc9-6a1ed6980c5b\") "
Oct 31 17:43:13.801633 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:13.801474    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/c3c40ebe-5cac-4981-913d-18f12d0bf439-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c3c40ebe-5cac-4981-913d-18f12d0bf439-istio-podinfo\") pod \"c3c40ebe-5cac-4981-913d-18f12d0bf439\" (UID: \"c3c40ebe-5cac-4981-913d-18f12d0bf439\") "
Oct 31 17:43:13.801633 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:13.801515    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/28cbdb7c-1bc2-49b4-834c-de871be33524-kube-api-access-schh7\" (UniqueName: \"kubernetes.io/projected/28cbdb7c-1bc2-49b4-834c-de871be33524-kube-api-access-schh7\") pod \"28cbdb7c-1bc2-49b4-834c-de871be33524\" (UID: \"28cbdb7c-1bc2-49b4-834c-de871be33524\") "
Oct 31 17:43:13.801633 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:13.801523    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33671198-f448-4f99-9bc9-6a1ed6980c5b-credential-socket" pod "33671198-f448-4f99-9bc9-6a1ed6980c5b" (UID: "33671198-f448-4f99-9bc9-6a1ed6980c5b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:43:13.801633 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:13.801565    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c3c40ebe-5cac-4981-913d-18f12d0bf439-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c3c40ebe-5cac-4981-913d-18f12d0bf439-istiod-ca-cert\") pod \"c3c40ebe-5cac-4981-913d-18f12d0bf439\" (UID: \"c3c40ebe-5cac-4981-913d-18f12d0bf439\") "
Oct 31 17:43:13.801633 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:13.801597    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/33671198-f448-4f99-9bc9-6a1ed6980c5b-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/33671198-f448-4f99-9bc9-6a1ed6980c5b-istio-podinfo\") pod \"33671198-f448-4f99-9bc9-6a1ed6980c5b\" (UID: \"33671198-f448-4f99-9bc9-6a1ed6980c5b\") "
Oct 31 17:43:13.801633 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:13.801601    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/28cbdb7c-1bc2-49b4-834c-de871be33524-credential-socket" pod "28cbdb7c-1bc2-49b4-834c-de871be33524" (UID: "28cbdb7c-1bc2-49b4-834c-de871be33524"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:43:13.801979 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:13.801658    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c3c40ebe-5cac-4981-913d-18f12d0bf439-istio-token\" (UniqueName: \"kubernetes.io/projected/c3c40ebe-5cac-4981-913d-18f12d0bf439-istio-token\") pod \"c3c40ebe-5cac-4981-913d-18f12d0bf439\" (UID: \"c3c40ebe-5cac-4981-913d-18f12d0bf439\") "
Oct 31 17:43:13.801979 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:13.801689    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/28cbdb7c-1bc2-49b4-834c-de871be33524-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/28cbdb7c-1bc2-49b4-834c-de871be33524-istio-podinfo\") pod \"28cbdb7c-1bc2-49b4-834c-de871be33524\" (UID: \"28cbdb7c-1bc2-49b4-834c-de871be33524\") "
Oct 31 17:43:13.801979 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:13.801737    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c3c40ebe-5cac-4981-913d-18f12d0bf439-kube-api-access-w7v7k\" (UniqueName: \"kubernetes.io/projected/c3c40ebe-5cac-4981-913d-18f12d0bf439-kube-api-access-w7v7k\") pod \"c3c40ebe-5cac-4981-913d-18f12d0bf439\" (UID: \"c3c40ebe-5cac-4981-913d-18f12d0bf439\") "
Oct 31 17:43:13.801979 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:13.801766    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/28cbdb7c-1bc2-49b4-834c-de871be33524-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/28cbdb7c-1bc2-49b4-834c-de871be33524-istio-data\") pod \"28cbdb7c-1bc2-49b4-834c-de871be33524\" (UID: \"28cbdb7c-1bc2-49b4-834c-de871be33524\") "
Oct 31 17:43:13.801979 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:13.801825    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c3c40ebe-5cac-4981-913d-18f12d0bf439-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c3c40ebe-5cac-4981-913d-18f12d0bf439-workload-certs\") pod \"c3c40ebe-5cac-4981-913d-18f12d0bf439\" (UID: \"c3c40ebe-5cac-4981-913d-18f12d0bf439\") "
Oct 31 17:43:13.801979 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:13.801908    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c3c40ebe-5cac-4981-913d-18f12d0bf439-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3c40ebe-5cac-4981-913d-18f12d0bf439-workload-socket\") pod \"c3c40ebe-5cac-4981-913d-18f12d0bf439\" (UID: \"c3c40ebe-5cac-4981-913d-18f12d0bf439\") "
Oct 31 17:43:13.801979 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:13.801939    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/28cbdb7c-1bc2-49b4-834c-de871be33524-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/28cbdb7c-1bc2-49b4-834c-de871be33524-workload-socket\") pod \"28cbdb7c-1bc2-49b4-834c-de871be33524\" (UID: \"28cbdb7c-1bc2-49b4-834c-de871be33524\") "
Oct 31 17:43:13.801979 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:13.801967    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/33671198-f448-4f99-9bc9-6a1ed6980c5b-istio-token\" (UniqueName: \"kubernetes.io/projected/33671198-f448-4f99-9bc9-6a1ed6980c5b-istio-token\") pod \"33671198-f448-4f99-9bc9-6a1ed6980c5b\" (UID: \"33671198-f448-4f99-9bc9-6a1ed6980c5b\") "
Oct 31 17:43:13.802222 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:13.801998    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c3c40ebe-5cac-4981-913d-18f12d0bf439-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3c40ebe-5cac-4981-913d-18f12d0bf439-credential-socket\") pod \"c3c40ebe-5cac-4981-913d-18f12d0bf439\" (UID: \"c3c40ebe-5cac-4981-913d-18f12d0bf439\") "
Oct 31 17:43:13.802222 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:13.802025    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/28cbdb7c-1bc2-49b4-834c-de871be33524-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/28cbdb7c-1bc2-49b4-834c-de871be33524-istiod-ca-cert\") pod \"28cbdb7c-1bc2-49b4-834c-de871be33524\" (UID: \"28cbdb7c-1bc2-49b4-834c-de871be33524\") "
Oct 31 17:43:13.802222 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:13.802054    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/28cbdb7c-1bc2-49b4-834c-de871be33524-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/28cbdb7c-1bc2-49b4-834c-de871be33524-workload-certs\") pod \"28cbdb7c-1bc2-49b4-834c-de871be33524\" (UID: \"28cbdb7c-1bc2-49b4-834c-de871be33524\") "
Oct 31 17:43:13.802222 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:13.802085    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c3c40ebe-5cac-4981-913d-18f12d0bf439-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c3c40ebe-5cac-4981-913d-18f12d0bf439-istio-data\") pod \"c3c40ebe-5cac-4981-913d-18f12d0bf439\" (UID: \"c3c40ebe-5cac-4981-913d-18f12d0bf439\") "
Oct 31 17:43:13.802222 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:13.802116    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/33671198-f448-4f99-9bc9-6a1ed6980c5b-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/33671198-f448-4f99-9bc9-6a1ed6980c5b-istio-data\") pod \"33671198-f448-4f99-9bc9-6a1ed6980c5b\" (UID: \"33671198-f448-4f99-9bc9-6a1ed6980c5b\") "
Oct 31 17:43:13.802222 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:13.802167    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/33671198-f448-4f99-9bc9-6a1ed6980c5b-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/33671198-f448-4f99-9bc9-6a1ed6980c5b-workload-socket\") pod \"33671198-f448-4f99-9bc9-6a1ed6980c5b\" (UID: \"33671198-f448-4f99-9bc9-6a1ed6980c5b\") "
Oct 31 17:43:13.802222 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:13.802207    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/28cbdb7c-1bc2-49b4-834c-de871be33524-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/28cbdb7c-1bc2-49b4-834c-de871be33524-istio-envoy\") pod \"28cbdb7c-1bc2-49b4-834c-de871be33524\" (UID: \"28cbdb7c-1bc2-49b4-834c-de871be33524\") "
Oct 31 17:43:13.802471 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:13.802295    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/33671198-f448-4f99-9bc9-6a1ed6980c5b-istiod-ca-cert" pod "33671198-f448-4f99-9bc9-6a1ed6980c5b" (UID: "33671198-f448-4f99-9bc9-6a1ed6980c5b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 17:43:13.802561 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:13.802537    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33671198-f448-4f99-9bc9-6a1ed6980c5b-workload-certs" pod "33671198-f448-4f99-9bc9-6a1ed6980c5b" (UID: "33671198-f448-4f99-9bc9-6a1ed6980c5b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:43:13.803006 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:13.802905    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c3c40ebe-5cac-4981-913d-18f12d0bf439-istiod-ca-cert" pod "c3c40ebe-5cac-4981-913d-18f12d0bf439" (UID: "c3c40ebe-5cac-4981-913d-18f12d0bf439"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 17:43:13.803185 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:13.802956    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/28cbdb7c-1bc2-49b4-834c-de871be33524-workload-socket" pod "28cbdb7c-1bc2-49b4-834c-de871be33524" (UID: "28cbdb7c-1bc2-49b4-834c-de871be33524"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:43:13.803652 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:13.803600    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/28cbdb7c-1bc2-49b4-834c-de871be33524-workload-certs" pod "28cbdb7c-1bc2-49b4-834c-de871be33524" (UID: "28cbdb7c-1bc2-49b4-834c-de871be33524"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:43:13.804213 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:13.803752    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c3c40ebe-5cac-4981-913d-18f12d0bf439-workload-certs" pod "c3c40ebe-5cac-4981-913d-18f12d0bf439" (UID: "c3c40ebe-5cac-4981-913d-18f12d0bf439"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:43:13.804213 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:13.804038    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c3c40ebe-5cac-4981-913d-18f12d0bf439-credential-socket" pod "c3c40ebe-5cac-4981-913d-18f12d0bf439" (UID: "c3c40ebe-5cac-4981-913d-18f12d0bf439"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:43:13.804213 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:13.803954    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/28cbdb7c-1bc2-49b4-834c-de871be33524-istiod-ca-cert" pod "28cbdb7c-1bc2-49b4-834c-de871be33524" (UID: "28cbdb7c-1bc2-49b4-834c-de871be33524"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 17:43:13.804432 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:13.804239    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33671198-f448-4f99-9bc9-6a1ed6980c5b-workload-socket" pod "33671198-f448-4f99-9bc9-6a1ed6980c5b" (UID: "33671198-f448-4f99-9bc9-6a1ed6980c5b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:43:13.804432 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:13.804288    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c3c40ebe-5cac-4981-913d-18f12d0bf439-istio-envoy" pod "c3c40ebe-5cac-4981-913d-18f12d0bf439" (UID: "c3c40ebe-5cac-4981-913d-18f12d0bf439"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:43:13.804432 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:13.804364    4276 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7hrd5\" (UniqueName: \"kubernetes.io/projected/f675ff71-27f9-44e8-aa32-89f4332d4c3e-kube-api-access-7hrd5\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:43:13.804432 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:13.804389    4276 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/33671198-f448-4f99-9bc9-6a1ed6980c5b-istiod-ca-cert\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:43:13.804432 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:13.804406    4276 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f675ff71-27f9-44e8-aa32-89f4332d4c3e-istio-token\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:43:13.804432 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:13.804419    4276 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f675ff71-27f9-44e8-aa32-89f4332d4c3e-istio-data\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:43:13.804772 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:13.804457    4276 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c3c40ebe-5cac-4981-913d-18f12d0bf439-istiod-ca-cert\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:43:13.804772 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:13.804470    4276 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f675ff71-27f9-44e8-aa32-89f4332d4c3e-istio-podinfo\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:43:13.804772 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:13.804483    4276 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f675ff71-27f9-44e8-aa32-89f4332d4c3e-credential-socket\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:43:13.804772 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:13.804496    4276 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f675ff71-27f9-44e8-aa32-89f4332d4c3e-istiod-ca-cert\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:43:13.804772 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:13.804524    4276 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f675ff71-27f9-44e8-aa32-89f4332d4c3e-workload-certs\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:43:13.804772 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:13.804537    4276 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/28cbdb7c-1bc2-49b4-834c-de871be33524-workload-socket\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:43:13.804772 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:13.804541    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/28cbdb7c-1bc2-49b4-834c-de871be33524-istio-data" pod "28cbdb7c-1bc2-49b4-834c-de871be33524" (UID: "28cbdb7c-1bc2-49b4-834c-de871be33524"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:43:13.804772 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:13.804552    4276 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/28cbdb7c-1bc2-49b4-834c-de871be33524-workload-certs\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:43:13.804772 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:13.804602    4276 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f675ff71-27f9-44e8-aa32-89f4332d4c3e-istio-envoy\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:43:13.804772 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:13.804617    4276 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/33671198-f448-4f99-9bc9-6a1ed6980c5b-credential-socket\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:43:13.804772 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:13.804619    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c3c40ebe-5cac-4981-913d-18f12d0bf439-workload-socket" pod "c3c40ebe-5cac-4981-913d-18f12d0bf439" (UID: "c3c40ebe-5cac-4981-913d-18f12d0bf439"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:43:13.804772 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:13.804631    4276 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/28cbdb7c-1bc2-49b4-834c-de871be33524-credential-socket\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:43:13.804772 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:13.804668    4276 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f675ff71-27f9-44e8-aa32-89f4332d4c3e-workload-socket\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:43:13.804772 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:13.804682    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33671198-f448-4f99-9bc9-6a1ed6980c5b-istio-data" pod "33671198-f448-4f99-9bc9-6a1ed6980c5b" (UID: "33671198-f448-4f99-9bc9-6a1ed6980c5b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:43:13.804772 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:13.804688    4276 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/33671198-f448-4f99-9bc9-6a1ed6980c5b-workload-certs\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:43:13.804772 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:13.804745    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c3c40ebe-5cac-4981-913d-18f12d0bf439-istio-data" pod "c3c40ebe-5cac-4981-913d-18f12d0bf439" (UID: "c3c40ebe-5cac-4981-913d-18f12d0bf439"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:43:13.805595 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:13.804896    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/33671198-f448-4f99-9bc9-6a1ed6980c5b-kube-api-access-7tmrh" pod "33671198-f448-4f99-9bc9-6a1ed6980c5b" (UID: "33671198-f448-4f99-9bc9-6a1ed6980c5b"). InnerVolumeSpecName "kube-api-access-7tmrh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 17:43:13.805595 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:13.805010    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/33671198-f448-4f99-9bc9-6a1ed6980c5b-istio-podinfo" pod "33671198-f448-4f99-9bc9-6a1ed6980c5b" (UID: "33671198-f448-4f99-9bc9-6a1ed6980c5b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 17:43:13.805850 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:13.805806    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/28cbdb7c-1bc2-49b4-834c-de871be33524-istio-token" pod "28cbdb7c-1bc2-49b4-834c-de871be33524" (UID: "28cbdb7c-1bc2-49b4-834c-de871be33524"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 17:43:13.805943 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:13.805920    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/28cbdb7c-1bc2-49b4-834c-de871be33524-istio-envoy" pod "28cbdb7c-1bc2-49b4-834c-de871be33524" (UID: "28cbdb7c-1bc2-49b4-834c-de871be33524"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:43:13.806139 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:13.806114    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/28cbdb7c-1bc2-49b4-834c-de871be33524-kube-api-access-schh7" pod "28cbdb7c-1bc2-49b4-834c-de871be33524" (UID: "28cbdb7c-1bc2-49b4-834c-de871be33524"). InnerVolumeSpecName "kube-api-access-schh7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 17:43:13.806231 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:13.806137    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33671198-f448-4f99-9bc9-6a1ed6980c5b-istio-envoy" pod "33671198-f448-4f99-9bc9-6a1ed6980c5b" (UID: "33671198-f448-4f99-9bc9-6a1ed6980c5b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:43:13.806298 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:13.806274    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c3c40ebe-5cac-4981-913d-18f12d0bf439-istio-podinfo" pod "c3c40ebe-5cac-4981-913d-18f12d0bf439" (UID: "c3c40ebe-5cac-4981-913d-18f12d0bf439"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 17:43:13.806342 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:13.806309    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/33671198-f448-4f99-9bc9-6a1ed6980c5b-istio-token" pod "33671198-f448-4f99-9bc9-6a1ed6980c5b" (UID: "33671198-f448-4f99-9bc9-6a1ed6980c5b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 17:43:13.806662 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:13.806642    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c3c40ebe-5cac-4981-913d-18f12d0bf439-istio-token" pod "c3c40ebe-5cac-4981-913d-18f12d0bf439" (UID: "c3c40ebe-5cac-4981-913d-18f12d0bf439"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 17:43:13.806816 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:13.806801    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/28cbdb7c-1bc2-49b4-834c-de871be33524-istio-podinfo" pod "28cbdb7c-1bc2-49b4-834c-de871be33524" (UID: "28cbdb7c-1bc2-49b4-834c-de871be33524"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 17:43:13.807431 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:13.807416    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c3c40ebe-5cac-4981-913d-18f12d0bf439-kube-api-access-w7v7k" pod "c3c40ebe-5cac-4981-913d-18f12d0bf439" (UID: "c3c40ebe-5cac-4981-913d-18f12d0bf439"). InnerVolumeSpecName "kube-api-access-w7v7k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 17:43:13.906114 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:13.906032    4276 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c3c40ebe-5cac-4981-913d-18f12d0bf439-istio-data\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:43:13.906114 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:13.906062    4276 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/33671198-f448-4f99-9bc9-6a1ed6980c5b-istio-data\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:43:13.906114 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:13.906075    4276 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/33671198-f448-4f99-9bc9-6a1ed6980c5b-workload-socket\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:43:13.906114 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:13.906088    4276 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/28cbdb7c-1bc2-49b4-834c-de871be33524-istio-envoy\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:43:13.906114 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:13.906101    4276 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/28cbdb7c-1bc2-49b4-834c-de871be33524-istio-token\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:43:13.906114 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:13.906113    4276 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7tmrh\" (UniqueName: \"kubernetes.io/projected/33671198-f448-4f99-9bc9-6a1ed6980c5b-kube-api-access-7tmrh\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:43:13.906483 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:13.906126    4276 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c3c40ebe-5cac-4981-913d-18f12d0bf439-istio-envoy\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:43:13.906483 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:13.906137    4276 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/33671198-f448-4f99-9bc9-6a1ed6980c5b-istio-envoy\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:43:13.906483 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:13.906148    4276 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c3c40ebe-5cac-4981-913d-18f12d0bf439-istio-podinfo\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:43:13.906483 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:13.906158    4276 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-schh7\" (UniqueName: \"kubernetes.io/projected/28cbdb7c-1bc2-49b4-834c-de871be33524-kube-api-access-schh7\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:43:13.906483 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:13.906169    4276 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/33671198-f448-4f99-9bc9-6a1ed6980c5b-istio-podinfo\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:43:13.906483 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:13.906181    4276 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c3c40ebe-5cac-4981-913d-18f12d0bf439-istio-token\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:43:13.906483 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:13.906196    4276 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/28cbdb7c-1bc2-49b4-834c-de871be33524-istio-podinfo\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:43:13.906483 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:13.906208    4276 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w7v7k\" (UniqueName: \"kubernetes.io/projected/c3c40ebe-5cac-4981-913d-18f12d0bf439-kube-api-access-w7v7k\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:43:13.906483 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:13.906219    4276 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/28cbdb7c-1bc2-49b4-834c-de871be33524-istio-data\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:43:13.906483 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:13.906231    4276 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c3c40ebe-5cac-4981-913d-18f12d0bf439-workload-certs\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:43:13.906483 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:13.906243    4276 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3c40ebe-5cac-4981-913d-18f12d0bf439-workload-socket\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:43:13.906483 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:13.906254    4276 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/33671198-f448-4f99-9bc9-6a1ed6980c5b-istio-token\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:43:13.906483 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:13.906265    4276 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3c40ebe-5cac-4981-913d-18f12d0bf439-credential-socket\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:43:13.906483 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:13.906276    4276 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/28cbdb7c-1bc2-49b4-834c-de871be33524-istiod-ca-cert\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:43:14.029662 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:14.029625    4276 generic.go:356] "Generic (PLEG): container finished" podID="f675ff71-27f9-44e8-aa32-89f4332d4c3e" containerID="1019d46050d0bb7e6f4da0f693b5f99640f2050d06769321e7a4b15b04524aa5" exitCode=0
Oct 31 17:43:14.029849 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:14.029698    4276 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-m7vsj"
Oct 31 17:43:14.029849 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:14.029709    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-m7vsj" event={"ID":"f675ff71-27f9-44e8-aa32-89f4332d4c3e","Type":"ContainerDied","Data":"1019d46050d0bb7e6f4da0f693b5f99640f2050d06769321e7a4b15b04524aa5"}
Oct 31 17:43:14.029849 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:14.029800    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-m7vsj" event={"ID":"f675ff71-27f9-44e8-aa32-89f4332d4c3e","Type":"ContainerDied","Data":"a65b177789025d69ab7307359f5a74e5bb514e1f3db3d8ebe1d3ead08b2a7d94"}
Oct 31 17:43:14.029849 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:14.029834    4276 scope.go:122] "RemoveContainer" containerID="1019d46050d0bb7e6f4da0f693b5f99640f2050d06769321e7a4b15b04524aa5"
Oct 31 17:43:14.031457 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:14.031425    4276 generic.go:356] "Generic (PLEG): container finished" podID="33671198-f448-4f99-9bc9-6a1ed6980c5b" containerID="09b8ee7e51fe351cc0bcc6f9d3dbcf11853e736da058d8d30cb89388f02fbd0c" exitCode=0
Oct 31 17:43:14.031572 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:14.031509    4276 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-ghwks"
Oct 31 17:43:14.031572 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:14.031511    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-ghwks" event={"ID":"33671198-f448-4f99-9bc9-6a1ed6980c5b","Type":"ContainerDied","Data":"09b8ee7e51fe351cc0bcc6f9d3dbcf11853e736da058d8d30cb89388f02fbd0c"}
Oct 31 17:43:14.031572 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:14.031561    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-ghwks" event={"ID":"33671198-f448-4f99-9bc9-6a1ed6980c5b","Type":"ContainerDied","Data":"4558c1ed5122febee619605d8afdf42296fb026613c51bfe4bfbe4374ed469aa"}
Oct 31 17:43:14.032935 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:14.032909    4276 generic.go:356] "Generic (PLEG): container finished" podID="c3c40ebe-5cac-4981-913d-18f12d0bf439" containerID="395370f5dee6ae088f35afe2eeb8f02c629d745ac4ddf5cacb9129b95407042a" exitCode=0
Oct 31 17:43:14.033059 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:14.032954    4276 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-qlhn8"
Oct 31 17:43:14.033059 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:14.032999    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-qlhn8" event={"ID":"c3c40ebe-5cac-4981-913d-18f12d0bf439","Type":"ContainerDied","Data":"395370f5dee6ae088f35afe2eeb8f02c629d745ac4ddf5cacb9129b95407042a"}
Oct 31 17:43:14.033059 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:14.033047    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-qlhn8" event={"ID":"c3c40ebe-5cac-4981-913d-18f12d0bf439","Type":"ContainerDied","Data":"2e3de26ce2cb6849e35588910fe921c455392f80c54b6040b7926b943fa54d0b"}
Oct 31 17:43:14.034706 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:14.034681    4276 generic.go:356] "Generic (PLEG): container finished" podID="28cbdb7c-1bc2-49b4-834c-de871be33524" containerID="1b62347ed05ecd31efa2e9cb7a621deeb263f006afe6411c319be80c5cd86a75" exitCode=0
Oct 31 17:43:14.034826 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:14.034705    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-8bf7k" event={"ID":"28cbdb7c-1bc2-49b4-834c-de871be33524","Type":"ContainerDied","Data":"1b62347ed05ecd31efa2e9cb7a621deeb263f006afe6411c319be80c5cd86a75"}
Oct 31 17:43:14.034826 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:14.034730    4276 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-8bf7k"
Oct 31 17:43:14.034826 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:14.034755    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-8bf7k" event={"ID":"28cbdb7c-1bc2-49b4-834c-de871be33524","Type":"ContainerDied","Data":"1d7beaa120a8808d12c28e8f6a2848000f03d35d7863fae917d05aa6a96bca72"}
Oct 31 17:43:14.045042 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:14.044348    4276 scope.go:122] "RemoveContainer" containerID="1019d46050d0bb7e6f4da0f693b5f99640f2050d06769321e7a4b15b04524aa5"
Oct 31 17:43:14.047382 ip-10-0-129-127 kubenswrapper[4276]: E1031 17:43:14.047355    4276 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1019d46050d0bb7e6f4da0f693b5f99640f2050d06769321e7a4b15b04524aa5\": container with ID starting with 1019d46050d0bb7e6f4da0f693b5f99640f2050d06769321e7a4b15b04524aa5 not found: ID does not exist" containerID="1019d46050d0bb7e6f4da0f693b5f99640f2050d06769321e7a4b15b04524aa5"
Oct 31 17:43:14.047505 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:14.047392    4276 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1019d46050d0bb7e6f4da0f693b5f99640f2050d06769321e7a4b15b04524aa5"} err="failed to get container status \"1019d46050d0bb7e6f4da0f693b5f99640f2050d06769321e7a4b15b04524aa5\": rpc error: code = NotFound desc = could not find container \"1019d46050d0bb7e6f4da0f693b5f99640f2050d06769321e7a4b15b04524aa5\": container with ID starting with 1019d46050d0bb7e6f4da0f693b5f99640f2050d06769321e7a4b15b04524aa5 not found: ID does not exist"
Oct 31 17:43:14.047505 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:14.047417    4276 scope.go:122] "RemoveContainer" containerID="09b8ee7e51fe351cc0bcc6f9d3dbcf11853e736da058d8d30cb89388f02fbd0c"
Oct 31 17:43:14.057235 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:14.057217    4276 scope.go:122] "RemoveContainer" containerID="7aec3c6fddcf595c3e7e91ec3662c167ee611c0ab99dc0d5a11652e39a6c6d26"
Oct 31 17:43:14.065850 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:14.065830    4276 scope.go:122] "RemoveContainer" containerID="09b8ee7e51fe351cc0bcc6f9d3dbcf11853e736da058d8d30cb89388f02fbd0c"
Oct 31 17:43:14.066104 ip-10-0-129-127 kubenswrapper[4276]: E1031 17:43:14.066082    4276 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"09b8ee7e51fe351cc0bcc6f9d3dbcf11853e736da058d8d30cb89388f02fbd0c\": container with ID starting with 09b8ee7e51fe351cc0bcc6f9d3dbcf11853e736da058d8d30cb89388f02fbd0c not found: ID does not exist" containerID="09b8ee7e51fe351cc0bcc6f9d3dbcf11853e736da058d8d30cb89388f02fbd0c"
Oct 31 17:43:14.066179 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:14.066114    4276 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"09b8ee7e51fe351cc0bcc6f9d3dbcf11853e736da058d8d30cb89388f02fbd0c"} err="failed to get container status \"09b8ee7e51fe351cc0bcc6f9d3dbcf11853e736da058d8d30cb89388f02fbd0c\": rpc error: code = NotFound desc = could not find container \"09b8ee7e51fe351cc0bcc6f9d3dbcf11853e736da058d8d30cb89388f02fbd0c\": container with ID starting with 09b8ee7e51fe351cc0bcc6f9d3dbcf11853e736da058d8d30cb89388f02fbd0c not found: ID does not exist"
Oct 31 17:43:14.066179 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:14.066135    4276 scope.go:122] "RemoveContainer" containerID="7aec3c6fddcf595c3e7e91ec3662c167ee611c0ab99dc0d5a11652e39a6c6d26"
Oct 31 17:43:14.066371 ip-10-0-129-127 kubenswrapper[4276]: E1031 17:43:14.066354    4276 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7aec3c6fddcf595c3e7e91ec3662c167ee611c0ab99dc0d5a11652e39a6c6d26\": container with ID starting with 7aec3c6fddcf595c3e7e91ec3662c167ee611c0ab99dc0d5a11652e39a6c6d26 not found: ID does not exist" containerID="7aec3c6fddcf595c3e7e91ec3662c167ee611c0ab99dc0d5a11652e39a6c6d26"
Oct 31 17:43:14.066437 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:14.066379    4276 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7aec3c6fddcf595c3e7e91ec3662c167ee611c0ab99dc0d5a11652e39a6c6d26"} err="failed to get container status \"7aec3c6fddcf595c3e7e91ec3662c167ee611c0ab99dc0d5a11652e39a6c6d26\": rpc error: code = NotFound desc = could not find container \"7aec3c6fddcf595c3e7e91ec3662c167ee611c0ab99dc0d5a11652e39a6c6d26\": container with ID starting with 7aec3c6fddcf595c3e7e91ec3662c167ee611c0ab99dc0d5a11652e39a6c6d26 not found: ID does not exist"
Oct 31 17:43:14.066437 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:14.066401    4276 scope.go:122] "RemoveContainer" containerID="395370f5dee6ae088f35afe2eeb8f02c629d745ac4ddf5cacb9129b95407042a"
Oct 31 17:43:14.075636 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:14.075617    4276 scope.go:122] "RemoveContainer" containerID="395370f5dee6ae088f35afe2eeb8f02c629d745ac4ddf5cacb9129b95407042a"
Oct 31 17:43:14.075884 ip-10-0-129-127 kubenswrapper[4276]: E1031 17:43:14.075868    4276 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"395370f5dee6ae088f35afe2eeb8f02c629d745ac4ddf5cacb9129b95407042a\": container with ID starting with 395370f5dee6ae088f35afe2eeb8f02c629d745ac4ddf5cacb9129b95407042a not found: ID does not exist" containerID="395370f5dee6ae088f35afe2eeb8f02c629d745ac4ddf5cacb9129b95407042a"
Oct 31 17:43:14.075971 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:14.075898    4276 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"395370f5dee6ae088f35afe2eeb8f02c629d745ac4ddf5cacb9129b95407042a"} err="failed to get container status \"395370f5dee6ae088f35afe2eeb8f02c629d745ac4ddf5cacb9129b95407042a\": rpc error: code = NotFound desc = could not find container \"395370f5dee6ae088f35afe2eeb8f02c629d745ac4ddf5cacb9129b95407042a\": container with ID starting with 395370f5dee6ae088f35afe2eeb8f02c629d745ac4ddf5cacb9129b95407042a not found: ID does not exist"
Oct 31 17:43:14.075971 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:14.075914    4276 scope.go:122] "RemoveContainer" containerID="1b62347ed05ecd31efa2e9cb7a621deeb263f006afe6411c319be80c5cd86a75"
Oct 31 17:43:14.083946 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:14.083929    4276 scope.go:122] "RemoveContainer" containerID="1b62347ed05ecd31efa2e9cb7a621deeb263f006afe6411c319be80c5cd86a75"
Oct 31 17:43:14.084206 ip-10-0-129-127 kubenswrapper[4276]: E1031 17:43:14.084190    4276 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1b62347ed05ecd31efa2e9cb7a621deeb263f006afe6411c319be80c5cd86a75\": container with ID starting with 1b62347ed05ecd31efa2e9cb7a621deeb263f006afe6411c319be80c5cd86a75 not found: ID does not exist" containerID="1b62347ed05ecd31efa2e9cb7a621deeb263f006afe6411c319be80c5cd86a75"
Oct 31 17:43:14.084254 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:14.084218    4276 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1b62347ed05ecd31efa2e9cb7a621deeb263f006afe6411c319be80c5cd86a75"} err="failed to get container status \"1b62347ed05ecd31efa2e9cb7a621deeb263f006afe6411c319be80c5cd86a75\": rpc error: code = NotFound desc = could not find container \"1b62347ed05ecd31efa2e9cb7a621deeb263f006afe6411c319be80c5cd86a75\": container with ID starting with 1b62347ed05ecd31efa2e9cb7a621deeb263f006afe6411c319be80c5cd86a75 not found: ID does not exist"
Oct 31 17:43:14.136356 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:14.136326    4276 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-8bf7k"]
Oct 31 17:43:14.139988 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:14.139967    4276 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-8bf7k"]
Oct 31 17:43:14.181958 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:14.181933    4276 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-m7vsj"]
Oct 31 17:43:14.188545 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:14.188522    4276 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-m7vsj"]
Oct 31 17:43:14.224274 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:14.224247    4276 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-ghwks"]
Oct 31 17:43:14.227718 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:14.227694    4276 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-ghwks"]
Oct 31 17:43:14.262956 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:14.262923    4276 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-qlhn8"]
Oct 31 17:43:14.268241 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:14.268217    4276 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-1c0bfd-openshift-default-f7f657c57-qlhn8"]
Oct 31 17:43:14.474835 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:14.474811    4276 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-kfkwb"
Oct 31 17:43:14.525849 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:14.525820    4276 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-m99sg"
Oct 31 17:43:14.540716 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:14.540693    4276 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-wm2s9"
Oct 31 17:43:14.614174 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:14.614139    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4b04b1b0-d2cc-4304-92e6-90e48105f442-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4b04b1b0-d2cc-4304-92e6-90e48105f442-istiod-ca-cert\") pod \"4b04b1b0-d2cc-4304-92e6-90e48105f442\" (UID: \"4b04b1b0-d2cc-4304-92e6-90e48105f442\") "
Oct 31 17:43:14.614472 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:14.614365    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/56c4ff3e-b2f2-4dce-938d-48672451d1fe-istio-token\" (UniqueName: \"kubernetes.io/projected/56c4ff3e-b2f2-4dce-938d-48672451d1fe-istio-token\") pod \"56c4ff3e-b2f2-4dce-938d-48672451d1fe\" (UID: \"56c4ff3e-b2f2-4dce-938d-48672451d1fe\") "
Oct 31 17:43:14.614472 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:14.614420    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4b04b1b0-d2cc-4304-92e6-90e48105f442-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4b04b1b0-d2cc-4304-92e6-90e48105f442-istio-data\") pod \"4b04b1b0-d2cc-4304-92e6-90e48105f442\" (UID: \"4b04b1b0-d2cc-4304-92e6-90e48105f442\") "
Oct 31 17:43:14.617111 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:14.614679    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/34243752-91ce-4e92-b0a6-72490b1f7c9e-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/34243752-91ce-4e92-b0a6-72490b1f7c9e-workload-socket\") pod \"34243752-91ce-4e92-b0a6-72490b1f7c9e\" (UID: \"34243752-91ce-4e92-b0a6-72490b1f7c9e\") "
Oct 31 17:43:14.617111 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:14.614728    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/56c4ff3e-b2f2-4dce-938d-48672451d1fe-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/56c4ff3e-b2f2-4dce-938d-48672451d1fe-credential-socket\") pod \"56c4ff3e-b2f2-4dce-938d-48672451d1fe\" (UID: \"56c4ff3e-b2f2-4dce-938d-48672451d1fe\") "
Oct 31 17:43:14.617111 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:14.614770    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/4b04b1b0-d2cc-4304-92e6-90e48105f442-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4b04b1b0-d2cc-4304-92e6-90e48105f442-istio-podinfo\") pod \"4b04b1b0-d2cc-4304-92e6-90e48105f442\" (UID: \"4b04b1b0-d2cc-4304-92e6-90e48105f442\") "
Oct 31 17:43:14.617111 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:14.614796    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4b04b1b0-d2cc-4304-92e6-90e48105f442-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4b04b1b0-d2cc-4304-92e6-90e48105f442-workload-certs\") pod \"4b04b1b0-d2cc-4304-92e6-90e48105f442\" (UID: \"4b04b1b0-d2cc-4304-92e6-90e48105f442\") "
Oct 31 17:43:14.617111 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:14.614840    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/34243752-91ce-4e92-b0a6-72490b1f7c9e-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/34243752-91ce-4e92-b0a6-72490b1f7c9e-istiod-ca-cert\") pod \"34243752-91ce-4e92-b0a6-72490b1f7c9e\" (UID: \"34243752-91ce-4e92-b0a6-72490b1f7c9e\") "
Oct 31 17:43:14.617111 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:14.614872    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/34243752-91ce-4e92-b0a6-72490b1f7c9e-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/34243752-91ce-4e92-b0a6-72490b1f7c9e-istio-data\") pod \"34243752-91ce-4e92-b0a6-72490b1f7c9e\" (UID: \"34243752-91ce-4e92-b0a6-72490b1f7c9e\") "
Oct 31 17:43:14.617111 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:14.614912    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4b04b1b0-d2cc-4304-92e6-90e48105f442-istio-token\" (UniqueName: \"kubernetes.io/projected/4b04b1b0-d2cc-4304-92e6-90e48105f442-istio-token\") pod \"4b04b1b0-d2cc-4304-92e6-90e48105f442\" (UID: \"4b04b1b0-d2cc-4304-92e6-90e48105f442\") "
Oct 31 17:43:14.617111 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:14.614988    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/56c4ff3e-b2f2-4dce-938d-48672451d1fe-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/56c4ff3e-b2f2-4dce-938d-48672451d1fe-istiod-ca-cert\") pod \"56c4ff3e-b2f2-4dce-938d-48672451d1fe\" (UID: \"56c4ff3e-b2f2-4dce-938d-48672451d1fe\") "
Oct 31 17:43:14.617111 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:14.615020    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/34243752-91ce-4e92-b0a6-72490b1f7c9e-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/34243752-91ce-4e92-b0a6-72490b1f7c9e-istio-envoy\") pod \"34243752-91ce-4e92-b0a6-72490b1f7c9e\" (UID: \"34243752-91ce-4e92-b0a6-72490b1f7c9e\") "
Oct 31 17:43:14.617111 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:14.615049    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/34243752-91ce-4e92-b0a6-72490b1f7c9e-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/34243752-91ce-4e92-b0a6-72490b1f7c9e-credential-socket\") pod \"34243752-91ce-4e92-b0a6-72490b1f7c9e\" (UID: \"34243752-91ce-4e92-b0a6-72490b1f7c9e\") "
Oct 31 17:43:14.617111 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:14.615076    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4b04b1b0-d2cc-4304-92e6-90e48105f442-kube-api-access-vv9w2\" (UniqueName: \"kubernetes.io/projected/4b04b1b0-d2cc-4304-92e6-90e48105f442-kube-api-access-vv9w2\") pod \"4b04b1b0-d2cc-4304-92e6-90e48105f442\" (UID: \"4b04b1b0-d2cc-4304-92e6-90e48105f442\") "
Oct 31 17:43:14.617111 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:14.615110    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/56c4ff3e-b2f2-4dce-938d-48672451d1fe-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/56c4ff3e-b2f2-4dce-938d-48672451d1fe-workload-certs\") pod \"56c4ff3e-b2f2-4dce-938d-48672451d1fe\" (UID: \"56c4ff3e-b2f2-4dce-938d-48672451d1fe\") "
Oct 31 17:43:14.617111 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:14.615135    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/56c4ff3e-b2f2-4dce-938d-48672451d1fe-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/56c4ff3e-b2f2-4dce-938d-48672451d1fe-istio-envoy\") pod \"56c4ff3e-b2f2-4dce-938d-48672451d1fe\" (UID: \"56c4ff3e-b2f2-4dce-938d-48672451d1fe\") "
Oct 31 17:43:14.617111 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:14.615179    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4b04b1b0-d2cc-4304-92e6-90e48105f442-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b04b1b0-d2cc-4304-92e6-90e48105f442-credential-socket\") pod \"4b04b1b0-d2cc-4304-92e6-90e48105f442\" (UID: \"4b04b1b0-d2cc-4304-92e6-90e48105f442\") "
Oct 31 17:43:14.617111 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:14.615207    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/34243752-91ce-4e92-b0a6-72490b1f7c9e-istio-token\" (UniqueName: \"kubernetes.io/projected/34243752-91ce-4e92-b0a6-72490b1f7c9e-istio-token\") pod \"34243752-91ce-4e92-b0a6-72490b1f7c9e\" (UID: \"34243752-91ce-4e92-b0a6-72490b1f7c9e\") "
Oct 31 17:43:14.617111 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:14.615241    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/56c4ff3e-b2f2-4dce-938d-48672451d1fe-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/56c4ff3e-b2f2-4dce-938d-48672451d1fe-istio-data\") pod \"56c4ff3e-b2f2-4dce-938d-48672451d1fe\" (UID: \"56c4ff3e-b2f2-4dce-938d-48672451d1fe\") "
Oct 31 17:43:14.617111 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:14.615285    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4b04b1b0-d2cc-4304-92e6-90e48105f442-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b04b1b0-d2cc-4304-92e6-90e48105f442-workload-socket\") pod \"4b04b1b0-d2cc-4304-92e6-90e48105f442\" (UID: \"4b04b1b0-d2cc-4304-92e6-90e48105f442\") "
Oct 31 17:43:14.617111 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:14.615332    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/56c4ff3e-b2f2-4dce-938d-48672451d1fe-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/56c4ff3e-b2f2-4dce-938d-48672451d1fe-istio-podinfo\") pod \"56c4ff3e-b2f2-4dce-938d-48672451d1fe\" (UID: \"56c4ff3e-b2f2-4dce-938d-48672451d1fe\") "
Oct 31 17:43:14.617111 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:14.615365    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/56c4ff3e-b2f2-4dce-938d-48672451d1fe-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/56c4ff3e-b2f2-4dce-938d-48672451d1fe-workload-socket\") pod \"56c4ff3e-b2f2-4dce-938d-48672451d1fe\" (UID: \"56c4ff3e-b2f2-4dce-938d-48672451d1fe\") "
Oct 31 17:43:14.617111 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:14.615415    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/34243752-91ce-4e92-b0a6-72490b1f7c9e-istio-data" pod "34243752-91ce-4e92-b0a6-72490b1f7c9e" (UID: "34243752-91ce-4e92-b0a6-72490b1f7c9e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:43:14.617111 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:14.615464    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/34243752-91ce-4e92-b0a6-72490b1f7c9e-kube-api-access-tzwwp\" (UniqueName: \"kubernetes.io/projected/34243752-91ce-4e92-b0a6-72490b1f7c9e-kube-api-access-tzwwp\") pod \"34243752-91ce-4e92-b0a6-72490b1f7c9e\" (UID: \"34243752-91ce-4e92-b0a6-72490b1f7c9e\") "
Oct 31 17:43:14.617111 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:14.615500    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4b04b1b0-d2cc-4304-92e6-90e48105f442-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4b04b1b0-d2cc-4304-92e6-90e48105f442-istio-envoy\") pod \"4b04b1b0-d2cc-4304-92e6-90e48105f442\" (UID: \"4b04b1b0-d2cc-4304-92e6-90e48105f442\") "
Oct 31 17:43:14.617111 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:14.615529    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/56c4ff3e-b2f2-4dce-938d-48672451d1fe-kube-api-access-gk7gj\" (UniqueName: \"kubernetes.io/projected/56c4ff3e-b2f2-4dce-938d-48672451d1fe-kube-api-access-gk7gj\") pod \"56c4ff3e-b2f2-4dce-938d-48672451d1fe\" (UID: \"56c4ff3e-b2f2-4dce-938d-48672451d1fe\") "
Oct 31 17:43:14.617111 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:14.615608    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/34243752-91ce-4e92-b0a6-72490b1f7c9e-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/34243752-91ce-4e92-b0a6-72490b1f7c9e-istio-podinfo\") pod \"34243752-91ce-4e92-b0a6-72490b1f7c9e\" (UID: \"34243752-91ce-4e92-b0a6-72490b1f7c9e\") "
Oct 31 17:43:14.617111 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:14.615625    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/56c4ff3e-b2f2-4dce-938d-48672451d1fe-credential-socket" pod "56c4ff3e-b2f2-4dce-938d-48672451d1fe" (UID: "56c4ff3e-b2f2-4dce-938d-48672451d1fe"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:43:14.617111 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:14.615641    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/34243752-91ce-4e92-b0a6-72490b1f7c9e-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/34243752-91ce-4e92-b0a6-72490b1f7c9e-workload-certs\") pod \"34243752-91ce-4e92-b0a6-72490b1f7c9e\" (UID: \"34243752-91ce-4e92-b0a6-72490b1f7c9e\") "
Oct 31 17:43:14.617111 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:14.615858    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b04b1b0-d2cc-4304-92e6-90e48105f442-istio-data" pod "4b04b1b0-d2cc-4304-92e6-90e48105f442" (UID: "4b04b1b0-d2cc-4304-92e6-90e48105f442"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:43:14.617111 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:14.616171    4276 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4b04b1b0-d2cc-4304-92e6-90e48105f442-istio-data\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:43:14.617111 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:14.616192    4276 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/56c4ff3e-b2f2-4dce-938d-48672451d1fe-credential-socket\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:43:14.617111 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:14.616500    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b04b1b0-d2cc-4304-92e6-90e48105f442-workload-certs" pod "4b04b1b0-d2cc-4304-92e6-90e48105f442" (UID: "4b04b1b0-d2cc-4304-92e6-90e48105f442"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:43:14.617111 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:14.616507    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/56c4ff3e-b2f2-4dce-938d-48672451d1fe-istio-data" pod "56c4ff3e-b2f2-4dce-938d-48672451d1fe" (UID: "56c4ff3e-b2f2-4dce-938d-48672451d1fe"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:43:14.617111 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:14.616725    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b04b1b0-d2cc-4304-92e6-90e48105f442-workload-socket" pod "4b04b1b0-d2cc-4304-92e6-90e48105f442" (UID: "4b04b1b0-d2cc-4304-92e6-90e48105f442"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:43:14.617111 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:14.616864    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/34243752-91ce-4e92-b0a6-72490b1f7c9e-workload-socket" pod "34243752-91ce-4e92-b0a6-72490b1f7c9e" (UID: "34243752-91ce-4e92-b0a6-72490b1f7c9e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:43:14.617111 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:14.616889    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/34243752-91ce-4e92-b0a6-72490b1f7c9e-istiod-ca-cert" pod "34243752-91ce-4e92-b0a6-72490b1f7c9e" (UID: "34243752-91ce-4e92-b0a6-72490b1f7c9e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 17:43:14.619987 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:14.617248    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4b04b1b0-d2cc-4304-92e6-90e48105f442-istiod-ca-cert" pod "4b04b1b0-d2cc-4304-92e6-90e48105f442" (UID: "4b04b1b0-d2cc-4304-92e6-90e48105f442"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 17:43:14.619987 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:14.617798    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/56c4ff3e-b2f2-4dce-938d-48672451d1fe-workload-certs" pod "56c4ff3e-b2f2-4dce-938d-48672451d1fe" (UID: "56c4ff3e-b2f2-4dce-938d-48672451d1fe"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:43:14.619987 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:14.617814    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/56c4ff3e-b2f2-4dce-938d-48672451d1fe-workload-socket" pod "56c4ff3e-b2f2-4dce-938d-48672451d1fe" (UID: "56c4ff3e-b2f2-4dce-938d-48672451d1fe"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:43:14.619987 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:14.617864    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/34243752-91ce-4e92-b0a6-72490b1f7c9e-credential-socket" pod "34243752-91ce-4e92-b0a6-72490b1f7c9e" (UID: "34243752-91ce-4e92-b0a6-72490b1f7c9e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:43:14.619987 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:14.618300    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/56c4ff3e-b2f2-4dce-938d-48672451d1fe-istiod-ca-cert" pod "56c4ff3e-b2f2-4dce-938d-48672451d1fe" (UID: "56c4ff3e-b2f2-4dce-938d-48672451d1fe"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 17:43:14.619987 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:14.618341    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b04b1b0-d2cc-4304-92e6-90e48105f442-credential-socket" pod "4b04b1b0-d2cc-4304-92e6-90e48105f442" (UID: "4b04b1b0-d2cc-4304-92e6-90e48105f442"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:43:14.619987 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:14.618390    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4b04b1b0-d2cc-4304-92e6-90e48105f442-istio-podinfo" pod "4b04b1b0-d2cc-4304-92e6-90e48105f442" (UID: "4b04b1b0-d2cc-4304-92e6-90e48105f442"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 17:43:14.620466 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:14.620411    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/34243752-91ce-4e92-b0a6-72490b1f7c9e-workload-certs" pod "34243752-91ce-4e92-b0a6-72490b1f7c9e" (UID: "34243752-91ce-4e92-b0a6-72490b1f7c9e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:43:14.620908 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:14.620803    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/34243752-91ce-4e92-b0a6-72490b1f7c9e-istio-token" pod "34243752-91ce-4e92-b0a6-72490b1f7c9e" (UID: "34243752-91ce-4e92-b0a6-72490b1f7c9e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 17:43:14.620908 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:14.620836    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/56c4ff3e-b2f2-4dce-938d-48672451d1fe-istio-token" pod "56c4ff3e-b2f2-4dce-938d-48672451d1fe" (UID: "56c4ff3e-b2f2-4dce-938d-48672451d1fe"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 17:43:14.621038 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:14.620917    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/56c4ff3e-b2f2-4dce-938d-48672451d1fe-istio-envoy" pod "56c4ff3e-b2f2-4dce-938d-48672451d1fe" (UID: "56c4ff3e-b2f2-4dce-938d-48672451d1fe"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:43:14.621483 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:14.621458    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/56c4ff3e-b2f2-4dce-938d-48672451d1fe-istio-podinfo" pod "56c4ff3e-b2f2-4dce-938d-48672451d1fe" (UID: "56c4ff3e-b2f2-4dce-938d-48672451d1fe"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 17:43:14.621618 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:14.621586    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4b04b1b0-d2cc-4304-92e6-90e48105f442-kube-api-access-vv9w2" pod "4b04b1b0-d2cc-4304-92e6-90e48105f442" (UID: "4b04b1b0-d2cc-4304-92e6-90e48105f442"). InnerVolumeSpecName "kube-api-access-vv9w2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 17:43:14.622146 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:14.622113    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/56c4ff3e-b2f2-4dce-938d-48672451d1fe-kube-api-access-gk7gj" pod "56c4ff3e-b2f2-4dce-938d-48672451d1fe" (UID: "56c4ff3e-b2f2-4dce-938d-48672451d1fe"). InnerVolumeSpecName "kube-api-access-gk7gj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 17:43:14.623629 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:14.623499    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/34243752-91ce-4e92-b0a6-72490b1f7c9e-istio-podinfo" pod "34243752-91ce-4e92-b0a6-72490b1f7c9e" (UID: "34243752-91ce-4e92-b0a6-72490b1f7c9e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 17:43:14.623629 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:14.623593    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b04b1b0-d2cc-4304-92e6-90e48105f442-istio-envoy" pod "4b04b1b0-d2cc-4304-92e6-90e48105f442" (UID: "4b04b1b0-d2cc-4304-92e6-90e48105f442"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:43:14.625496 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:14.625428    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/34243752-91ce-4e92-b0a6-72490b1f7c9e-kube-api-access-tzwwp" pod "34243752-91ce-4e92-b0a6-72490b1f7c9e" (UID: "34243752-91ce-4e92-b0a6-72490b1f7c9e"). InnerVolumeSpecName "kube-api-access-tzwwp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 17:43:14.626688 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:14.626667    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4b04b1b0-d2cc-4304-92e6-90e48105f442-istio-token" pod "4b04b1b0-d2cc-4304-92e6-90e48105f442" (UID: "4b04b1b0-d2cc-4304-92e6-90e48105f442"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 17:43:14.629328 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:14.629307    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/34243752-91ce-4e92-b0a6-72490b1f7c9e-istio-envoy" pod "34243752-91ce-4e92-b0a6-72490b1f7c9e" (UID: "34243752-91ce-4e92-b0a6-72490b1f7c9e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:43:14.716995 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:14.716915    4276 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/56c4ff3e-b2f2-4dce-938d-48672451d1fe-istio-podinfo\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:43:14.716995 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:14.716944    4276 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/56c4ff3e-b2f2-4dce-938d-48672451d1fe-workload-socket\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:43:14.716995 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:14.716958    4276 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tzwwp\" (UniqueName: \"kubernetes.io/projected/34243752-91ce-4e92-b0a6-72490b1f7c9e-kube-api-access-tzwwp\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:43:14.716995 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:14.716971    4276 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4b04b1b0-d2cc-4304-92e6-90e48105f442-istio-envoy\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:43:14.716995 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:14.716983    4276 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gk7gj\" (UniqueName: \"kubernetes.io/projected/56c4ff3e-b2f2-4dce-938d-48672451d1fe-kube-api-access-gk7gj\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:43:14.716995 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:14.716994    4276 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/34243752-91ce-4e92-b0a6-72490b1f7c9e-istio-podinfo\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:43:14.717343 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:14.717007    4276 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/34243752-91ce-4e92-b0a6-72490b1f7c9e-workload-certs\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:43:14.717343 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:14.717018    4276 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4b04b1b0-d2cc-4304-92e6-90e48105f442-istiod-ca-cert\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:43:14.717343 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:14.717028    4276 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/56c4ff3e-b2f2-4dce-938d-48672451d1fe-istio-token\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:43:14.717343 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:14.717038    4276 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/34243752-91ce-4e92-b0a6-72490b1f7c9e-workload-socket\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:43:14.717343 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:14.717051    4276 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4b04b1b0-d2cc-4304-92e6-90e48105f442-istio-podinfo\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:43:14.717343 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:14.717063    4276 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4b04b1b0-d2cc-4304-92e6-90e48105f442-workload-certs\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:43:14.717343 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:14.717074    4276 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/34243752-91ce-4e92-b0a6-72490b1f7c9e-istiod-ca-cert\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:43:14.717343 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:14.717086    4276 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/34243752-91ce-4e92-b0a6-72490b1f7c9e-istio-data\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:43:14.717343 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:14.717096    4276 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4b04b1b0-d2cc-4304-92e6-90e48105f442-istio-token\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:43:14.717343 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:14.717107    4276 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/56c4ff3e-b2f2-4dce-938d-48672451d1fe-istiod-ca-cert\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:43:14.717343 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:14.717120    4276 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/34243752-91ce-4e92-b0a6-72490b1f7c9e-istio-envoy\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:43:14.717343 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:14.717130    4276 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/34243752-91ce-4e92-b0a6-72490b1f7c9e-credential-socket\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:43:14.717343 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:14.717141    4276 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vv9w2\" (UniqueName: \"kubernetes.io/projected/4b04b1b0-d2cc-4304-92e6-90e48105f442-kube-api-access-vv9w2\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:43:14.717343 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:14.717153    4276 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/56c4ff3e-b2f2-4dce-938d-48672451d1fe-workload-certs\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:43:14.717343 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:14.717166    4276 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/56c4ff3e-b2f2-4dce-938d-48672451d1fe-istio-envoy\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:43:14.717343 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:14.717177    4276 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b04b1b0-d2cc-4304-92e6-90e48105f442-credential-socket\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:43:14.717343 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:14.717191    4276 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/34243752-91ce-4e92-b0a6-72490b1f7c9e-istio-token\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:43:14.717343 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:14.717205    4276 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/56c4ff3e-b2f2-4dce-938d-48672451d1fe-istio-data\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:43:14.717343 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:14.717217    4276 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b04b1b0-d2cc-4304-92e6-90e48105f442-workload-socket\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:43:14.801499 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:14.801464    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:43:15.043589 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:15.043485    4276 generic.go:356] "Generic (PLEG): container finished" podID="4b04b1b0-d2cc-4304-92e6-90e48105f442" containerID="a819b54f19dd2fb68538d2c781d08d3ab2856ca18c49b5f01808e5bc0c355a28" exitCode=0
Oct 31 17:43:15.043589 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:15.043540    4276 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-kfkwb"
Oct 31 17:43:15.043589 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:15.043572    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-kfkwb" event={"ID":"4b04b1b0-d2cc-4304-92e6-90e48105f442","Type":"ContainerDied","Data":"a819b54f19dd2fb68538d2c781d08d3ab2856ca18c49b5f01808e5bc0c355a28"}
Oct 31 17:43:15.043862 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:15.043651    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-kfkwb" event={"ID":"4b04b1b0-d2cc-4304-92e6-90e48105f442","Type":"ContainerDied","Data":"0ad611b8b13fdeb9160e042202344ebfba288506288598aea295199c24338850"}
Oct 31 17:43:15.043862 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:15.043681    4276 scope.go:122] "RemoveContainer" containerID="a819b54f19dd2fb68538d2c781d08d3ab2856ca18c49b5f01808e5bc0c355a28"
Oct 31 17:43:15.045410 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:15.045202    4276 generic.go:356] "Generic (PLEG): container finished" podID="56c4ff3e-b2f2-4dce-938d-48672451d1fe" containerID="4f62c3ff869ee3e1962a35435aba4b5f419648311c604b0634911e8968b4c93f" exitCode=0
Oct 31 17:43:15.045410 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:15.045258    4276 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-m99sg"
Oct 31 17:43:15.045410 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:15.045339    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-m99sg" event={"ID":"56c4ff3e-b2f2-4dce-938d-48672451d1fe","Type":"ContainerDied","Data":"4f62c3ff869ee3e1962a35435aba4b5f419648311c604b0634911e8968b4c93f"}
Oct 31 17:43:15.045410 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:15.045378    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-m99sg" event={"ID":"56c4ff3e-b2f2-4dce-938d-48672451d1fe","Type":"ContainerDied","Data":"66c58ec7b7bdcbc58d60926d1895953f0a02d105f87cac6d376071f1b2490915"}
Oct 31 17:43:15.049583 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:15.049559    4276 generic.go:356] "Generic (PLEG): container finished" podID="34243752-91ce-4e92-b0a6-72490b1f7c9e" containerID="89e2d0c4c006a4779f619c4ee4c592e212bcd6cf3946b5efe9b95513bc818853" exitCode=0
Oct 31 17:43:15.049675 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:15.049611    4276 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-wm2s9"
Oct 31 17:43:15.049732 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:15.049616    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-wm2s9" event={"ID":"34243752-91ce-4e92-b0a6-72490b1f7c9e","Type":"ContainerDied","Data":"89e2d0c4c006a4779f619c4ee4c592e212bcd6cf3946b5efe9b95513bc818853"}
Oct 31 17:43:15.049732 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:15.049724    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-wm2s9" event={"ID":"34243752-91ce-4e92-b0a6-72490b1f7c9e","Type":"ContainerDied","Data":"0a12fed32e7edd00e3ca31f1b5917fc70ded33e995e88ddaddaf85f4e779ba43"}
Oct 31 17:43:15.054890 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:15.054808    4276 scope.go:122] "RemoveContainer" containerID="a819b54f19dd2fb68538d2c781d08d3ab2856ca18c49b5f01808e5bc0c355a28"
Oct 31 17:43:15.055133 ip-10-0-129-127 kubenswrapper[4276]: E1031 17:43:15.055110    4276 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a819b54f19dd2fb68538d2c781d08d3ab2856ca18c49b5f01808e5bc0c355a28\": container with ID starting with a819b54f19dd2fb68538d2c781d08d3ab2856ca18c49b5f01808e5bc0c355a28 not found: ID does not exist" containerID="a819b54f19dd2fb68538d2c781d08d3ab2856ca18c49b5f01808e5bc0c355a28"
Oct 31 17:43:15.055189 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:15.055140    4276 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a819b54f19dd2fb68538d2c781d08d3ab2856ca18c49b5f01808e5bc0c355a28"} err="failed to get container status \"a819b54f19dd2fb68538d2c781d08d3ab2856ca18c49b5f01808e5bc0c355a28\": rpc error: code = NotFound desc = could not find container \"a819b54f19dd2fb68538d2c781d08d3ab2856ca18c49b5f01808e5bc0c355a28\": container with ID starting with a819b54f19dd2fb68538d2c781d08d3ab2856ca18c49b5f01808e5bc0c355a28 not found: ID does not exist"
Oct 31 17:43:15.055189 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:15.055156    4276 scope.go:122] "RemoveContainer" containerID="4f62c3ff869ee3e1962a35435aba4b5f419648311c604b0634911e8968b4c93f"
Oct 31 17:43:15.063866 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:15.063845    4276 scope.go:122] "RemoveContainer" containerID="4f62c3ff869ee3e1962a35435aba4b5f419648311c604b0634911e8968b4c93f"
Oct 31 17:43:15.064131 ip-10-0-129-127 kubenswrapper[4276]: E1031 17:43:15.064114    4276 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4f62c3ff869ee3e1962a35435aba4b5f419648311c604b0634911e8968b4c93f\": container with ID starting with 4f62c3ff869ee3e1962a35435aba4b5f419648311c604b0634911e8968b4c93f not found: ID does not exist" containerID="4f62c3ff869ee3e1962a35435aba4b5f419648311c604b0634911e8968b4c93f"
Oct 31 17:43:15.064199 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:15.064141    4276 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4f62c3ff869ee3e1962a35435aba4b5f419648311c604b0634911e8968b4c93f"} err="failed to get container status \"4f62c3ff869ee3e1962a35435aba4b5f419648311c604b0634911e8968b4c93f\": rpc error: code = NotFound desc = could not find container \"4f62c3ff869ee3e1962a35435aba4b5f419648311c604b0634911e8968b4c93f\": container with ID starting with 4f62c3ff869ee3e1962a35435aba4b5f419648311c604b0634911e8968b4c93f not found: ID does not exist"
Oct 31 17:43:15.064199 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:15.064168    4276 scope.go:122] "RemoveContainer" containerID="89e2d0c4c006a4779f619c4ee4c592e212bcd6cf3946b5efe9b95513bc818853"
Oct 31 17:43:15.072337 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:15.072320    4276 scope.go:122] "RemoveContainer" containerID="89e2d0c4c006a4779f619c4ee4c592e212bcd6cf3946b5efe9b95513bc818853"
Oct 31 17:43:15.072631 ip-10-0-129-127 kubenswrapper[4276]: E1031 17:43:15.072608    4276 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"89e2d0c4c006a4779f619c4ee4c592e212bcd6cf3946b5efe9b95513bc818853\": container with ID starting with 89e2d0c4c006a4779f619c4ee4c592e212bcd6cf3946b5efe9b95513bc818853 not found: ID does not exist" containerID="89e2d0c4c006a4779f619c4ee4c592e212bcd6cf3946b5efe9b95513bc818853"
Oct 31 17:43:15.072708 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:15.072640    4276 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"89e2d0c4c006a4779f619c4ee4c592e212bcd6cf3946b5efe9b95513bc818853"} err="failed to get container status \"89e2d0c4c006a4779f619c4ee4c592e212bcd6cf3946b5efe9b95513bc818853\": rpc error: code = NotFound desc = could not find container \"89e2d0c4c006a4779f619c4ee4c592e212bcd6cf3946b5efe9b95513bc818853\": container with ID starting with 89e2d0c4c006a4779f619c4ee4c592e212bcd6cf3946b5efe9b95513bc818853 not found: ID does not exist"
Oct 31 17:43:15.106428 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:15.106399    4276 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-m99sg"]
Oct 31 17:43:15.120302 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:15.120275    4276 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-m99sg"]
Oct 31 17:43:15.164291 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:15.164258    4276 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-kfkwb"]
Oct 31 17:43:15.168601 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:15.168571    4276 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-kfkwb"]
Oct 31 17:43:15.208015 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:15.207990    4276 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-wm2s9"]
Oct 31 17:43:15.211851 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:15.211829    4276 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-ee816e-openshift-default-c4db85b5c-wm2s9"]
Oct 31 17:43:15.366671 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:15.366644    4276 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="28cbdb7c-1bc2-49b4-834c-de871be33524" path="/var/lib/kubelet/pods/28cbdb7c-1bc2-49b4-834c-de871be33524/volumes"
Oct 31 17:43:15.367130 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:15.367115    4276 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="33671198-f448-4f99-9bc9-6a1ed6980c5b" path="/var/lib/kubelet/pods/33671198-f448-4f99-9bc9-6a1ed6980c5b/volumes"
Oct 31 17:43:15.367526 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:15.367514    4276 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="34243752-91ce-4e92-b0a6-72490b1f7c9e" path="/var/lib/kubelet/pods/34243752-91ce-4e92-b0a6-72490b1f7c9e/volumes"
Oct 31 17:43:15.368204 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:15.368187    4276 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4b04b1b0-d2cc-4304-92e6-90e48105f442" path="/var/lib/kubelet/pods/4b04b1b0-d2cc-4304-92e6-90e48105f442/volumes"
Oct 31 17:43:15.368599 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:15.368587    4276 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="56c4ff3e-b2f2-4dce-938d-48672451d1fe" path="/var/lib/kubelet/pods/56c4ff3e-b2f2-4dce-938d-48672451d1fe/volumes"
Oct 31 17:43:15.368964 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:15.368957    4276 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c3c40ebe-5cac-4981-913d-18f12d0bf439" path="/var/lib/kubelet/pods/c3c40ebe-5cac-4981-913d-18f12d0bf439/volumes"
Oct 31 17:43:15.369656 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:15.369588    4276 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f675ff71-27f9-44e8-aa32-89f4332d4c3e" path="/var/lib/kubelet/pods/f675ff71-27f9-44e8-aa32-89f4332d4c3e/volumes"
Oct 31 17:43:15.764708 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:15.764678    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:43:15.868077 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:15.868056    4276 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-g646p"
Oct 31 17:43:15.948519 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:15.947653    4276 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-25kdb"
Oct 31 17:43:15.957314 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:15.957291    4276 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-cfvtd"
Oct 31 17:43:15.960512 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:15.960492    4276 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-hgtdx"
Oct 31 17:43:16.029694 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:16.029669    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6ec6accb-4b2a-45a8-bab2-fe65f2f5df2d-istio-token\" (UniqueName: \"kubernetes.io/projected/6ec6accb-4b2a-45a8-bab2-fe65f2f5df2d-istio-token\") pod \"6ec6accb-4b2a-45a8-bab2-fe65f2f5df2d\" (UID: \"6ec6accb-4b2a-45a8-bab2-fe65f2f5df2d\") "
Oct 31 17:43:16.029872 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:16.029703    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a59e3d47-2d57-4bf7-9a6c-5584d2d58d63-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a59e3d47-2d57-4bf7-9a6c-5584d2d58d63-istio-data\") pod \"a59e3d47-2d57-4bf7-9a6c-5584d2d58d63\" (UID: \"a59e3d47-2d57-4bf7-9a6c-5584d2d58d63\") "
Oct 31 17:43:16.029872 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:16.029731    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/14900971-460e-4126-9089-c4b110d539ab-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/14900971-460e-4126-9089-c4b110d539ab-istio-podinfo\") pod \"14900971-460e-4126-9089-c4b110d539ab\" (UID: \"14900971-460e-4126-9089-c4b110d539ab\") "
Oct 31 17:43:16.030121 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:16.029761    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/bc5595c7-f428-4092-ac29-a30af2e01065-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bc5595c7-f428-4092-ac29-a30af2e01065-istio-podinfo\") pod \"bc5595c7-f428-4092-ac29-a30af2e01065\" (UID: \"bc5595c7-f428-4092-ac29-a30af2e01065\") "
Oct 31 17:43:16.030234 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:16.030159    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a59e3d47-2d57-4bf7-9a6c-5584d2d58d63-istio-token\" (UniqueName: \"kubernetes.io/projected/a59e3d47-2d57-4bf7-9a6c-5584d2d58d63-istio-token\") pod \"a59e3d47-2d57-4bf7-9a6c-5584d2d58d63\" (UID: \"a59e3d47-2d57-4bf7-9a6c-5584d2d58d63\") "
Oct 31 17:43:16.030234 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:16.030218    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bc5595c7-f428-4092-ac29-a30af2e01065-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bc5595c7-f428-4092-ac29-a30af2e01065-istio-envoy\") pod \"bc5595c7-f428-4092-ac29-a30af2e01065\" (UID: \"bc5595c7-f428-4092-ac29-a30af2e01065\") "
Oct 31 17:43:16.030349 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:16.030268    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/14900971-460e-4126-9089-c4b110d539ab-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/14900971-460e-4126-9089-c4b110d539ab-istiod-ca-cert\") pod \"14900971-460e-4126-9089-c4b110d539ab\" (UID: \"14900971-460e-4126-9089-c4b110d539ab\") "
Oct 31 17:43:16.030349 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:16.030312    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6ec6accb-4b2a-45a8-bab2-fe65f2f5df2d-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6ec6accb-4b2a-45a8-bab2-fe65f2f5df2d-istio-envoy\") pod \"6ec6accb-4b2a-45a8-bab2-fe65f2f5df2d\" (UID: \"6ec6accb-4b2a-45a8-bab2-fe65f2f5df2d\") "
Oct 31 17:43:16.030349 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:16.030341    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bc5595c7-f428-4092-ac29-a30af2e01065-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bc5595c7-f428-4092-ac29-a30af2e01065-workload-certs\") pod \"bc5595c7-f428-4092-ac29-a30af2e01065\" (UID: \"bc5595c7-f428-4092-ac29-a30af2e01065\") "
Oct 31 17:43:16.030515 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:16.030369    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a59e3d47-2d57-4bf7-9a6c-5584d2d58d63-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a59e3d47-2d57-4bf7-9a6c-5584d2d58d63-workload-socket\") pod \"a59e3d47-2d57-4bf7-9a6c-5584d2d58d63\" (UID: \"a59e3d47-2d57-4bf7-9a6c-5584d2d58d63\") "
Oct 31 17:43:16.030515 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:16.030455    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6ec6accb-4b2a-45a8-bab2-fe65f2f5df2d-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6ec6accb-4b2a-45a8-bab2-fe65f2f5df2d-istiod-ca-cert\") pod \"6ec6accb-4b2a-45a8-bab2-fe65f2f5df2d\" (UID: \"6ec6accb-4b2a-45a8-bab2-fe65f2f5df2d\") "
Oct 31 17:43:16.030515 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:16.030484    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a59e3d47-2d57-4bf7-9a6c-5584d2d58d63-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a59e3d47-2d57-4bf7-9a6c-5584d2d58d63-credential-socket\") pod \"a59e3d47-2d57-4bf7-9a6c-5584d2d58d63\" (UID: \"a59e3d47-2d57-4bf7-9a6c-5584d2d58d63\") "
Oct 31 17:43:16.030717 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:16.030514    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bc5595c7-f428-4092-ac29-a30af2e01065-istio-token\" (UniqueName: \"kubernetes.io/projected/bc5595c7-f428-4092-ac29-a30af2e01065-istio-token\") pod \"bc5595c7-f428-4092-ac29-a30af2e01065\" (UID: \"bc5595c7-f428-4092-ac29-a30af2e01065\") "
Oct 31 17:43:16.030717 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:16.030552    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bc5595c7-f428-4092-ac29-a30af2e01065-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bc5595c7-f428-4092-ac29-a30af2e01065-credential-socket\") pod \"bc5595c7-f428-4092-ac29-a30af2e01065\" (UID: \"bc5595c7-f428-4092-ac29-a30af2e01065\") "
Oct 31 17:43:16.030717 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:16.030583    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6ec6accb-4b2a-45a8-bab2-fe65f2f5df2d-kube-api-access-kp64p\" (UniqueName: \"kubernetes.io/projected/6ec6accb-4b2a-45a8-bab2-fe65f2f5df2d-kube-api-access-kp64p\") pod \"6ec6accb-4b2a-45a8-bab2-fe65f2f5df2d\" (UID: \"6ec6accb-4b2a-45a8-bab2-fe65f2f5df2d\") "
Oct 31 17:43:16.030717 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:16.030631    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/14900971-460e-4126-9089-c4b110d539ab-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/14900971-460e-4126-9089-c4b110d539ab-workload-certs\") pod \"14900971-460e-4126-9089-c4b110d539ab\" (UID: \"14900971-460e-4126-9089-c4b110d539ab\") "
Oct 31 17:43:16.030717 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:16.030681    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bc5595c7-f428-4092-ac29-a30af2e01065-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bc5595c7-f428-4092-ac29-a30af2e01065-istio-data\") pod \"bc5595c7-f428-4092-ac29-a30af2e01065\" (UID: \"bc5595c7-f428-4092-ac29-a30af2e01065\") "
Oct 31 17:43:16.030717 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:16.030713    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/14900971-460e-4126-9089-c4b110d539ab-istio-token\" (UniqueName: \"kubernetes.io/projected/14900971-460e-4126-9089-c4b110d539ab-istio-token\") pod \"14900971-460e-4126-9089-c4b110d539ab\" (UID: \"14900971-460e-4126-9089-c4b110d539ab\") "
Oct 31 17:43:16.031019 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:16.030744    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/14900971-460e-4126-9089-c4b110d539ab-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/14900971-460e-4126-9089-c4b110d539ab-workload-socket\") pod \"14900971-460e-4126-9089-c4b110d539ab\" (UID: \"14900971-460e-4126-9089-c4b110d539ab\") "
Oct 31 17:43:16.031019 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:16.030797    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bc5595c7-f428-4092-ac29-a30af2e01065-kube-api-access-6pxdr\" (UniqueName: \"kubernetes.io/projected/bc5595c7-f428-4092-ac29-a30af2e01065-kube-api-access-6pxdr\") pod \"bc5595c7-f428-4092-ac29-a30af2e01065\" (UID: \"bc5595c7-f428-4092-ac29-a30af2e01065\") "
Oct 31 17:43:16.031019 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:16.030827    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6ec6accb-4b2a-45a8-bab2-fe65f2f5df2d-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ec6accb-4b2a-45a8-bab2-fe65f2f5df2d-credential-socket\") pod \"6ec6accb-4b2a-45a8-bab2-fe65f2f5df2d\" (UID: \"6ec6accb-4b2a-45a8-bab2-fe65f2f5df2d\") "
Oct 31 17:43:16.031019 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:16.030866    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bc5595c7-f428-4092-ac29-a30af2e01065-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bc5595c7-f428-4092-ac29-a30af2e01065-workload-socket\") pod \"bc5595c7-f428-4092-ac29-a30af2e01065\" (UID: \"bc5595c7-f428-4092-ac29-a30af2e01065\") "
Oct 31 17:43:16.031019 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:16.030911    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a59e3d47-2d57-4bf7-9a6c-5584d2d58d63-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a59e3d47-2d57-4bf7-9a6c-5584d2d58d63-istiod-ca-cert\") pod \"a59e3d47-2d57-4bf7-9a6c-5584d2d58d63\" (UID: \"a59e3d47-2d57-4bf7-9a6c-5584d2d58d63\") "
Oct 31 17:43:16.031019 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:16.030956    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a59e3d47-2d57-4bf7-9a6c-5584d2d58d63-kube-api-access-v6ng4\" (UniqueName: \"kubernetes.io/projected/a59e3d47-2d57-4bf7-9a6c-5584d2d58d63-kube-api-access-v6ng4\") pod \"a59e3d47-2d57-4bf7-9a6c-5584d2d58d63\" (UID: \"a59e3d47-2d57-4bf7-9a6c-5584d2d58d63\") "
Oct 31 17:43:16.031019 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:16.030987    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/14900971-460e-4126-9089-c4b110d539ab-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/14900971-460e-4126-9089-c4b110d539ab-istio-envoy\") pod \"14900971-460e-4126-9089-c4b110d539ab\" (UID: \"14900971-460e-4126-9089-c4b110d539ab\") "
Oct 31 17:43:16.031346 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:16.031016    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6ec6accb-4b2a-45a8-bab2-fe65f2f5df2d-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ec6accb-4b2a-45a8-bab2-fe65f2f5df2d-workload-socket\") pod \"6ec6accb-4b2a-45a8-bab2-fe65f2f5df2d\" (UID: \"6ec6accb-4b2a-45a8-bab2-fe65f2f5df2d\") "
Oct 31 17:43:16.031346 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:16.031046    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a59e3d47-2d57-4bf7-9a6c-5584d2d58d63-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a59e3d47-2d57-4bf7-9a6c-5584d2d58d63-istio-envoy\") pod \"a59e3d47-2d57-4bf7-9a6c-5584d2d58d63\" (UID: \"a59e3d47-2d57-4bf7-9a6c-5584d2d58d63\") "
Oct 31 17:43:16.031346 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:16.031091    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a59e3d47-2d57-4bf7-9a6c-5584d2d58d63-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a59e3d47-2d57-4bf7-9a6c-5584d2d58d63-workload-certs\") pod \"a59e3d47-2d57-4bf7-9a6c-5584d2d58d63\" (UID: \"a59e3d47-2d57-4bf7-9a6c-5584d2d58d63\") "
Oct 31 17:43:16.031346 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:16.031140    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/6ec6accb-4b2a-45a8-bab2-fe65f2f5df2d-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6ec6accb-4b2a-45a8-bab2-fe65f2f5df2d-istio-podinfo\") pod \"6ec6accb-4b2a-45a8-bab2-fe65f2f5df2d\" (UID: \"6ec6accb-4b2a-45a8-bab2-fe65f2f5df2d\") "
Oct 31 17:43:16.031346 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:16.031171    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/bc5595c7-f428-4092-ac29-a30af2e01065-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bc5595c7-f428-4092-ac29-a30af2e01065-istiod-ca-cert\") pod \"bc5595c7-f428-4092-ac29-a30af2e01065\" (UID: \"bc5595c7-f428-4092-ac29-a30af2e01065\") "
Oct 31 17:43:16.031346 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:16.031208    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/14900971-460e-4126-9089-c4b110d539ab-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/14900971-460e-4126-9089-c4b110d539ab-credential-socket\") pod \"14900971-460e-4126-9089-c4b110d539ab\" (UID: \"14900971-460e-4126-9089-c4b110d539ab\") "
Oct 31 17:43:16.031346 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:16.031253    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/a59e3d47-2d57-4bf7-9a6c-5584d2d58d63-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a59e3d47-2d57-4bf7-9a6c-5584d2d58d63-istio-podinfo\") pod \"a59e3d47-2d57-4bf7-9a6c-5584d2d58d63\" (UID: \"a59e3d47-2d57-4bf7-9a6c-5584d2d58d63\") "
Oct 31 17:43:16.031346 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:16.031303    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/14900971-460e-4126-9089-c4b110d539ab-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/14900971-460e-4126-9089-c4b110d539ab-istio-data\") pod \"14900971-460e-4126-9089-c4b110d539ab\" (UID: \"14900971-460e-4126-9089-c4b110d539ab\") "
Oct 31 17:43:16.031346 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:16.031336    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/14900971-460e-4126-9089-c4b110d539ab-kube-api-access-p6jb8\" (UniqueName: \"kubernetes.io/projected/14900971-460e-4126-9089-c4b110d539ab-kube-api-access-p6jb8\") pod \"14900971-460e-4126-9089-c4b110d539ab\" (UID: \"14900971-460e-4126-9089-c4b110d539ab\") "
Oct 31 17:43:16.031780 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:16.031393    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6ec6accb-4b2a-45a8-bab2-fe65f2f5df2d-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6ec6accb-4b2a-45a8-bab2-fe65f2f5df2d-istio-data\") pod \"6ec6accb-4b2a-45a8-bab2-fe65f2f5df2d\" (UID: \"6ec6accb-4b2a-45a8-bab2-fe65f2f5df2d\") "
Oct 31 17:43:16.031780 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:16.031422    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6ec6accb-4b2a-45a8-bab2-fe65f2f5df2d-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6ec6accb-4b2a-45a8-bab2-fe65f2f5df2d-workload-certs\") pod \"6ec6accb-4b2a-45a8-bab2-fe65f2f5df2d\" (UID: \"6ec6accb-4b2a-45a8-bab2-fe65f2f5df2d\") "
Oct 31 17:43:16.034475 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:16.029986    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a59e3d47-2d57-4bf7-9a6c-5584d2d58d63-istio-data" pod "a59e3d47-2d57-4bf7-9a6c-5584d2d58d63" (UID: "a59e3d47-2d57-4bf7-9a6c-5584d2d58d63"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:43:16.034475 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:16.031871    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6ec6accb-4b2a-45a8-bab2-fe65f2f5df2d-istiod-ca-cert" pod "6ec6accb-4b2a-45a8-bab2-fe65f2f5df2d" (UID: "6ec6accb-4b2a-45a8-bab2-fe65f2f5df2d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 17:43:16.034475 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:16.032094    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6ec6accb-4b2a-45a8-bab2-fe65f2f5df2d-workload-certs" pod "6ec6accb-4b2a-45a8-bab2-fe65f2f5df2d" (UID: "6ec6accb-4b2a-45a8-bab2-fe65f2f5df2d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:43:16.034475 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:16.032209    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bc5595c7-f428-4092-ac29-a30af2e01065-workload-socket" pod "bc5595c7-f428-4092-ac29-a30af2e01065" (UID: "bc5595c7-f428-4092-ac29-a30af2e01065"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:43:16.034475 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:16.032220    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6ec6accb-4b2a-45a8-bab2-fe65f2f5df2d-credential-socket" pod "6ec6accb-4b2a-45a8-bab2-fe65f2f5df2d" (UID: "6ec6accb-4b2a-45a8-bab2-fe65f2f5df2d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:43:16.034475 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:16.034315    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/14900971-460e-4126-9089-c4b110d539ab-credential-socket" pod "14900971-460e-4126-9089-c4b110d539ab" (UID: "14900971-460e-4126-9089-c4b110d539ab"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:43:16.034475 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:16.032158    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bc5595c7-f428-4092-ac29-a30af2e01065-credential-socket" pod "bc5595c7-f428-4092-ac29-a30af2e01065" (UID: "bc5595c7-f428-4092-ac29-a30af2e01065"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:43:16.034475 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:16.032461    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a59e3d47-2d57-4bf7-9a6c-5584d2d58d63-istiod-ca-cert" pod "a59e3d47-2d57-4bf7-9a6c-5584d2d58d63" (UID: "a59e3d47-2d57-4bf7-9a6c-5584d2d58d63"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 17:43:16.034475 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:16.032714    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a59e3d47-2d57-4bf7-9a6c-5584d2d58d63-credential-socket" pod "a59e3d47-2d57-4bf7-9a6c-5584d2d58d63" (UID: "a59e3d47-2d57-4bf7-9a6c-5584d2d58d63"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:43:16.034475 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:16.032978    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bc5595c7-f428-4092-ac29-a30af2e01065-workload-certs" pod "bc5595c7-f428-4092-ac29-a30af2e01065" (UID: "bc5595c7-f428-4092-ac29-a30af2e01065"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:43:16.034475 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:16.033180    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a59e3d47-2d57-4bf7-9a6c-5584d2d58d63-workload-socket" pod "a59e3d47-2d57-4bf7-9a6c-5584d2d58d63" (UID: "a59e3d47-2d57-4bf7-9a6c-5584d2d58d63"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:43:16.034475 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:16.033236    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/14900971-460e-4126-9089-c4b110d539ab-istio-podinfo" pod "14900971-460e-4126-9089-c4b110d539ab" (UID: "14900971-460e-4126-9089-c4b110d539ab"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 17:43:16.034475 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:16.033272    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/bc5595c7-f428-4092-ac29-a30af2e01065-istio-podinfo" pod "bc5595c7-f428-4092-ac29-a30af2e01065" (UID: "bc5595c7-f428-4092-ac29-a30af2e01065"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 17:43:16.034475 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:16.033791    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/14900971-460e-4126-9089-c4b110d539ab-istiod-ca-cert" pod "14900971-460e-4126-9089-c4b110d539ab" (UID: "14900971-460e-4126-9089-c4b110d539ab"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 17:43:16.034475 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:16.033854    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/14900971-460e-4126-9089-c4b110d539ab-workload-socket" pod "14900971-460e-4126-9089-c4b110d539ab" (UID: "14900971-460e-4126-9089-c4b110d539ab"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:43:16.034475 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:16.034063    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/14900971-460e-4126-9089-c4b110d539ab-workload-certs" pod "14900971-460e-4126-9089-c4b110d539ab" (UID: "14900971-460e-4126-9089-c4b110d539ab"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:43:16.034475 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:16.034091    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bc5595c7-f428-4092-ac29-a30af2e01065-istiod-ca-cert" pod "bc5595c7-f428-4092-ac29-a30af2e01065" (UID: "bc5595c7-f428-4092-ac29-a30af2e01065"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 17:43:16.035371 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:16.034679    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bc5595c7-f428-4092-ac29-a30af2e01065-istio-data" pod "bc5595c7-f428-4092-ac29-a30af2e01065" (UID: "bc5595c7-f428-4092-ac29-a30af2e01065"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:43:16.035371 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:16.034848    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/14900971-460e-4126-9089-c4b110d539ab-istio-data" pod "14900971-460e-4126-9089-c4b110d539ab" (UID: "14900971-460e-4126-9089-c4b110d539ab"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:43:16.035371 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:16.034892    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6ec6accb-4b2a-45a8-bab2-fe65f2f5df2d-istio-data" pod "6ec6accb-4b2a-45a8-bab2-fe65f2f5df2d" (UID: "6ec6accb-4b2a-45a8-bab2-fe65f2f5df2d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:43:16.035371 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:16.034960    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6ec6accb-4b2a-45a8-bab2-fe65f2f5df2d-workload-socket" pod "6ec6accb-4b2a-45a8-bab2-fe65f2f5df2d" (UID: "6ec6accb-4b2a-45a8-bab2-fe65f2f5df2d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:43:16.035371 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:16.035237    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6ec6accb-4b2a-45a8-bab2-fe65f2f5df2d-kube-api-access-kp64p" pod "6ec6accb-4b2a-45a8-bab2-fe65f2f5df2d" (UID: "6ec6accb-4b2a-45a8-bab2-fe65f2f5df2d"). InnerVolumeSpecName "kube-api-access-kp64p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 17:43:16.035771 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:16.035746    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a59e3d47-2d57-4bf7-9a6c-5584d2d58d63-workload-certs" pod "a59e3d47-2d57-4bf7-9a6c-5584d2d58d63" (UID: "a59e3d47-2d57-4bf7-9a6c-5584d2d58d63"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:43:16.036262 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:16.036242    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bc5595c7-f428-4092-ac29-a30af2e01065-istio-token" pod "bc5595c7-f428-4092-ac29-a30af2e01065" (UID: "bc5595c7-f428-4092-ac29-a30af2e01065"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 17:43:16.036360 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:16.036270    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bc5595c7-f428-4092-ac29-a30af2e01065-kube-api-access-6pxdr" pod "bc5595c7-f428-4092-ac29-a30af2e01065" (UID: "bc5595c7-f428-4092-ac29-a30af2e01065"). InnerVolumeSpecName "kube-api-access-6pxdr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 17:43:16.036570 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:16.036551    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6ec6accb-4b2a-45a8-bab2-fe65f2f5df2d-istio-podinfo" pod "6ec6accb-4b2a-45a8-bab2-fe65f2f5df2d" (UID: "6ec6accb-4b2a-45a8-bab2-fe65f2f5df2d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 17:43:16.037239 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:16.037205    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a59e3d47-2d57-4bf7-9a6c-5584d2d58d63-kube-api-access-v6ng4" pod "a59e3d47-2d57-4bf7-9a6c-5584d2d58d63" (UID: "a59e3d47-2d57-4bf7-9a6c-5584d2d58d63"). InnerVolumeSpecName "kube-api-access-v6ng4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 17:43:16.037540 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:16.037485    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/14900971-460e-4126-9089-c4b110d539ab-istio-token" pod "14900971-460e-4126-9089-c4b110d539ab" (UID: "14900971-460e-4126-9089-c4b110d539ab"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 17:43:16.037642 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:16.037614    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/14900971-460e-4126-9089-c4b110d539ab-kube-api-access-p6jb8" pod "14900971-460e-4126-9089-c4b110d539ab" (UID: "14900971-460e-4126-9089-c4b110d539ab"). InnerVolumeSpecName "kube-api-access-p6jb8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 17:43:16.037866 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:16.037843    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a59e3d47-2d57-4bf7-9a6c-5584d2d58d63-istio-envoy" pod "a59e3d47-2d57-4bf7-9a6c-5584d2d58d63" (UID: "a59e3d47-2d57-4bf7-9a6c-5584d2d58d63"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:43:16.041005 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:16.040966    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a59e3d47-2d57-4bf7-9a6c-5584d2d58d63-istio-token" pod "a59e3d47-2d57-4bf7-9a6c-5584d2d58d63" (UID: "a59e3d47-2d57-4bf7-9a6c-5584d2d58d63"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 17:43:16.041270 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:16.041228    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bc5595c7-f428-4092-ac29-a30af2e01065-istio-envoy" pod "bc5595c7-f428-4092-ac29-a30af2e01065" (UID: "bc5595c7-f428-4092-ac29-a30af2e01065"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:43:16.041352 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:16.041284    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6ec6accb-4b2a-45a8-bab2-fe65f2f5df2d-istio-envoy" pod "6ec6accb-4b2a-45a8-bab2-fe65f2f5df2d" (UID: "6ec6accb-4b2a-45a8-bab2-fe65f2f5df2d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:43:16.041352 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:16.041340    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/14900971-460e-4126-9089-c4b110d539ab-istio-envoy" pod "14900971-460e-4126-9089-c4b110d539ab" (UID: "14900971-460e-4126-9089-c4b110d539ab"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:43:16.042023 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:16.041975    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6ec6accb-4b2a-45a8-bab2-fe65f2f5df2d-istio-token" pod "6ec6accb-4b2a-45a8-bab2-fe65f2f5df2d" (UID: "6ec6accb-4b2a-45a8-bab2-fe65f2f5df2d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 17:43:16.042661 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:16.042641    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a59e3d47-2d57-4bf7-9a6c-5584d2d58d63-istio-podinfo" pod "a59e3d47-2d57-4bf7-9a6c-5584d2d58d63" (UID: "a59e3d47-2d57-4bf7-9a6c-5584d2d58d63"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 17:43:16.058702 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:16.058677    4276 generic.go:356] "Generic (PLEG): container finished" podID="a59e3d47-2d57-4bf7-9a6c-5584d2d58d63" containerID="c11a9fcac7492f02a86e3e250a3a1b0c7b66682168c63268a347b1c9db01093e" exitCode=0
Oct 31 17:43:16.058817 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:16.058743    4276 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-hgtdx"
Oct 31 17:43:16.058817 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:16.058765    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-hgtdx" event={"ID":"a59e3d47-2d57-4bf7-9a6c-5584d2d58d63","Type":"ContainerDied","Data":"c11a9fcac7492f02a86e3e250a3a1b0c7b66682168c63268a347b1c9db01093e"}
Oct 31 17:43:16.058938 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:16.058859    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-hgtdx" event={"ID":"a59e3d47-2d57-4bf7-9a6c-5584d2d58d63","Type":"ContainerDied","Data":"8509102b76b002a2836ee593e236405c1e58aef0590999ed688a837d48ea1f60"}
Oct 31 17:43:16.058938 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:16.058896    4276 scope.go:122] "RemoveContainer" containerID="c11a9fcac7492f02a86e3e250a3a1b0c7b66682168c63268a347b1c9db01093e"
Oct 31 17:43:16.062018 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:16.061993    4276 generic.go:356] "Generic (PLEG): container finished" podID="bc5595c7-f428-4092-ac29-a30af2e01065" containerID="94875b822b89638cdb029172564865300c6eabb87124433d1493ab36ff0356d6" exitCode=0
Oct 31 17:43:16.062133 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:16.062063    4276 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-cfvtd"
Oct 31 17:43:16.062133 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:16.062079    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-cfvtd" event={"ID":"bc5595c7-f428-4092-ac29-a30af2e01065","Type":"ContainerDied","Data":"94875b822b89638cdb029172564865300c6eabb87124433d1493ab36ff0356d6"}
Oct 31 17:43:16.062133 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:16.062130    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-cfvtd" event={"ID":"bc5595c7-f428-4092-ac29-a30af2e01065","Type":"ContainerDied","Data":"f63c2f4768c0c357bd9e08a6a9f96e198508803704ee7a6f08e71db8e2c5c7b3"}
Oct 31 17:43:16.063652 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:16.063631    4276 generic.go:356] "Generic (PLEG): container finished" podID="6ec6accb-4b2a-45a8-bab2-fe65f2f5df2d" containerID="a1287b2dab22e7f754ce2563530796e068f02327c3f3a8993b891d470489e006" exitCode=0
Oct 31 17:43:16.063759 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:16.063680    4276 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-25kdb"
Oct 31 17:43:16.063759 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:16.063708    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-25kdb" event={"ID":"6ec6accb-4b2a-45a8-bab2-fe65f2f5df2d","Type":"ContainerDied","Data":"a1287b2dab22e7f754ce2563530796e068f02327c3f3a8993b891d470489e006"}
Oct 31 17:43:16.063759 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:16.063744    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-25kdb" event={"ID":"6ec6accb-4b2a-45a8-bab2-fe65f2f5df2d","Type":"ContainerDied","Data":"c7b22e980dc77eb919d97c899ab99174f89c58c05a99a1385ff91d76d8886cd1"}
Oct 31 17:43:16.074434 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:16.069681    4276 generic.go:356] "Generic (PLEG): container finished" podID="14900971-460e-4126-9089-c4b110d539ab" containerID="661fccd383b68b0099aeec974f6cc1dd943864f8e2e0df4870d8bcd831a23a1a" exitCode=0
Oct 31 17:43:16.074434 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:16.069785    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-g646p" event={"ID":"14900971-460e-4126-9089-c4b110d539ab","Type":"ContainerDied","Data":"661fccd383b68b0099aeec974f6cc1dd943864f8e2e0df4870d8bcd831a23a1a"}
Oct 31 17:43:16.074434 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:16.069821    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-g646p" event={"ID":"14900971-460e-4126-9089-c4b110d539ab","Type":"ContainerDied","Data":"1b2bf4b873e4dbe5f87f6511810d902f3434a7072e5690bf22601c643ad47228"}
Oct 31 17:43:16.074434 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:16.069895    4276 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-g646p"
Oct 31 17:43:16.074434 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:16.071591    4276 scope.go:122] "RemoveContainer" containerID="c11a9fcac7492f02a86e3e250a3a1b0c7b66682168c63268a347b1c9db01093e"
Oct 31 17:43:16.075495 ip-10-0-129-127 kubenswrapper[4276]: E1031 17:43:16.075464    4276 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c11a9fcac7492f02a86e3e250a3a1b0c7b66682168c63268a347b1c9db01093e\": container with ID starting with c11a9fcac7492f02a86e3e250a3a1b0c7b66682168c63268a347b1c9db01093e not found: ID does not exist" containerID="c11a9fcac7492f02a86e3e250a3a1b0c7b66682168c63268a347b1c9db01093e"
Oct 31 17:43:16.075593 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:16.075506    4276 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c11a9fcac7492f02a86e3e250a3a1b0c7b66682168c63268a347b1c9db01093e"} err="failed to get container status \"c11a9fcac7492f02a86e3e250a3a1b0c7b66682168c63268a347b1c9db01093e\": rpc error: code = NotFound desc = could not find container \"c11a9fcac7492f02a86e3e250a3a1b0c7b66682168c63268a347b1c9db01093e\": container with ID starting with c11a9fcac7492f02a86e3e250a3a1b0c7b66682168c63268a347b1c9db01093e not found: ID does not exist"
Oct 31 17:43:16.075593 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:16.075530    4276 scope.go:122] "RemoveContainer" containerID="94875b822b89638cdb029172564865300c6eabb87124433d1493ab36ff0356d6"
Oct 31 17:43:16.086309 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:16.086293    4276 scope.go:122] "RemoveContainer" containerID="94875b822b89638cdb029172564865300c6eabb87124433d1493ab36ff0356d6"
Oct 31 17:43:16.086708 ip-10-0-129-127 kubenswrapper[4276]: E1031 17:43:16.086679    4276 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"94875b822b89638cdb029172564865300c6eabb87124433d1493ab36ff0356d6\": container with ID starting with 94875b822b89638cdb029172564865300c6eabb87124433d1493ab36ff0356d6 not found: ID does not exist" containerID="94875b822b89638cdb029172564865300c6eabb87124433d1493ab36ff0356d6"
Oct 31 17:43:16.086856 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:16.086717    4276 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"94875b822b89638cdb029172564865300c6eabb87124433d1493ab36ff0356d6"} err="failed to get container status \"94875b822b89638cdb029172564865300c6eabb87124433d1493ab36ff0356d6\": rpc error: code = NotFound desc = could not find container \"94875b822b89638cdb029172564865300c6eabb87124433d1493ab36ff0356d6\": container with ID starting with 94875b822b89638cdb029172564865300c6eabb87124433d1493ab36ff0356d6 not found: ID does not exist"
Oct 31 17:43:16.086856 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:16.086760    4276 scope.go:122] "RemoveContainer" containerID="a1287b2dab22e7f754ce2563530796e068f02327c3f3a8993b891d470489e006"
Oct 31 17:43:16.099298 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:16.099269    4276 scope.go:122] "RemoveContainer" containerID="a1287b2dab22e7f754ce2563530796e068f02327c3f3a8993b891d470489e006"
Oct 31 17:43:16.099687 ip-10-0-129-127 kubenswrapper[4276]: E1031 17:43:16.099662    4276 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a1287b2dab22e7f754ce2563530796e068f02327c3f3a8993b891d470489e006\": container with ID starting with a1287b2dab22e7f754ce2563530796e068f02327c3f3a8993b891d470489e006 not found: ID does not exist" containerID="a1287b2dab22e7f754ce2563530796e068f02327c3f3a8993b891d470489e006"
Oct 31 17:43:16.099763 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:16.099694    4276 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a1287b2dab22e7f754ce2563530796e068f02327c3f3a8993b891d470489e006"} err="failed to get container status \"a1287b2dab22e7f754ce2563530796e068f02327c3f3a8993b891d470489e006\": rpc error: code = NotFound desc = could not find container \"a1287b2dab22e7f754ce2563530796e068f02327c3f3a8993b891d470489e006\": container with ID starting with a1287b2dab22e7f754ce2563530796e068f02327c3f3a8993b891d470489e006 not found: ID does not exist"
Oct 31 17:43:16.099763 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:16.099712    4276 scope.go:122] "RemoveContainer" containerID="661fccd383b68b0099aeec974f6cc1dd943864f8e2e0df4870d8bcd831a23a1a"
Oct 31 17:43:16.109178 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:16.109090    4276 scope.go:122] "RemoveContainer" containerID="661fccd383b68b0099aeec974f6cc1dd943864f8e2e0df4870d8bcd831a23a1a"
Oct 31 17:43:16.109485 ip-10-0-129-127 kubenswrapper[4276]: E1031 17:43:16.109435    4276 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"661fccd383b68b0099aeec974f6cc1dd943864f8e2e0df4870d8bcd831a23a1a\": container with ID starting with 661fccd383b68b0099aeec974f6cc1dd943864f8e2e0df4870d8bcd831a23a1a not found: ID does not exist" containerID="661fccd383b68b0099aeec974f6cc1dd943864f8e2e0df4870d8bcd831a23a1a"
Oct 31 17:43:16.109575 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:16.109495    4276 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"661fccd383b68b0099aeec974f6cc1dd943864f8e2e0df4870d8bcd831a23a1a"} err="failed to get container status \"661fccd383b68b0099aeec974f6cc1dd943864f8e2e0df4870d8bcd831a23a1a\": rpc error: code = NotFound desc = could not find container \"661fccd383b68b0099aeec974f6cc1dd943864f8e2e0df4870d8bcd831a23a1a\": container with ID starting with 661fccd383b68b0099aeec974f6cc1dd943864f8e2e0df4870d8bcd831a23a1a not found: ID does not exist"
Oct 31 17:43:16.132372 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:16.132349    4276 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-25kdb"]
Oct 31 17:43:16.132798 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:16.132781    4276 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6ec6accb-4b2a-45a8-bab2-fe65f2f5df2d-istio-podinfo\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:43:16.132839 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:16.132802    4276 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bc5595c7-f428-4092-ac29-a30af2e01065-istiod-ca-cert\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:43:16.132839 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:16.132811    4276 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/14900971-460e-4126-9089-c4b110d539ab-credential-socket\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:43:16.132839 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:16.132830    4276 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a59e3d47-2d57-4bf7-9a6c-5584d2d58d63-istio-podinfo\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:43:16.132932 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:16.132839    4276 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/14900971-460e-4126-9089-c4b110d539ab-istio-data\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:43:16.132932 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:16.132848    4276 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p6jb8\" (UniqueName: \"kubernetes.io/projected/14900971-460e-4126-9089-c4b110d539ab-kube-api-access-p6jb8\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:43:16.132932 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:16.132857    4276 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6ec6accb-4b2a-45a8-bab2-fe65f2f5df2d-istio-data\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:43:16.132932 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:16.132866    4276 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6ec6accb-4b2a-45a8-bab2-fe65f2f5df2d-workload-certs\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:43:16.132932 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:16.132873    4276 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6ec6accb-4b2a-45a8-bab2-fe65f2f5df2d-istio-token\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:43:16.132932 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:16.132880    4276 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a59e3d47-2d57-4bf7-9a6c-5584d2d58d63-istio-data\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:43:16.132932 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:16.132887    4276 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/14900971-460e-4126-9089-c4b110d539ab-istio-podinfo\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:43:16.132932 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:16.132894    4276 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bc5595c7-f428-4092-ac29-a30af2e01065-istio-podinfo\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:43:16.132932 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:16.132902    4276 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a59e3d47-2d57-4bf7-9a6c-5584d2d58d63-istio-token\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:43:16.132932 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:16.132909    4276 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bc5595c7-f428-4092-ac29-a30af2e01065-istio-envoy\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:43:16.132932 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:16.132916    4276 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/14900971-460e-4126-9089-c4b110d539ab-istiod-ca-cert\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:43:16.132932 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:16.132924    4276 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6ec6accb-4b2a-45a8-bab2-fe65f2f5df2d-istio-envoy\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:43:16.132932 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:16.132932    4276 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bc5595c7-f428-4092-ac29-a30af2e01065-workload-certs\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:43:16.133427 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:16.132940    4276 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a59e3d47-2d57-4bf7-9a6c-5584d2d58d63-workload-socket\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:43:16.133427 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:16.132948    4276 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6ec6accb-4b2a-45a8-bab2-fe65f2f5df2d-istiod-ca-cert\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:43:16.133427 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:16.132957    4276 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a59e3d47-2d57-4bf7-9a6c-5584d2d58d63-credential-socket\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:43:16.133427 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:16.132965    4276 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bc5595c7-f428-4092-ac29-a30af2e01065-istio-token\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:43:16.133427 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:16.132974    4276 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bc5595c7-f428-4092-ac29-a30af2e01065-credential-socket\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:43:16.133427 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:16.132982    4276 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kp64p\" (UniqueName: \"kubernetes.io/projected/6ec6accb-4b2a-45a8-bab2-fe65f2f5df2d-kube-api-access-kp64p\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:43:16.133427 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:16.132991    4276 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/14900971-460e-4126-9089-c4b110d539ab-workload-certs\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:43:16.133427 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:16.132998    4276 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bc5595c7-f428-4092-ac29-a30af2e01065-istio-data\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:43:16.133427 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:16.133006    4276 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/14900971-460e-4126-9089-c4b110d539ab-istio-token\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:43:16.133427 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:16.133013    4276 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/14900971-460e-4126-9089-c4b110d539ab-workload-socket\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:43:16.133427 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:16.133020    4276 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6pxdr\" (UniqueName: \"kubernetes.io/projected/bc5595c7-f428-4092-ac29-a30af2e01065-kube-api-access-6pxdr\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:43:16.133427 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:16.133029    4276 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ec6accb-4b2a-45a8-bab2-fe65f2f5df2d-credential-socket\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:43:16.133427 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:16.133036    4276 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bc5595c7-f428-4092-ac29-a30af2e01065-workload-socket\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:43:16.133427 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:16.133064    4276 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a59e3d47-2d57-4bf7-9a6c-5584d2d58d63-istiod-ca-cert\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:43:16.133427 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:16.133072    4276 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v6ng4\" (UniqueName: \"kubernetes.io/projected/a59e3d47-2d57-4bf7-9a6c-5584d2d58d63-kube-api-access-v6ng4\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:43:16.133427 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:16.133080    4276 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/14900971-460e-4126-9089-c4b110d539ab-istio-envoy\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:43:16.133427 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:16.133092    4276 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ec6accb-4b2a-45a8-bab2-fe65f2f5df2d-workload-socket\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:43:16.133427 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:16.133100    4276 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a59e3d47-2d57-4bf7-9a6c-5584d2d58d63-istio-envoy\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:43:16.133427 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:16.133108    4276 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a59e3d47-2d57-4bf7-9a6c-5584d2d58d63-workload-certs\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:43:16.138283 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:16.138265    4276 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-25kdb"]
Oct 31 17:43:16.173530 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:16.173510    4276 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-cfvtd"]
Oct 31 17:43:16.179304 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:16.179285    4276 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-cfvtd"]
Oct 31 17:43:16.217797 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:16.217743    4276 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-hgtdx"]
Oct 31 17:43:16.224857 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:16.224838    4276 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-hgtdx"]
Oct 31 17:43:16.260212 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:16.260182    4276 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-g646p"]
Oct 31 17:43:16.262551 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:16.262530    4276 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-d80080-openshift-default-956cddf57-g646p"]
Oct 31 17:43:17.056903 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:17.056876    4276 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-58b696d4f-bg2ch"]
Oct 31 17:43:17.057253 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:17.057242    4276 cpu_manager.go:414] "containerMap: removing container" podUID="33671198-f448-4f99-9bc9-6a1ed6980c5b" containerName="istio-proxy"
Oct 31 17:43:17.057290 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:17.057253    4276 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="33671198-f448-4f99-9bc9-6a1ed6980c5b" containerName="istio-proxy"
Oct 31 17:43:17.057290 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:17.057262    4276 cpu_manager.go:414] "containerMap: removing container" podUID="4c3cb076-d13d-4212-8fbb-86c851b14fe7" containerName="istio-proxy"
Oct 31 17:43:17.057290 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:17.057268    4276 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4c3cb076-d13d-4212-8fbb-86c851b14fe7" containerName="istio-proxy"
Oct 31 17:43:17.057290 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:17.057277    4276 cpu_manager.go:414] "containerMap: removing container" podUID="79a9c2b2-8458-4a74-afde-ee00374beb5b" containerName="istio-proxy"
Oct 31 17:43:17.057290 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:17.057282    4276 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="79a9c2b2-8458-4a74-afde-ee00374beb5b" containerName="istio-proxy"
Oct 31 17:43:17.057471 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:17.057291    4276 cpu_manager.go:414] "containerMap: removing container" podUID="b2b7f15e-756a-40e6-90af-961551719a7d" containerName="istio-proxy"
Oct 31 17:43:17.057471 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:17.057296    4276 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b2b7f15e-756a-40e6-90af-961551719a7d" containerName="istio-proxy"
Oct 31 17:43:17.057471 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:17.057303    4276 cpu_manager.go:414] "containerMap: removing container" podUID="14900971-460e-4126-9089-c4b110d539ab" containerName="istio-proxy"
Oct 31 17:43:17.057471 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:17.057311    4276 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="14900971-460e-4126-9089-c4b110d539ab" containerName="istio-proxy"
Oct 31 17:43:17.057471 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:17.057321    4276 cpu_manager.go:414] "containerMap: removing container" podUID="b9c2f445-a191-4b00-802a-1b0ded068411" containerName="main"
Oct 31 17:43:17.057471 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:17.057325    4276 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b9c2f445-a191-4b00-802a-1b0ded068411" containerName="main"
Oct 31 17:43:17.057471 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:17.057336    4276 cpu_manager.go:414] "containerMap: removing container" podUID="6e4109e9-1a59-48c7-9143-e14a01758994" containerName="payload-pre-processing"
Oct 31 17:43:17.057471 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:17.057340    4276 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6e4109e9-1a59-48c7-9143-e14a01758994" containerName="payload-pre-processing"
Oct 31 17:43:17.057471 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:17.057350    4276 cpu_manager.go:414] "containerMap: removing container" podUID="95755676-9f02-4b03-a7e5-f6c8ab4fa965" containerName="maas-api"
Oct 31 17:43:17.057471 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:17.057355    4276 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="95755676-9f02-4b03-a7e5-f6c8ab4fa965" containerName="maas-api"
Oct 31 17:43:17.057471 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:17.057362    4276 cpu_manager.go:414] "containerMap: removing container" podUID="6418b389-728b-4040-9b73-50cf013c00f3" containerName="istio-proxy"
Oct 31 17:43:17.057471 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:17.057366    4276 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6418b389-728b-4040-9b73-50cf013c00f3" containerName="istio-proxy"
Oct 31 17:43:17.057471 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:17.057373    4276 cpu_manager.go:414] "containerMap: removing container" podUID="34243752-91ce-4e92-b0a6-72490b1f7c9e" containerName="istio-proxy"
Oct 31 17:43:17.057471 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:17.057377    4276 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="34243752-91ce-4e92-b0a6-72490b1f7c9e" containerName="istio-proxy"
Oct 31 17:43:17.057471 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:17.057385    4276 cpu_manager.go:414] "containerMap: removing container" podUID="4b04b1b0-d2cc-4304-92e6-90e48105f442" containerName="istio-proxy"
Oct 31 17:43:17.057471 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:17.057392    4276 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4b04b1b0-d2cc-4304-92e6-90e48105f442" containerName="istio-proxy"
Oct 31 17:43:17.057471 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:17.057404    4276 cpu_manager.go:414] "containerMap: removing container" podUID="3716260a-fa38-4064-991c-48322b1b48fa" containerName="istio-proxy"
Oct 31 17:43:17.057471 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:17.057409    4276 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3716260a-fa38-4064-991c-48322b1b48fa" containerName="istio-proxy"
Oct 31 17:43:17.057471 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:17.057417    4276 cpu_manager.go:414] "containerMap: removing container" podUID="6418b389-728b-4040-9b73-50cf013c00f3" containerName="istio-proxy"
Oct 31 17:43:17.057471 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:17.057421    4276 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6418b389-728b-4040-9b73-50cf013c00f3" containerName="istio-proxy"
Oct 31 17:43:17.057471 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:17.057428    4276 cpu_manager.go:414] "containerMap: removing container" podUID="6ec6accb-4b2a-45a8-bab2-fe65f2f5df2d" containerName="istio-proxy"
Oct 31 17:43:17.057471 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:17.057432    4276 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6ec6accb-4b2a-45a8-bab2-fe65f2f5df2d" containerName="istio-proxy"
Oct 31 17:43:17.057471 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:17.057438    4276 cpu_manager.go:414] "containerMap: removing container" podUID="f675ff71-27f9-44e8-aa32-89f4332d4c3e" containerName="istio-proxy"
Oct 31 17:43:17.057471 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:17.057457    4276 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f675ff71-27f9-44e8-aa32-89f4332d4c3e" containerName="istio-proxy"
Oct 31 17:43:17.057471 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:17.057468    4276 cpu_manager.go:414] "containerMap: removing container" podUID="653b8145-5630-4b28-b3c8-624fb92032e3" containerName="revoke-keys"
Oct 31 17:43:17.057471 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:17.057476    4276 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="653b8145-5630-4b28-b3c8-624fb92032e3" containerName="revoke-keys"
Oct 31 17:43:17.057471 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:17.057483    4276 cpu_manager.go:414] "containerMap: removing container" podUID="d46713a5-7bba-4dfb-b77f-8fae0b39271e" containerName="istio-proxy"
Oct 31 17:43:17.057471 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:17.057487    4276 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d46713a5-7bba-4dfb-b77f-8fae0b39271e" containerName="istio-proxy"
Oct 31 17:43:17.058207 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:17.057495    4276 cpu_manager.go:414] "containerMap: removing container" podUID="e948a32e-25a4-4d29-b9dd-01a0539d337c" containerName="istio-proxy"
Oct 31 17:43:17.058207 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:17.057499    4276 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e948a32e-25a4-4d29-b9dd-01a0539d337c" containerName="istio-proxy"
Oct 31 17:43:17.058207 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:17.057505    4276 cpu_manager.go:414] "containerMap: removing container" podUID="9c4606cd-947c-4226-8152-74f8870458e6" containerName="revoke-keys"
Oct 31 17:43:17.058207 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:17.057509    4276 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9c4606cd-947c-4226-8152-74f8870458e6" containerName="revoke-keys"
Oct 31 17:43:17.058207 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:17.057518    4276 cpu_manager.go:414] "containerMap: removing container" podUID="175639e5-138e-428e-bb53-7fe484543496" containerName="maas-api"
Oct 31 17:43:17.058207 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:17.057522    4276 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="175639e5-138e-428e-bb53-7fe484543496" containerName="maas-api"
Oct 31 17:43:17.058207 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:17.057528    4276 cpu_manager.go:414] "containerMap: removing container" podUID="56c4ff3e-b2f2-4dce-938d-48672451d1fe" containerName="istio-proxy"
Oct 31 17:43:17.058207 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:17.057532    4276 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="56c4ff3e-b2f2-4dce-938d-48672451d1fe" containerName="istio-proxy"
Oct 31 17:43:17.058207 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:17.057538    4276 cpu_manager.go:414] "containerMap: removing container" podUID="f7894ac3-7f53-4a3f-8d20-17359e24f69e" containerName="payload-pre-processing"
Oct 31 17:43:17.058207 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:17.057543    4276 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f7894ac3-7f53-4a3f-8d20-17359e24f69e" containerName="payload-pre-processing"
Oct 31 17:43:17.058207 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:17.057554    4276 cpu_manager.go:414] "containerMap: removing container" podUID="c3c40ebe-5cac-4981-913d-18f12d0bf439" containerName="istio-proxy"
Oct 31 17:43:17.058207 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:17.057558    4276 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c3c40ebe-5cac-4981-913d-18f12d0bf439" containerName="istio-proxy"
Oct 31 17:43:17.058207 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:17.057564    4276 cpu_manager.go:414] "containerMap: removing container" podUID="bc5595c7-f428-4092-ac29-a30af2e01065" containerName="istio-proxy"
Oct 31 17:43:17.058207 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:17.057568    4276 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bc5595c7-f428-4092-ac29-a30af2e01065" containerName="istio-proxy"
Oct 31 17:43:17.058207 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:17.057573    4276 cpu_manager.go:414] "containerMap: removing container" podUID="a59e3d47-2d57-4bf7-9a6c-5584d2d58d63" containerName="istio-proxy"
Oct 31 17:43:17.058207 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:17.057577    4276 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a59e3d47-2d57-4bf7-9a6c-5584d2d58d63" containerName="istio-proxy"
Oct 31 17:43:17.058207 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:17.057585    4276 cpu_manager.go:414] "containerMap: removing container" podUID="28cbdb7c-1bc2-49b4-834c-de871be33524" containerName="istio-proxy"
Oct 31 17:43:17.058207 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:17.057589    4276 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="28cbdb7c-1bc2-49b4-834c-de871be33524" containerName="istio-proxy"
Oct 31 17:43:17.058207 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:17.057648    4276 memory_manager.go:365] "RemoveStaleState removing state" podUID="33671198-f448-4f99-9bc9-6a1ed6980c5b" containerName="istio-proxy"
Oct 31 17:43:17.058207 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:17.057656    4276 memory_manager.go:365] "RemoveStaleState removing state" podUID="95755676-9f02-4b03-a7e5-f6c8ab4fa965" containerName="maas-api"
Oct 31 17:43:17.058207 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:17.057662    4276 memory_manager.go:365] "RemoveStaleState removing state" podUID="6418b389-728b-4040-9b73-50cf013c00f3" containerName="istio-proxy"
Oct 31 17:43:17.058207 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:17.057667    4276 memory_manager.go:365] "RemoveStaleState removing state" podUID="34243752-91ce-4e92-b0a6-72490b1f7c9e" containerName="istio-proxy"
Oct 31 17:43:17.058207 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:17.057672    4276 memory_manager.go:365] "RemoveStaleState removing state" podUID="4b04b1b0-d2cc-4304-92e6-90e48105f442" containerName="istio-proxy"
Oct 31 17:43:17.058207 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:17.057678    4276 memory_manager.go:365] "RemoveStaleState removing state" podUID="3716260a-fa38-4064-991c-48322b1b48fa" containerName="istio-proxy"
Oct 31 17:43:17.058207 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:17.057684    4276 memory_manager.go:365] "RemoveStaleState removing state" podUID="6418b389-728b-4040-9b73-50cf013c00f3" containerName="istio-proxy"
Oct 31 17:43:17.058207 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:17.057689    4276 memory_manager.go:365] "RemoveStaleState removing state" podUID="6ec6accb-4b2a-45a8-bab2-fe65f2f5df2d" containerName="istio-proxy"
Oct 31 17:43:17.058207 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:17.057693    4276 memory_manager.go:365] "RemoveStaleState removing state" podUID="f675ff71-27f9-44e8-aa32-89f4332d4c3e" containerName="istio-proxy"
Oct 31 17:43:17.058207 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:17.057703    4276 memory_manager.go:365] "RemoveStaleState removing state" podUID="653b8145-5630-4b28-b3c8-624fb92032e3" containerName="revoke-keys"
Oct 31 17:43:17.058207 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:17.057710    4276 memory_manager.go:365] "RemoveStaleState removing state" podUID="d46713a5-7bba-4dfb-b77f-8fae0b39271e" containerName="istio-proxy"
Oct 31 17:43:17.058207 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:17.057715    4276 memory_manager.go:365] "RemoveStaleState removing state" podUID="e948a32e-25a4-4d29-b9dd-01a0539d337c" containerName="istio-proxy"
Oct 31 17:43:17.058207 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:17.057720    4276 memory_manager.go:365] "RemoveStaleState removing state" podUID="9c4606cd-947c-4226-8152-74f8870458e6" containerName="revoke-keys"
Oct 31 17:43:17.058207 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:17.057725    4276 memory_manager.go:365] "RemoveStaleState removing state" podUID="175639e5-138e-428e-bb53-7fe484543496" containerName="maas-api"
Oct 31 17:43:17.058207 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:17.057730    4276 memory_manager.go:365] "RemoveStaleState removing state" podUID="56c4ff3e-b2f2-4dce-938d-48672451d1fe" containerName="istio-proxy"
Oct 31 17:43:17.058207 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:17.057735    4276 memory_manager.go:365] "RemoveStaleState removing state" podUID="f7894ac3-7f53-4a3f-8d20-17359e24f69e" containerName="payload-pre-processing"
Oct 31 17:43:17.058207 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:17.057740    4276 memory_manager.go:365] "RemoveStaleState removing state" podUID="c3c40ebe-5cac-4981-913d-18f12d0bf439" containerName="istio-proxy"
Oct 31 17:43:17.058207 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:17.057744    4276 memory_manager.go:365] "RemoveStaleState removing state" podUID="bc5595c7-f428-4092-ac29-a30af2e01065" containerName="istio-proxy"
Oct 31 17:43:17.058207 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:17.057750    4276 memory_manager.go:365] "RemoveStaleState removing state" podUID="a59e3d47-2d57-4bf7-9a6c-5584d2d58d63" containerName="istio-proxy"
Oct 31 17:43:17.058207 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:17.057755    4276 memory_manager.go:365] "RemoveStaleState removing state" podUID="28cbdb7c-1bc2-49b4-834c-de871be33524" containerName="istio-proxy"
Oct 31 17:43:17.058207 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:17.057760    4276 memory_manager.go:365] "RemoveStaleState removing state" podUID="4c3cb076-d13d-4212-8fbb-86c851b14fe7" containerName="istio-proxy"
Oct 31 17:43:17.058207 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:17.057764    4276 memory_manager.go:365] "RemoveStaleState removing state" podUID="79a9c2b2-8458-4a74-afde-ee00374beb5b" containerName="istio-proxy"
Oct 31 17:43:17.058207 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:17.057769    4276 memory_manager.go:365] "RemoveStaleState removing state" podUID="b2b7f15e-756a-40e6-90af-961551719a7d" containerName="istio-proxy"
Oct 31 17:43:17.058207 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:17.057774    4276 memory_manager.go:365] "RemoveStaleState removing state" podUID="14900971-460e-4126-9089-c4b110d539ab" containerName="istio-proxy"
Oct 31 17:43:17.058207 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:17.057782    4276 memory_manager.go:365] "RemoveStaleState removing state" podUID="b9c2f445-a191-4b00-802a-1b0ded068411" containerName="main"
Oct 31 17:43:17.058207 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:17.057790    4276 memory_manager.go:365] "RemoveStaleState removing state" podUID="6e4109e9-1a59-48c7-9143-e14a01758994" containerName="payload-pre-processing"
Oct 31 17:43:17.062556 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:17.062539    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-58b696d4f-bg2ch"
Oct 31 17:43:17.078521 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:17.078499    4276 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Oct 31 17:43:17.092407 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:17.092381    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-8bpqv\""
Oct 31 17:43:17.092545 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:17.092435    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Oct 31 17:43:17.092545 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:17.092503    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Oct 31 17:43:17.094636 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:17.094614    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-58b696d4f-bg2ch"]
Oct 31 17:43:17.141329 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:17.141306    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2402f942-dbc0-4ea4-85b5-4bfe719e146f-metrics-certs\") pod \"maas-controller-58b696d4f-bg2ch\" (UID: \"2402f942-dbc0-4ea4-85b5-4bfe719e146f\") " pod="opendatahub/maas-controller-58b696d4f-bg2ch"
Oct 31 17:43:17.141430 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:17.141350    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2402f942-dbc0-4ea4-85b5-4bfe719e146f-webhook-cert\") pod \"maas-controller-58b696d4f-bg2ch\" (UID: \"2402f942-dbc0-4ea4-85b5-4bfe719e146f\") " pod="opendatahub/maas-controller-58b696d4f-bg2ch"
Oct 31 17:43:17.141526 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:17.141505    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xzl5j\" (UniqueName: \"kubernetes.io/projected/2402f942-dbc0-4ea4-85b5-4bfe719e146f-kube-api-access-xzl5j\") pod \"maas-controller-58b696d4f-bg2ch\" (UID: \"2402f942-dbc0-4ea4-85b5-4bfe719e146f\") " pod="opendatahub/maas-controller-58b696d4f-bg2ch"
Oct 31 17:43:17.242924 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:17.242899    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2402f942-dbc0-4ea4-85b5-4bfe719e146f-metrics-certs\") pod \"maas-controller-58b696d4f-bg2ch\" (UID: \"2402f942-dbc0-4ea4-85b5-4bfe719e146f\") " pod="opendatahub/maas-controller-58b696d4f-bg2ch"
Oct 31 17:43:17.243034 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:17.242951    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2402f942-dbc0-4ea4-85b5-4bfe719e146f-webhook-cert\") pod \"maas-controller-58b696d4f-bg2ch\" (UID: \"2402f942-dbc0-4ea4-85b5-4bfe719e146f\") " pod="opendatahub/maas-controller-58b696d4f-bg2ch"
Oct 31 17:43:17.243034 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:17.242986    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xzl5j\" (UniqueName: \"kubernetes.io/projected/2402f942-dbc0-4ea4-85b5-4bfe719e146f-kube-api-access-xzl5j\") pod \"maas-controller-58b696d4f-bg2ch\" (UID: \"2402f942-dbc0-4ea4-85b5-4bfe719e146f\") " pod="opendatahub/maas-controller-58b696d4f-bg2ch"
Oct 31 17:43:17.245213 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:17.245189    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2402f942-dbc0-4ea4-85b5-4bfe719e146f-webhook-cert\") pod \"maas-controller-58b696d4f-bg2ch\" (UID: \"2402f942-dbc0-4ea4-85b5-4bfe719e146f\") " pod="opendatahub/maas-controller-58b696d4f-bg2ch"
Oct 31 17:43:17.245308 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:17.245285    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2402f942-dbc0-4ea4-85b5-4bfe719e146f-metrics-certs\") pod \"maas-controller-58b696d4f-bg2ch\" (UID: \"2402f942-dbc0-4ea4-85b5-4bfe719e146f\") " pod="opendatahub/maas-controller-58b696d4f-bg2ch"
Oct 31 17:43:17.264031 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:17.264009    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xzl5j\" (UniqueName: \"kubernetes.io/projected/2402f942-dbc0-4ea4-85b5-4bfe719e146f-kube-api-access-xzl5j\") pod \"maas-controller-58b696d4f-bg2ch\" (UID: \"2402f942-dbc0-4ea4-85b5-4bfe719e146f\") " pod="opendatahub/maas-controller-58b696d4f-bg2ch"
Oct 31 17:43:17.365618 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:17.365552    4276 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="14900971-460e-4126-9089-c4b110d539ab" path="/var/lib/kubelet/pods/14900971-460e-4126-9089-c4b110d539ab/volumes"
Oct 31 17:43:17.365966 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:17.365953    4276 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6ec6accb-4b2a-45a8-bab2-fe65f2f5df2d" path="/var/lib/kubelet/pods/6ec6accb-4b2a-45a8-bab2-fe65f2f5df2d/volumes"
Oct 31 17:43:17.366322 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:17.366311    4276 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a59e3d47-2d57-4bf7-9a6c-5584d2d58d63" path="/var/lib/kubelet/pods/a59e3d47-2d57-4bf7-9a6c-5584d2d58d63/volumes"
Oct 31 17:43:17.367015 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:17.366999    4276 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="bc5595c7-f428-4092-ac29-a30af2e01065" path="/var/lib/kubelet/pods/bc5595c7-f428-4092-ac29-a30af2e01065/volumes"
Oct 31 17:43:17.392144 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:17.392114    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-8bpqv\""
Oct 31 17:43:17.397325 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:17.397308    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-58b696d4f-bg2ch"
Oct 31 17:43:17.702371 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:17.702349    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-58b696d4f-bg2ch"]
Oct 31 17:43:17.703834 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:43:17.703808    4276 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2402f942_dbc0_4ea4_85b5_4bfe719e146f.slice/crio-c2223be2efd057170bae40899d14b87f68557fabced7f1ed9e749f29ae147abb WatchSource:0}: Error finding container c2223be2efd057170bae40899d14b87f68557fabced7f1ed9e749f29ae147abb: Status 404 returned error can't find the container with id c2223be2efd057170bae40899d14b87f68557fabced7f1ed9e749f29ae147abb
Oct 31 17:43:18.101003 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:18.100964    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-58b696d4f-bg2ch" event={"ID":"2402f942-dbc0-4ea4-85b5-4bfe719e146f","Type":"ContainerStarted","Data":"c2223be2efd057170bae40899d14b87f68557fabced7f1ed9e749f29ae147abb"}
Oct 31 17:43:19.439531 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:19.439505    4276 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Oct 31 17:43:20.114138 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:20.114105    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-58b696d4f-bg2ch" event={"ID":"2402f942-dbc0-4ea4-85b5-4bfe719e146f","Type":"ContainerStarted","Data":"5b85c128f0ccf09bcf30b91a21e2a5aea25753666946b8aa0dc658a51423346c"}
Oct 31 17:43:20.114521 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:20.114192    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-58b696d4f-bg2ch"
Oct 31 17:43:20.178568 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:20.178524    4276 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/maas-controller-58b696d4f-bg2ch" podStartSLOduration=1.459660589 podStartE2EDuration="3.178512184s" totalImagesPullingTime="1.718851595s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 17:43:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 17:43:20.177217932 +0000 UTC m=+962.974962861" watchObservedRunningTime="2026-10-31 17:43:20.178512184 +0000 UTC m=+962.976257113"
Oct 31 17:43:20.311271 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:20.311237    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:43:31.120850 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:31.120820    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-58b696d4f-bg2ch"
Oct 31 17:43:42.233839 ip-10-0-129-127 kubenswrapper[4276]: E1031 17:43:42.233794    4276 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tg5qk" containerName="extractor"
Oct 31 17:43:47.841174 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:47.841138    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:43:47.974246 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:47.974206    4276 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-58b696d4f-bg2ch"]
Oct 31 17:43:47.974477 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:47.974420    4276 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/maas-controller-58b696d4f-bg2ch" podUID="2402f942-dbc0-4ea4-85b5-4bfe719e146f" containerName="manager" containerID="cri-o://5b85c128f0ccf09bcf30b91a21e2a5aea25753666946b8aa0dc658a51423346c" gracePeriod=30
Oct 31 17:43:48.221938 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:48.221912    4276 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-58b696d4f-bg2ch"
Oct 31 17:43:48.312697 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:48.312661    4276 generic.go:356] "Generic (PLEG): container finished" podID="2402f942-dbc0-4ea4-85b5-4bfe719e146f" containerID="5b85c128f0ccf09bcf30b91a21e2a5aea25753666946b8aa0dc658a51423346c" exitCode=0
Oct 31 17:43:48.312697 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:48.312697    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-58b696d4f-bg2ch" event={"ID":"2402f942-dbc0-4ea4-85b5-4bfe719e146f","Type":"ContainerDied","Data":"5b85c128f0ccf09bcf30b91a21e2a5aea25753666946b8aa0dc658a51423346c"}
Oct 31 17:43:48.312966 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:48.312721    4276 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-58b696d4f-bg2ch"
Oct 31 17:43:48.312966 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:48.312773    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-58b696d4f-bg2ch" event={"ID":"2402f942-dbc0-4ea4-85b5-4bfe719e146f","Type":"ContainerDied","Data":"c2223be2efd057170bae40899d14b87f68557fabced7f1ed9e749f29ae147abb"}
Oct 31 17:43:48.312966 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:48.312797    4276 scope.go:122] "RemoveContainer" containerID="5b85c128f0ccf09bcf30b91a21e2a5aea25753666946b8aa0dc658a51423346c"
Oct 31 17:43:48.321284 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:48.321268    4276 scope.go:122] "RemoveContainer" containerID="5b85c128f0ccf09bcf30b91a21e2a5aea25753666946b8aa0dc658a51423346c"
Oct 31 17:43:48.321565 ip-10-0-129-127 kubenswrapper[4276]: E1031 17:43:48.321549    4276 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5b85c128f0ccf09bcf30b91a21e2a5aea25753666946b8aa0dc658a51423346c\": container with ID starting with 5b85c128f0ccf09bcf30b91a21e2a5aea25753666946b8aa0dc658a51423346c not found: ID does not exist" containerID="5b85c128f0ccf09bcf30b91a21e2a5aea25753666946b8aa0dc658a51423346c"
Oct 31 17:43:48.321635 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:48.321576    4276 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5b85c128f0ccf09bcf30b91a21e2a5aea25753666946b8aa0dc658a51423346c"} err="failed to get container status \"5b85c128f0ccf09bcf30b91a21e2a5aea25753666946b8aa0dc658a51423346c\": rpc error: code = NotFound desc = could not find container \"5b85c128f0ccf09bcf30b91a21e2a5aea25753666946b8aa0dc658a51423346c\": container with ID starting with 5b85c128f0ccf09bcf30b91a21e2a5aea25753666946b8aa0dc658a51423346c not found: ID does not exist"
Oct 31 17:43:48.355328 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:48.355275    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2402f942-dbc0-4ea4-85b5-4bfe719e146f-kube-api-access-xzl5j\" (UniqueName: \"kubernetes.io/projected/2402f942-dbc0-4ea4-85b5-4bfe719e146f-kube-api-access-xzl5j\") pod \"2402f942-dbc0-4ea4-85b5-4bfe719e146f\" (UID: \"2402f942-dbc0-4ea4-85b5-4bfe719e146f\") "
Oct 31 17:43:48.355430 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:48.355367    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2402f942-dbc0-4ea4-85b5-4bfe719e146f-webhook-cert\" (UniqueName: \"kubernetes.io/secret/2402f942-dbc0-4ea4-85b5-4bfe719e146f-webhook-cert\") pod \"2402f942-dbc0-4ea4-85b5-4bfe719e146f\" (UID: \"2402f942-dbc0-4ea4-85b5-4bfe719e146f\") "
Oct 31 17:43:48.355512 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:48.355427    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2402f942-dbc0-4ea4-85b5-4bfe719e146f-metrics-certs\" (UniqueName: \"kubernetes.io/secret/2402f942-dbc0-4ea4-85b5-4bfe719e146f-metrics-certs\") pod \"2402f942-dbc0-4ea4-85b5-4bfe719e146f\" (UID: \"2402f942-dbc0-4ea4-85b5-4bfe719e146f\") "
Oct 31 17:43:48.357265 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:48.357238    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2402f942-dbc0-4ea4-85b5-4bfe719e146f-webhook-cert" pod "2402f942-dbc0-4ea4-85b5-4bfe719e146f" (UID: "2402f942-dbc0-4ea4-85b5-4bfe719e146f"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 17:43:48.357407 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:48.357284    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2402f942-dbc0-4ea4-85b5-4bfe719e146f-metrics-certs" pod "2402f942-dbc0-4ea4-85b5-4bfe719e146f" (UID: "2402f942-dbc0-4ea4-85b5-4bfe719e146f"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 17:43:48.357407 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:48.357321    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2402f942-dbc0-4ea4-85b5-4bfe719e146f-kube-api-access-xzl5j" pod "2402f942-dbc0-4ea4-85b5-4bfe719e146f" (UID: "2402f942-dbc0-4ea4-85b5-4bfe719e146f"). InnerVolumeSpecName "kube-api-access-xzl5j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 17:43:48.457169 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:48.457136    4276 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xzl5j\" (UniqueName: \"kubernetes.io/projected/2402f942-dbc0-4ea4-85b5-4bfe719e146f-kube-api-access-xzl5j\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:43:48.457169 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:48.457168    4276 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2402f942-dbc0-4ea4-85b5-4bfe719e146f-webhook-cert\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:43:48.457346 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:48.457180    4276 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2402f942-dbc0-4ea4-85b5-4bfe719e146f-metrics-certs\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:43:48.659854 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:48.659815    4276 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-58b696d4f-bg2ch"]
Oct 31 17:43:48.674010 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:48.673976    4276 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-58b696d4f-bg2ch"]
Oct 31 17:43:49.366142 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:49.366111    4276 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2402f942-dbc0-4ea4-85b5-4bfe719e146f" path="/var/lib/kubelet/pods/2402f942-dbc0-4ea4-85b5-4bfe719e146f/volumes"
Oct 31 17:43:49.706388 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:49.706317    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:43:49.830982 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:49.830948    4276 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-58b696d4f-nlh9h"]
Oct 31 17:43:49.831325 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:49.831299    4276 cpu_manager.go:414] "containerMap: removing container" podUID="33671198-f448-4f99-9bc9-6a1ed6980c5b" containerName="istio-proxy"
Oct 31 17:43:49.831325 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:49.831314    4276 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="33671198-f448-4f99-9bc9-6a1ed6980c5b" containerName="istio-proxy"
Oct 31 17:43:49.831477 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:49.831333    4276 cpu_manager.go:414] "containerMap: removing container" podUID="2402f942-dbc0-4ea4-85b5-4bfe719e146f" containerName="manager"
Oct 31 17:43:49.831477 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:49.831338    4276 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2402f942-dbc0-4ea4-85b5-4bfe719e146f" containerName="manager"
Oct 31 17:43:49.831477 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:49.831404    4276 memory_manager.go:365] "RemoveStaleState removing state" podUID="2402f942-dbc0-4ea4-85b5-4bfe719e146f" containerName="manager"
Oct 31 17:43:49.831477 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:49.831414    4276 memory_manager.go:365] "RemoveStaleState removing state" podUID="33671198-f448-4f99-9bc9-6a1ed6980c5b" containerName="istio-proxy"
Oct 31 17:43:49.835851 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:49.835830    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-58b696d4f-nlh9h"
Oct 31 17:43:49.851959 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:49.851929    4276 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Oct 31 17:43:49.852056 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:49.852039    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Oct 31 17:43:49.852207 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:49.852102    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-8bpqv\""
Oct 31 17:43:49.852207 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:49.852182    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Oct 31 17:43:49.852357 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:49.852341    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-58b696d4f-nlh9h"]
Oct 31 17:43:49.973752 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:49.973670    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/39302afc-f80b-481b-925d-eaeffc127236-webhook-cert\") pod \"maas-controller-58b696d4f-nlh9h\" (UID: \"39302afc-f80b-481b-925d-eaeffc127236\") " pod="opendatahub/maas-controller-58b696d4f-nlh9h"
Oct 31 17:43:49.973752 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:49.973715    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wh8ps\" (UniqueName: \"kubernetes.io/projected/39302afc-f80b-481b-925d-eaeffc127236-kube-api-access-wh8ps\") pod \"maas-controller-58b696d4f-nlh9h\" (UID: \"39302afc-f80b-481b-925d-eaeffc127236\") " pod="opendatahub/maas-controller-58b696d4f-nlh9h"
Oct 31 17:43:49.973752 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:49.973734    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/39302afc-f80b-481b-925d-eaeffc127236-metrics-certs\") pod \"maas-controller-58b696d4f-nlh9h\" (UID: \"39302afc-f80b-481b-925d-eaeffc127236\") " pod="opendatahub/maas-controller-58b696d4f-nlh9h"
Oct 31 17:43:50.074284 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:50.074260    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/39302afc-f80b-481b-925d-eaeffc127236-webhook-cert\") pod \"maas-controller-58b696d4f-nlh9h\" (UID: \"39302afc-f80b-481b-925d-eaeffc127236\") " pod="opendatahub/maas-controller-58b696d4f-nlh9h"
Oct 31 17:43:50.074433 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:50.074300    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wh8ps\" (UniqueName: \"kubernetes.io/projected/39302afc-f80b-481b-925d-eaeffc127236-kube-api-access-wh8ps\") pod \"maas-controller-58b696d4f-nlh9h\" (UID: \"39302afc-f80b-481b-925d-eaeffc127236\") " pod="opendatahub/maas-controller-58b696d4f-nlh9h"
Oct 31 17:43:50.074433 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:50.074320    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/39302afc-f80b-481b-925d-eaeffc127236-metrics-certs\") pod \"maas-controller-58b696d4f-nlh9h\" (UID: \"39302afc-f80b-481b-925d-eaeffc127236\") " pod="opendatahub/maas-controller-58b696d4f-nlh9h"
Oct 31 17:43:50.076747 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:50.076726    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/39302afc-f80b-481b-925d-eaeffc127236-webhook-cert\") pod \"maas-controller-58b696d4f-nlh9h\" (UID: \"39302afc-f80b-481b-925d-eaeffc127236\") " pod="opendatahub/maas-controller-58b696d4f-nlh9h"
Oct 31 17:43:50.076829 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:50.076793    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/39302afc-f80b-481b-925d-eaeffc127236-metrics-certs\") pod \"maas-controller-58b696d4f-nlh9h\" (UID: \"39302afc-f80b-481b-925d-eaeffc127236\") " pod="opendatahub/maas-controller-58b696d4f-nlh9h"
Oct 31 17:43:50.094850 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:50.094824    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wh8ps\" (UniqueName: \"kubernetes.io/projected/39302afc-f80b-481b-925d-eaeffc127236-kube-api-access-wh8ps\") pod \"maas-controller-58b696d4f-nlh9h\" (UID: \"39302afc-f80b-481b-925d-eaeffc127236\") " pod="opendatahub/maas-controller-58b696d4f-nlh9h"
Oct 31 17:43:50.148566 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:50.148547    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-58b696d4f-nlh9h"
Oct 31 17:43:50.448030 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:50.447952    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-58b696d4f-nlh9h"]
Oct 31 17:43:50.450328 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:43:50.450300    4276 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39302afc_f80b_481b_925d_eaeffc127236.slice/crio-9aea5d7f0b10d4f042d1948fc45e3661fb7fbe67617430215394e946ea444ecb WatchSource:0}: Error finding container 9aea5d7f0b10d4f042d1948fc45e3661fb7fbe67617430215394e946ea444ecb: Status 404 returned error can't find the container with id 9aea5d7f0b10d4f042d1948fc45e3661fb7fbe67617430215394e946ea444ecb
Oct 31 17:43:51.335295 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:51.335258    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-58b696d4f-nlh9h" event={"ID":"39302afc-f80b-481b-925d-eaeffc127236","Type":"ContainerStarted","Data":"844ced5281395ad372447d685a001b07743804991f7d9f6ddc356cecd54aaab6"}
Oct 31 17:43:51.335518 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:51.335328    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-58b696d4f-nlh9h" event={"ID":"39302afc-f80b-481b-925d-eaeffc127236","Type":"ContainerStarted","Data":"9aea5d7f0b10d4f042d1948fc45e3661fb7fbe67617430215394e946ea444ecb"}
Oct 31 17:43:51.335518 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:51.335380    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-58b696d4f-nlh9h"
Oct 31 17:43:51.379134 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:43:51.379086    4276 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/maas-controller-58b696d4f-nlh9h" podStartSLOduration=2.02829391 podStartE2EDuration="2.37907429s" totalImagesPullingTime="350.78038ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 17:43:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 17:43:51.376699954 +0000 UTC m=+994.174444882" watchObservedRunningTime="2026-10-31 17:43:51.37907429 +0000 UTC m=+994.176819217"
Oct 31 17:44:02.341988 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:44:02.341955    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-58b696d4f-nlh9h"
Oct 31 17:44:03.714400 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:44:03.714367    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:44:12.234259 ip-10-0-129-127 kubenswrapper[4276]: E1031 17:44:12.234222    4276 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tg5qk" containerName="extractor"
Oct 31 17:44:21.596536 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:44:21.596504    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:44:42.234270 ip-10-0-129-127 kubenswrapper[4276]: E1031 17:44:42.234181    4276 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tg5qk" containerName="extractor"
Oct 31 17:44:45.596551 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:44:45.596515    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:44:56.479840 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:44:56.479799    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:44:56.595954 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:44:56.595905    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:44:58.668330 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:44:58.668297    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:44:58.780426 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:44:58.780393    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:45:03.924799 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:45:03.924732    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:45:11.954021 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:45:11.953617    4276 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7d9897b54c-pfv5k container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.30:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 17:45:11.954021 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:45:11.953686    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-pfv5k" podUID="a0744b4f-5676-4385-b50c-5796ca2365a0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.30:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 17:45:12.234871 ip-10-0-129-127 kubenswrapper[4276]: E1031 17:45:12.234785    4276 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tg5qk" containerName="extractor"
Oct 31 17:45:42.233107 ip-10-0-129-127 kubenswrapper[4276]: E1031 17:45:42.233068    4276 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tg5qk" containerName="extractor"
Oct 31 17:46:04.041669 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:46:04.041432    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:46:08.839657 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:46:08.839623    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:46:12.233705 ip-10-0-129-127 kubenswrapper[4276]: E1031 17:46:12.233669    4276 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tg5qk" containerName="extractor"
Oct 31 17:46:13.936296 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:46:13.936245    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:46:42.233525 ip-10-0-129-127 kubenswrapper[4276]: E1031 17:46:42.233487    4276 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tg5qk" containerName="extractor"
Oct 31 17:47:04.144889 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:47:04.144853    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:47:06.091994 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:47:06.091956    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:47:09.060472 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:47:09.060088    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:47:12.233395 ip-10-0-129-127 kubenswrapper[4276]: E1031 17:47:12.233359    4276 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tg5qk" containerName="extractor"
Oct 31 17:47:14.044914 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:47:14.044879    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:47:17.475527 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:47:17.475491    4276 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zpf4d_5819fd64-9ac0-4436-b8fe-dcd3a27827ac/ovn-acl-logging/0.log"
Oct 31 17:47:17.480173 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:47:17.480154    4276 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zpf4d_5819fd64-9ac0-4436-b8fe-dcd3a27827ac/ovn-acl-logging/0.log"
Oct 31 17:47:37.096550 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:47:37.094510    4276 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7d9897b54c-b8gmm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.34:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 17:47:37.096550 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:47:37.094585    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-b8gmm" podUID="5530ee8f-be36-43d3-a268-051794343d8b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.34:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 17:47:42.233936 ip-10-0-129-127 kubenswrapper[4276]: E1031 17:47:42.233899    4276 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tg5qk" containerName="extractor"
Oct 31 17:47:47.659308 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:47:47.659272    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:47:47.772491 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:47:47.772461    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:47:48.476438 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:47:48.476406    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:47:54.027606 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:47:54.027577    4276 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-0a3ab6fd-62wxv"]
Oct 31 17:47:54.041376 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:47:54.041348    4276 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-0a3ab6fd-62wxv"]
Oct 31 17:47:54.042891 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:47:54.042878    4276 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-f61ac551-sc5kq"]
Oct 31 17:47:54.044772 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:47:54.044748    4276 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-f61ac551-sc5kq"]
Oct 31 17:47:55.365647 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:47:55.365616    4276 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="653b8145-5630-4b28-b3c8-624fb92032e3" path="/var/lib/kubelet/pods/653b8145-5630-4b28-b3c8-624fb92032e3/volumes"
Oct 31 17:47:55.366045 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:47:55.366032    4276 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9c4606cd-947c-4226-8152-74f8870458e6" path="/var/lib/kubelet/pods/9c4606cd-947c-4226-8152-74f8870458e6/volumes"
Oct 31 17:48:04.256424 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:48:04.256387    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:48:09.160525 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:48:09.160483    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:48:11.956577 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:48:11.956536    4276 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7d9897b54c-pfv5k container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.30:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 17:48:11.957199 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:48:11.957167    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-pfv5k" podUID="a0744b4f-5676-4385-b50c-5796ca2365a0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.30:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 17:48:12.334600 ip-10-0-129-127 kubenswrapper[4276]: E1031 17:48:12.334559    4276 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tg5qk" containerName="extractor"
Oct 31 17:48:14.148933 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:48:14.148899    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:48:24.756334 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:48:24.756301    4276 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-xxspc"]
Oct 31 17:48:24.756837 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:48:24.756539    4276 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-xxspc" podUID="00caa007-32e1-4ff0-a5d5-71ab1b73eca9" containerName="discovery" containerID="cri-o://594e00325125304f047f781d286f5fba5653108de898b1385b8fbe3f97737124" gracePeriod=30
Oct 31 17:48:25.006590 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:48:25.006528    4276 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-xxspc"
Oct 31 17:48:25.145166 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:48:25.145135    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/00caa007-32e1-4ff0-a5d5-71ab1b73eca9-istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/00caa007-32e1-4ff0-a5d5-71ab1b73eca9-istio-csr-dns-cert\") pod \"00caa007-32e1-4ff0-a5d5-71ab1b73eca9\" (UID: \"00caa007-32e1-4ff0-a5d5-71ab1b73eca9\") "
Oct 31 17:48:25.145334 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:48:25.145211    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/00caa007-32e1-4ff0-a5d5-71ab1b73eca9-istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/00caa007-32e1-4ff0-a5d5-71ab1b73eca9-istio-kubeconfig\") pod \"00caa007-32e1-4ff0-a5d5-71ab1b73eca9\" (UID: \"00caa007-32e1-4ff0-a5d5-71ab1b73eca9\") "
Oct 31 17:48:25.145334 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:48:25.145256    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/00caa007-32e1-4ff0-a5d5-71ab1b73eca9-istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/00caa007-32e1-4ff0-a5d5-71ab1b73eca9-istio-csr-ca-configmap\") pod \"00caa007-32e1-4ff0-a5d5-71ab1b73eca9\" (UID: \"00caa007-32e1-4ff0-a5d5-71ab1b73eca9\") "
Oct 31 17:48:25.145334 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:48:25.145276    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/00caa007-32e1-4ff0-a5d5-71ab1b73eca9-local-certs\" (UniqueName: \"kubernetes.io/empty-dir/00caa007-32e1-4ff0-a5d5-71ab1b73eca9-local-certs\") pod \"00caa007-32e1-4ff0-a5d5-71ab1b73eca9\" (UID: \"00caa007-32e1-4ff0-a5d5-71ab1b73eca9\") "
Oct 31 17:48:25.145334 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:48:25.145311    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/00caa007-32e1-4ff0-a5d5-71ab1b73eca9-istio-token\" (UniqueName: \"kubernetes.io/projected/00caa007-32e1-4ff0-a5d5-71ab1b73eca9-istio-token\") pod \"00caa007-32e1-4ff0-a5d5-71ab1b73eca9\" (UID: \"00caa007-32e1-4ff0-a5d5-71ab1b73eca9\") "
Oct 31 17:48:25.145554 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:48:25.145348    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/00caa007-32e1-4ff0-a5d5-71ab1b73eca9-cacerts\" (UniqueName: \"kubernetes.io/secret/00caa007-32e1-4ff0-a5d5-71ab1b73eca9-cacerts\") pod \"00caa007-32e1-4ff0-a5d5-71ab1b73eca9\" (UID: \"00caa007-32e1-4ff0-a5d5-71ab1b73eca9\") "
Oct 31 17:48:25.145554 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:48:25.145419    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/00caa007-32e1-4ff0-a5d5-71ab1b73eca9-kube-api-access-7bc98\" (UniqueName: \"kubernetes.io/projected/00caa007-32e1-4ff0-a5d5-71ab1b73eca9-kube-api-access-7bc98\") pod \"00caa007-32e1-4ff0-a5d5-71ab1b73eca9\" (UID: \"00caa007-32e1-4ff0-a5d5-71ab1b73eca9\") "
Oct 31 17:48:25.145845 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:48:25.145666    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/00caa007-32e1-4ff0-a5d5-71ab1b73eca9-istio-csr-ca-configmap" pod "00caa007-32e1-4ff0-a5d5-71ab1b73eca9" (UID: "00caa007-32e1-4ff0-a5d5-71ab1b73eca9"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 17:48:25.145845 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:48:25.145803    4276 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/00caa007-32e1-4ff0-a5d5-71ab1b73eca9-istio-csr-ca-configmap\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:48:25.147910 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:48:25.147884    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/00caa007-32e1-4ff0-a5d5-71ab1b73eca9-istio-csr-dns-cert" pod "00caa007-32e1-4ff0-a5d5-71ab1b73eca9" (UID: "00caa007-32e1-4ff0-a5d5-71ab1b73eca9"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 17:48:25.147910 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:48:25.147891    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/00caa007-32e1-4ff0-a5d5-71ab1b73eca9-local-certs" pod "00caa007-32e1-4ff0-a5d5-71ab1b73eca9" (UID: "00caa007-32e1-4ff0-a5d5-71ab1b73eca9"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:48:25.148098 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:48:25.148059    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/00caa007-32e1-4ff0-a5d5-71ab1b73eca9-istio-token" pod "00caa007-32e1-4ff0-a5d5-71ab1b73eca9" (UID: "00caa007-32e1-4ff0-a5d5-71ab1b73eca9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 17:48:25.148159 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:48:25.148092    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/00caa007-32e1-4ff0-a5d5-71ab1b73eca9-istio-kubeconfig" pod "00caa007-32e1-4ff0-a5d5-71ab1b73eca9" (UID: "00caa007-32e1-4ff0-a5d5-71ab1b73eca9"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 17:48:25.148159 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:48:25.148103    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/00caa007-32e1-4ff0-a5d5-71ab1b73eca9-cacerts" pod "00caa007-32e1-4ff0-a5d5-71ab1b73eca9" (UID: "00caa007-32e1-4ff0-a5d5-71ab1b73eca9"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 17:48:25.148159 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:48:25.148140    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/00caa007-32e1-4ff0-a5d5-71ab1b73eca9-kube-api-access-7bc98" pod "00caa007-32e1-4ff0-a5d5-71ab1b73eca9" (UID: "00caa007-32e1-4ff0-a5d5-71ab1b73eca9"). InnerVolumeSpecName "kube-api-access-7bc98". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 17:48:25.246785 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:48:25.246751    4276 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/00caa007-32e1-4ff0-a5d5-71ab1b73eca9-istio-kubeconfig\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:48:25.246785 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:48:25.246779    4276 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/00caa007-32e1-4ff0-a5d5-71ab1b73eca9-local-certs\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:48:25.246785 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:48:25.246788    4276 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/00caa007-32e1-4ff0-a5d5-71ab1b73eca9-istio-token\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:48:25.247031 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:48:25.246797    4276 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/00caa007-32e1-4ff0-a5d5-71ab1b73eca9-cacerts\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:48:25.247031 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:48:25.246805    4276 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7bc98\" (UniqueName: \"kubernetes.io/projected/00caa007-32e1-4ff0-a5d5-71ab1b73eca9-kube-api-access-7bc98\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:48:25.247031 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:48:25.246813    4276 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/00caa007-32e1-4ff0-a5d5-71ab1b73eca9-istio-csr-dns-cert\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:48:25.876884 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:48:25.876851    4276 generic.go:356] "Generic (PLEG): container finished" podID="00caa007-32e1-4ff0-a5d5-71ab1b73eca9" containerID="594e00325125304f047f781d286f5fba5653108de898b1385b8fbe3f97737124" exitCode=0
Oct 31 17:48:25.877279 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:48:25.876899    4276 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-xxspc"
Oct 31 17:48:25.877279 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:48:25.876927    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-xxspc" event={"ID":"00caa007-32e1-4ff0-a5d5-71ab1b73eca9","Type":"ContainerDied","Data":"594e00325125304f047f781d286f5fba5653108de898b1385b8fbe3f97737124"}
Oct 31 17:48:25.877279 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:48:25.876998    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-xxspc" event={"ID":"00caa007-32e1-4ff0-a5d5-71ab1b73eca9","Type":"ContainerDied","Data":"f35a37948565970d578770e547d4183b6fdcd78cd0431de102c80cdb0811cd5a"}
Oct 31 17:48:25.877279 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:48:25.877019    4276 scope.go:122] "RemoveContainer" containerID="594e00325125304f047f781d286f5fba5653108de898b1385b8fbe3f97737124"
Oct 31 17:48:25.888738 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:48:25.888718    4276 scope.go:122] "RemoveContainer" containerID="5fa1879574b282c012c1a5593a44970aaff45dac29486f07a895d00e879a92c2"
Oct 31 17:48:25.897280 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:48:25.897261    4276 scope.go:122] "RemoveContainer" containerID="594e00325125304f047f781d286f5fba5653108de898b1385b8fbe3f97737124"
Oct 31 17:48:25.897574 ip-10-0-129-127 kubenswrapper[4276]: E1031 17:48:25.897556    4276 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"594e00325125304f047f781d286f5fba5653108de898b1385b8fbe3f97737124\": container with ID starting with 594e00325125304f047f781d286f5fba5653108de898b1385b8fbe3f97737124 not found: ID does not exist" containerID="594e00325125304f047f781d286f5fba5653108de898b1385b8fbe3f97737124"
Oct 31 17:48:25.897574 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:48:25.897582    4276 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"594e00325125304f047f781d286f5fba5653108de898b1385b8fbe3f97737124"} err="failed to get container status \"594e00325125304f047f781d286f5fba5653108de898b1385b8fbe3f97737124\": rpc error: code = NotFound desc = could not find container \"594e00325125304f047f781d286f5fba5653108de898b1385b8fbe3f97737124\": container with ID starting with 594e00325125304f047f781d286f5fba5653108de898b1385b8fbe3f97737124 not found: ID does not exist"
Oct 31 17:48:25.897574 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:48:25.897604    4276 scope.go:122] "RemoveContainer" containerID="5fa1879574b282c012c1a5593a44970aaff45dac29486f07a895d00e879a92c2"
Oct 31 17:48:25.897908 ip-10-0-129-127 kubenswrapper[4276]: E1031 17:48:25.897891    4276 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5fa1879574b282c012c1a5593a44970aaff45dac29486f07a895d00e879a92c2\": container with ID starting with 5fa1879574b282c012c1a5593a44970aaff45dac29486f07a895d00e879a92c2 not found: ID does not exist" containerID="5fa1879574b282c012c1a5593a44970aaff45dac29486f07a895d00e879a92c2"
Oct 31 17:48:25.897985 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:48:25.897918    4276 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5fa1879574b282c012c1a5593a44970aaff45dac29486f07a895d00e879a92c2"} err="failed to get container status \"5fa1879574b282c012c1a5593a44970aaff45dac29486f07a895d00e879a92c2\": rpc error: code = NotFound desc = could not find container \"5fa1879574b282c012c1a5593a44970aaff45dac29486f07a895d00e879a92c2\": container with ID starting with 5fa1879574b282c012c1a5593a44970aaff45dac29486f07a895d00e879a92c2 not found: ID does not exist"
Oct 31 17:48:25.944636 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:48:25.944615    4276 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-xxspc"]
Oct 31 17:48:25.958433 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:48:25.958407    4276 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-xxspc"]
Oct 31 17:48:27.366157 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:48:27.366124    4276 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="00caa007-32e1-4ff0-a5d5-71ab1b73eca9" path="/var/lib/kubelet/pods/00caa007-32e1-4ff0-a5d5-71ab1b73eca9/volumes"
Oct 31 17:48:42.233199 ip-10-0-129-127 kubenswrapper[4276]: E1031 17:48:42.233153    4276 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tg5qk" containerName="extractor"
Oct 31 17:49:04.393873 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:04.393791    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:49:06.089048 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:06.089009    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:49:06.533305 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:06.533237    4276 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-single-main-60d748-openshift-default-7859bd475cskdsz"]
Oct 31 17:49:06.533608 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:06.533595    4276 cpu_manager.go:414] "containerMap: removing container" podUID="00caa007-32e1-4ff0-a5d5-71ab1b73eca9" containerName="discovery"
Oct 31 17:49:06.533665 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:06.533609    4276 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="00caa007-32e1-4ff0-a5d5-71ab1b73eca9" containerName="discovery"
Oct 31 17:49:06.533665 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:06.533619    4276 cpu_manager.go:414] "containerMap: removing container" podUID="00caa007-32e1-4ff0-a5d5-71ab1b73eca9" containerName="discovery"
Oct 31 17:49:06.533665 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:06.533624    4276 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="00caa007-32e1-4ff0-a5d5-71ab1b73eca9" containerName="discovery"
Oct 31 17:49:06.533773 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:06.533705    4276 memory_manager.go:365] "RemoveStaleState removing state" podUID="00caa007-32e1-4ff0-a5d5-71ab1b73eca9" containerName="discovery"
Oct 31 17:49:06.536853 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:06.536834    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-single-main-60d748-openshift-default-7859bd475cskdsz"
Oct 31 17:49:06.556738 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:06.556713    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-single-main-60d748-openshift-default-dockercfg-wfzv7\""
Oct 31 17:49:06.573502 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:06.573474    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-single-main-60d748-openshift-default-7859bd475cskdsz"]
Oct 31 17:49:06.604377 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:06.604353    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:49:06.636367 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:06.636334    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e5b8fc2-0092-4c1b-b689-cce7439e1c2e-workload-socket\") pod \"e2e-worker-single-main-60d748-openshift-default-7859bd475cskdsz\" (UID: \"5e5b8fc2-0092-4c1b-b689-cce7439e1c2e\") " pod="openshift-ingress/e2e-worker-single-main-60d748-openshift-default-7859bd475cskdsz"
Oct 31 17:49:06.636908 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:06.636880    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5e5b8fc2-0092-4c1b-b689-cce7439e1c2e-istio-envoy\") pod \"e2e-worker-single-main-60d748-openshift-default-7859bd475cskdsz\" (UID: \"5e5b8fc2-0092-4c1b-b689-cce7439e1c2e\") " pod="openshift-ingress/e2e-worker-single-main-60d748-openshift-default-7859bd475cskdsz"
Oct 31 17:49:06.637109 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:06.637066    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5e5b8fc2-0092-4c1b-b689-cce7439e1c2e-istio-podinfo\") pod \"e2e-worker-single-main-60d748-openshift-default-7859bd475cskdsz\" (UID: \"5e5b8fc2-0092-4c1b-b689-cce7439e1c2e\") " pod="openshift-ingress/e2e-worker-single-main-60d748-openshift-default-7859bd475cskdsz"
Oct 31 17:49:06.637109 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:06.637106    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5e5b8fc2-0092-4c1b-b689-cce7439e1c2e-istiod-ca-cert\") pod \"e2e-worker-single-main-60d748-openshift-default-7859bd475cskdsz\" (UID: \"5e5b8fc2-0092-4c1b-b689-cce7439e1c2e\") " pod="openshift-ingress/e2e-worker-single-main-60d748-openshift-default-7859bd475cskdsz"
Oct 31 17:49:06.638557 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:06.637201    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5e5b8fc2-0092-4c1b-b689-cce7439e1c2e-istio-data\") pod \"e2e-worker-single-main-60d748-openshift-default-7859bd475cskdsz\" (UID: \"5e5b8fc2-0092-4c1b-b689-cce7439e1c2e\") " pod="openshift-ingress/e2e-worker-single-main-60d748-openshift-default-7859bd475cskdsz"
Oct 31 17:49:06.638557 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:06.637236    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5e5b8fc2-0092-4c1b-b689-cce7439e1c2e-workload-certs\") pod \"e2e-worker-single-main-60d748-openshift-default-7859bd475cskdsz\" (UID: \"5e5b8fc2-0092-4c1b-b689-cce7439e1c2e\") " pod="openshift-ingress/e2e-worker-single-main-60d748-openshift-default-7859bd475cskdsz"
Oct 31 17:49:06.638557 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:06.637274    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e5b8fc2-0092-4c1b-b689-cce7439e1c2e-credential-socket\") pod \"e2e-worker-single-main-60d748-openshift-default-7859bd475cskdsz\" (UID: \"5e5b8fc2-0092-4c1b-b689-cce7439e1c2e\") " pod="openshift-ingress/e2e-worker-single-main-60d748-openshift-default-7859bd475cskdsz"
Oct 31 17:49:06.638557 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:06.637312    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5e5b8fc2-0092-4c1b-b689-cce7439e1c2e-istio-token\") pod \"e2e-worker-single-main-60d748-openshift-default-7859bd475cskdsz\" (UID: \"5e5b8fc2-0092-4c1b-b689-cce7439e1c2e\") " pod="openshift-ingress/e2e-worker-single-main-60d748-openshift-default-7859bd475cskdsz"
Oct 31 17:49:06.638557 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:06.637348    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tqf8w\" (UniqueName: \"kubernetes.io/projected/5e5b8fc2-0092-4c1b-b689-cce7439e1c2e-kube-api-access-tqf8w\") pod \"e2e-worker-single-main-60d748-openshift-default-7859bd475cskdsz\" (UID: \"5e5b8fc2-0092-4c1b-b689-cce7439e1c2e\") " pod="openshift-ingress/e2e-worker-single-main-60d748-openshift-default-7859bd475cskdsz"
Oct 31 17:49:06.738251 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:06.738222    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5e5b8fc2-0092-4c1b-b689-cce7439e1c2e-istio-data\") pod \"e2e-worker-single-main-60d748-openshift-default-7859bd475cskdsz\" (UID: \"5e5b8fc2-0092-4c1b-b689-cce7439e1c2e\") " pod="openshift-ingress/e2e-worker-single-main-60d748-openshift-default-7859bd475cskdsz"
Oct 31 17:49:06.738477 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:06.738257    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5e5b8fc2-0092-4c1b-b689-cce7439e1c2e-workload-certs\") pod \"e2e-worker-single-main-60d748-openshift-default-7859bd475cskdsz\" (UID: \"5e5b8fc2-0092-4c1b-b689-cce7439e1c2e\") " pod="openshift-ingress/e2e-worker-single-main-60d748-openshift-default-7859bd475cskdsz"
Oct 31 17:49:06.738477 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:06.738283    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e5b8fc2-0092-4c1b-b689-cce7439e1c2e-credential-socket\") pod \"e2e-worker-single-main-60d748-openshift-default-7859bd475cskdsz\" (UID: \"5e5b8fc2-0092-4c1b-b689-cce7439e1c2e\") " pod="openshift-ingress/e2e-worker-single-main-60d748-openshift-default-7859bd475cskdsz"
Oct 31 17:49:06.738477 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:06.738356    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5e5b8fc2-0092-4c1b-b689-cce7439e1c2e-istio-token\") pod \"e2e-worker-single-main-60d748-openshift-default-7859bd475cskdsz\" (UID: \"5e5b8fc2-0092-4c1b-b689-cce7439e1c2e\") " pod="openshift-ingress/e2e-worker-single-main-60d748-openshift-default-7859bd475cskdsz"
Oct 31 17:49:06.738477 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:06.738395    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tqf8w\" (UniqueName: \"kubernetes.io/projected/5e5b8fc2-0092-4c1b-b689-cce7439e1c2e-kube-api-access-tqf8w\") pod \"e2e-worker-single-main-60d748-openshift-default-7859bd475cskdsz\" (UID: \"5e5b8fc2-0092-4c1b-b689-cce7439e1c2e\") " pod="openshift-ingress/e2e-worker-single-main-60d748-openshift-default-7859bd475cskdsz"
Oct 31 17:49:06.738477 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:06.738422    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e5b8fc2-0092-4c1b-b689-cce7439e1c2e-workload-socket\") pod \"e2e-worker-single-main-60d748-openshift-default-7859bd475cskdsz\" (UID: \"5e5b8fc2-0092-4c1b-b689-cce7439e1c2e\") " pod="openshift-ingress/e2e-worker-single-main-60d748-openshift-default-7859bd475cskdsz"
Oct 31 17:49:06.738758 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:06.738504    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5e5b8fc2-0092-4c1b-b689-cce7439e1c2e-istio-envoy\") pod \"e2e-worker-single-main-60d748-openshift-default-7859bd475cskdsz\" (UID: \"5e5b8fc2-0092-4c1b-b689-cce7439e1c2e\") " pod="openshift-ingress/e2e-worker-single-main-60d748-openshift-default-7859bd475cskdsz"
Oct 31 17:49:06.738758 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:06.738538    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5e5b8fc2-0092-4c1b-b689-cce7439e1c2e-istio-podinfo\") pod \"e2e-worker-single-main-60d748-openshift-default-7859bd475cskdsz\" (UID: \"5e5b8fc2-0092-4c1b-b689-cce7439e1c2e\") " pod="openshift-ingress/e2e-worker-single-main-60d748-openshift-default-7859bd475cskdsz"
Oct 31 17:49:06.738758 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:06.738568    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5e5b8fc2-0092-4c1b-b689-cce7439e1c2e-istiod-ca-cert\") pod \"e2e-worker-single-main-60d748-openshift-default-7859bd475cskdsz\" (UID: \"5e5b8fc2-0092-4c1b-b689-cce7439e1c2e\") " pod="openshift-ingress/e2e-worker-single-main-60d748-openshift-default-7859bd475cskdsz"
Oct 31 17:49:06.738758 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:06.738700    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5e5b8fc2-0092-4c1b-b689-cce7439e1c2e-workload-certs\") pod \"e2e-worker-single-main-60d748-openshift-default-7859bd475cskdsz\" (UID: \"5e5b8fc2-0092-4c1b-b689-cce7439e1c2e\") " pod="openshift-ingress/e2e-worker-single-main-60d748-openshift-default-7859bd475cskdsz"
Oct 31 17:49:06.738758 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:06.738746    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e5b8fc2-0092-4c1b-b689-cce7439e1c2e-credential-socket\") pod \"e2e-worker-single-main-60d748-openshift-default-7859bd475cskdsz\" (UID: \"5e5b8fc2-0092-4c1b-b689-cce7439e1c2e\") " pod="openshift-ingress/e2e-worker-single-main-60d748-openshift-default-7859bd475cskdsz"
Oct 31 17:49:06.739160 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:06.739143    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5e5b8fc2-0092-4c1b-b689-cce7439e1c2e-istio-data\") pod \"e2e-worker-single-main-60d748-openshift-default-7859bd475cskdsz\" (UID: \"5e5b8fc2-0092-4c1b-b689-cce7439e1c2e\") " pod="openshift-ingress/e2e-worker-single-main-60d748-openshift-default-7859bd475cskdsz"
Oct 31 17:49:06.739231 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:06.739212    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e5b8fc2-0092-4c1b-b689-cce7439e1c2e-workload-socket\") pod \"e2e-worker-single-main-60d748-openshift-default-7859bd475cskdsz\" (UID: \"5e5b8fc2-0092-4c1b-b689-cce7439e1c2e\") " pod="openshift-ingress/e2e-worker-single-main-60d748-openshift-default-7859bd475cskdsz"
Oct 31 17:49:06.739389 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:06.739370    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5e5b8fc2-0092-4c1b-b689-cce7439e1c2e-istiod-ca-cert\") pod \"e2e-worker-single-main-60d748-openshift-default-7859bd475cskdsz\" (UID: \"5e5b8fc2-0092-4c1b-b689-cce7439e1c2e\") " pod="openshift-ingress/e2e-worker-single-main-60d748-openshift-default-7859bd475cskdsz"
Oct 31 17:49:06.740877 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:06.740857    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5e5b8fc2-0092-4c1b-b689-cce7439e1c2e-istio-envoy\") pod \"e2e-worker-single-main-60d748-openshift-default-7859bd475cskdsz\" (UID: \"5e5b8fc2-0092-4c1b-b689-cce7439e1c2e\") " pod="openshift-ingress/e2e-worker-single-main-60d748-openshift-default-7859bd475cskdsz"
Oct 31 17:49:06.741033 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:06.741015    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5e5b8fc2-0092-4c1b-b689-cce7439e1c2e-istio-podinfo\") pod \"e2e-worker-single-main-60d748-openshift-default-7859bd475cskdsz\" (UID: \"5e5b8fc2-0092-4c1b-b689-cce7439e1c2e\") " pod="openshift-ingress/e2e-worker-single-main-60d748-openshift-default-7859bd475cskdsz"
Oct 31 17:49:06.758752 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:06.758723    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tqf8w\" (UniqueName: \"kubernetes.io/projected/5e5b8fc2-0092-4c1b-b689-cce7439e1c2e-kube-api-access-tqf8w\") pod \"e2e-worker-single-main-60d748-openshift-default-7859bd475cskdsz\" (UID: \"5e5b8fc2-0092-4c1b-b689-cce7439e1c2e\") " pod="openshift-ingress/e2e-worker-single-main-60d748-openshift-default-7859bd475cskdsz"
Oct 31 17:49:06.759028 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:06.759011    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5e5b8fc2-0092-4c1b-b689-cce7439e1c2e-istio-token\") pod \"e2e-worker-single-main-60d748-openshift-default-7859bd475cskdsz\" (UID: \"5e5b8fc2-0092-4c1b-b689-cce7439e1c2e\") " pod="openshift-ingress/e2e-worker-single-main-60d748-openshift-default-7859bd475cskdsz"
Oct 31 17:49:06.802814 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:06.802779    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:49:06.850159 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:06.850131    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-single-main-60d748-openshift-default-7859bd475cskdsz"
Oct 31 17:49:07.165079 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:07.165056    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-single-main-60d748-openshift-default-7859bd475cskdsz"]
Oct 31 17:49:07.166885 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:49:07.166860    4276 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e5b8fc2_0092_4c1b_b689_cce7439e1c2e.slice/crio-4f2a76d3a620caf972f21348b37aa32b0f36819511b713bee38ab1618866ab11 WatchSource:0}: Error finding container 4f2a76d3a620caf972f21348b37aa32b0f36819511b713bee38ab1618866ab11: Status 404 returned error can't find the container with id 4f2a76d3a620caf972f21348b37aa32b0f36819511b713bee38ab1618866ab11
Oct 31 17:49:07.168827 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:07.168798    4276 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 17:49:07.168911 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:07.168861    4276 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 17:49:07.168911 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:07.168889    4276 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 17:49:08.128039 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:08.128004    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-60d748-openshift-default-7859bd475cskdsz" event={"ID":"5e5b8fc2-0092-4c1b-b689-cce7439e1c2e","Type":"ContainerStarted","Data":"6d9bb44b2d86f98bfd6eac3c363dc0dfbc6018c3c23066cb64f814434caddcfd"}
Oct 31 17:49:08.128215 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:08.128051    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-60d748-openshift-default-7859bd475cskdsz" event={"ID":"5e5b8fc2-0092-4c1b-b689-cce7439e1c2e","Type":"ContainerStarted","Data":"4f2a76d3a620caf972f21348b37aa32b0f36819511b713bee38ab1618866ab11"}
Oct 31 17:49:08.186338 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:08.186290    4276 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-single-main-60d748-openshift-default-7859bd475cskdsz" podStartSLOduration=2.186276869 podStartE2EDuration="2.186276869s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 17:49:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 17:49:08.184317777 +0000 UTC m=+1310.982062704" watchObservedRunningTime="2026-10-31 17:49:08.186276869 +0000 UTC m=+1310.984021788"
Oct 31 17:49:08.850620 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:08.850582    4276 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-single-main-60d748-openshift-default-7859bd475cskdsz"
Oct 31 17:49:08.855514 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:08.855487    4276 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-single-main-60d748-openshift-default-7859bd475cskdsz"
Oct 31 17:49:09.132346 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:09.132273    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-single-main-60d748-openshift-default-7859bd475cskdsz"
Oct 31 17:49:09.133023 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:09.133006    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-single-main-60d748-openshift-default-7859bd475cskdsz"
Oct 31 17:49:09.272841 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:09.272808    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:49:10.507872 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:10.507835    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:49:10.651913 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:10.651879    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:49:12.209535 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:12.209493    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:49:12.234546 ip-10-0-129-127 kubenswrapper[4276]: E1031 17:49:12.234513    4276 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tg5qk" containerName="extractor"
Oct 31 17:49:12.301001 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:12.300973    4276 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-60d748-58bcc565b56sk2"]
Oct 31 17:49:12.301412 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:12.301399    4276 memory_manager.go:365] "RemoveStaleState removing state" podUID="00caa007-32e1-4ff0-a5d5-71ab1b73eca9" containerName="discovery"
Oct 31 17:49:12.304516 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:12.304501    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-single-main-60d748-58bcc565b56sk2"
Oct 31 17:49:12.323777 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:12.323749    4276 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-single-main-60d748\""
Oct 31 17:49:12.327709 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:12.327678    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:49:12.339405 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:12.339384    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-60d748-58bcc565b56sk2"]
Oct 31 17:49:12.367750 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:12.367728    4276 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-60d748-648dfj5bj4"]
Oct 31 17:49:12.371062 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:12.371042    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-60d748-648dfj5bj4"
Oct 31 17:49:12.388073 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:12.388048    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-blwwt\" (UniqueName: \"kubernetes.io/projected/29854a6b-4cf9-47c8-a434-e9b3b3603f9b-kube-api-access-blwwt\") pod \"payload-processing-e2e-worker-single-main-60d748-58bcc565b56sk2\" (UID: \"29854a6b-4cf9-47c8-a434-e9b3b3603f9b\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-60d748-58bcc565b56sk2"
Oct 31 17:49:12.388235 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:12.388173    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/29854a6b-4cf9-47c8-a434-e9b3b3603f9b-plugins-config-volume\") pod \"payload-processing-e2e-worker-single-main-60d748-58bcc565b56sk2\" (UID: \"29854a6b-4cf9-47c8-a434-e9b3b3603f9b\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-60d748-58bcc565b56sk2"
Oct 31 17:49:12.389043 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:12.389021    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-60d748-648dfj5bj4"]
Oct 31 17:49:12.393391 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:12.393366    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-single-m-7b58b982-dockercfg-svgl7\""
Oct 31 17:49:12.489713 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:12.489633    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-blwwt\" (UniqueName: \"kubernetes.io/projected/29854a6b-4cf9-47c8-a434-e9b3b3603f9b-kube-api-access-blwwt\") pod \"payload-processing-e2e-worker-single-main-60d748-58bcc565b56sk2\" (UID: \"29854a6b-4cf9-47c8-a434-e9b3b3603f9b\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-60d748-58bcc565b56sk2"
Oct 31 17:49:12.489713 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:12.489679    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4b2afdc1-536d-4b96-a234-eb71ce244a5e-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-single-main-60d748-648dfj5bj4\" (UID: \"4b2afdc1-536d-4b96-a234-eb71ce244a5e\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-60d748-648dfj5bj4"
Oct 31 17:49:12.489947 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:12.489718    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h4b6c\" (UniqueName: \"kubernetes.io/projected/4b2afdc1-536d-4b96-a234-eb71ce244a5e-kube-api-access-h4b6c\") pod \"payload-pre-processing-e2e-worker-single-main-60d748-648dfj5bj4\" (UID: \"4b2afdc1-536d-4b96-a234-eb71ce244a5e\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-60d748-648dfj5bj4"
Oct 31 17:49:12.489947 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:12.489751    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/29854a6b-4cf9-47c8-a434-e9b3b3603f9b-plugins-config-volume\") pod \"payload-processing-e2e-worker-single-main-60d748-58bcc565b56sk2\" (UID: \"29854a6b-4cf9-47c8-a434-e9b3b3603f9b\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-60d748-58bcc565b56sk2"
Oct 31 17:49:12.490500 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:12.490474    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/29854a6b-4cf9-47c8-a434-e9b3b3603f9b-plugins-config-volume\") pod \"payload-processing-e2e-worker-single-main-60d748-58bcc565b56sk2\" (UID: \"29854a6b-4cf9-47c8-a434-e9b3b3603f9b\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-60d748-58bcc565b56sk2"
Oct 31 17:49:12.509985 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:12.509955    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-blwwt\" (UniqueName: \"kubernetes.io/projected/29854a6b-4cf9-47c8-a434-e9b3b3603f9b-kube-api-access-blwwt\") pod \"payload-processing-e2e-worker-single-main-60d748-58bcc565b56sk2\" (UID: \"29854a6b-4cf9-47c8-a434-e9b3b3603f9b\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-60d748-58bcc565b56sk2"
Oct 31 17:49:12.590780 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:12.590743    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4b2afdc1-536d-4b96-a234-eb71ce244a5e-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-single-main-60d748-648dfj5bj4\" (UID: \"4b2afdc1-536d-4b96-a234-eb71ce244a5e\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-60d748-648dfj5bj4"
Oct 31 17:49:12.590946 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:12.590822    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h4b6c\" (UniqueName: \"kubernetes.io/projected/4b2afdc1-536d-4b96-a234-eb71ce244a5e-kube-api-access-h4b6c\") pod \"payload-pre-processing-e2e-worker-single-main-60d748-648dfj5bj4\" (UID: \"4b2afdc1-536d-4b96-a234-eb71ce244a5e\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-60d748-648dfj5bj4"
Oct 31 17:49:12.591429 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:12.591408    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4b2afdc1-536d-4b96-a234-eb71ce244a5e-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-single-main-60d748-648dfj5bj4\" (UID: \"4b2afdc1-536d-4b96-a234-eb71ce244a5e\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-60d748-648dfj5bj4"
Oct 31 17:49:12.610837 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:12.610807    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h4b6c\" (UniqueName: \"kubernetes.io/projected/4b2afdc1-536d-4b96-a234-eb71ce244a5e-kube-api-access-h4b6c\") pod \"payload-pre-processing-e2e-worker-single-main-60d748-648dfj5bj4\" (UID: \"4b2afdc1-536d-4b96-a234-eb71ce244a5e\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-60d748-648dfj5bj4"
Oct 31 17:49:12.618529 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:12.618503    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-single-main-60d748-58bcc565b56sk2"
Oct 31 17:49:12.684313 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:12.684088    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-60d748-648dfj5bj4"
Oct 31 17:49:12.913365 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:12.913342    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-60d748-58bcc565b56sk2"]
Oct 31 17:49:12.915073 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:49:12.915048    4276 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29854a6b_4cf9_47c8_a434_e9b3b3603f9b.slice/crio-9c821af7931911368087259b7ed8fb5e3a5a24678f0b29fd847582fde2901eb5 WatchSource:0}: Error finding container 9c821af7931911368087259b7ed8fb5e3a5a24678f0b29fd847582fde2901eb5: Status 404 returned error can't find the container with id 9c821af7931911368087259b7ed8fb5e3a5a24678f0b29fd847582fde2901eb5
Oct 31 17:49:12.985051 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:12.985026    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-60d748-648dfj5bj4"]
Oct 31 17:49:12.986510 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:49:12.986484    4276 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b2afdc1_536d_4b96_a234_eb71ce244a5e.slice/crio-b4413bb73764171285c5120c789450d62e01696eefc332acfe360a2dff87d1ca WatchSource:0}: Error finding container b4413bb73764171285c5120c789450d62e01696eefc332acfe360a2dff87d1ca: Status 404 returned error can't find the container with id b4413bb73764171285c5120c789450d62e01696eefc332acfe360a2dff87d1ca
Oct 31 17:49:13.152546 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:13.152502    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-60d748-648dfj5bj4" event={"ID":"4b2afdc1-536d-4b96-a234-eb71ce244a5e","Type":"ContainerStarted","Data":"630765ab4c8087841d5e911a89055fe4703f46b5229bbb119a8ac0c3166a108a"}
Oct 31 17:49:13.152724 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:13.152695    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-60d748-648dfj5bj4"
Oct 31 17:49:13.152797 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:13.152742    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-60d748-648dfj5bj4" event={"ID":"4b2afdc1-536d-4b96-a234-eb71ce244a5e","Type":"ContainerStarted","Data":"b4413bb73764171285c5120c789450d62e01696eefc332acfe360a2dff87d1ca"}
Oct 31 17:49:13.154021 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:13.153997    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-single-main-60d748-58bcc565b56sk2" event={"ID":"29854a6b-4cf9-47c8-a434-e9b3b3603f9b","Type":"ContainerStarted","Data":"839f7e489eceab005ebdd9279feff8eed11d70e7780ba7dfe95f29949af292e7"}
Oct 31 17:49:13.154149 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:13.154040    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-single-main-60d748-58bcc565b56sk2" event={"ID":"29854a6b-4cf9-47c8-a434-e9b3b3603f9b","Type":"ContainerStarted","Data":"9c821af7931911368087259b7ed8fb5e3a5a24678f0b29fd847582fde2901eb5"}
Oct 31 17:49:13.154149 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:13.154067    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-single-main-60d748-58bcc565b56sk2"
Oct 31 17:49:13.199614 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:13.199562    4276 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-60d748-648dfj5bj4" podStartSLOduration=1.199549673 podStartE2EDuration="1.199549673s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 17:49:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 17:49:13.199324639 +0000 UTC m=+1315.997069573" watchObservedRunningTime="2026-10-31 17:49:13.199549673 +0000 UTC m=+1315.997294600"
Oct 31 17:49:13.249013 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:13.248938    4276 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-single-main-60d748-58bcc565b56sk2" podStartSLOduration=1.248926077 podStartE2EDuration="1.248926077s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 17:49:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 17:49:13.248356247 +0000 UTC m=+1316.046101175" watchObservedRunningTime="2026-10-31 17:49:13.248926077 +0000 UTC m=+1316.046671038"
Oct 31 17:49:13.581817 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:13.581781    4276 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-60d748-66d5c9c969-b95kl"]
Oct 31 17:49:13.585214 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:13.585194    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-60d748-66d5c9c969-b95kl"
Oct 31 17:49:13.603476 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:13.603426    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-60d748-66d5c9c969-b95kl"]
Oct 31 17:49:13.604219 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:13.604200    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-single-main-60d748-serving-cert\""
Oct 31 17:49:13.684081 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:13.684032    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:49:13.701539 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:13.701511    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zpwql\" (UniqueName: \"kubernetes.io/projected/025470d4-6fca-4ab3-b8f2-0067cb018b5f-kube-api-access-zpwql\") pod \"maas-api-e2e-worker-single-main-60d748-66d5c9c969-b95kl\" (UID: \"025470d4-6fca-4ab3-b8f2-0067cb018b5f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-60d748-66d5c9c969-b95kl"
Oct 31 17:49:13.701695 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:13.701549    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/025470d4-6fca-4ab3-b8f2-0067cb018b5f-maas-api-tls\") pod \"maas-api-e2e-worker-single-main-60d748-66d5c9c969-b95kl\" (UID: \"025470d4-6fca-4ab3-b8f2-0067cb018b5f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-60d748-66d5c9c969-b95kl"
Oct 31 17:49:13.701695 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:13.701574    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/025470d4-6fca-4ab3-b8f2-0067cb018b5f-metrics-certs\") pod \"maas-api-e2e-worker-single-main-60d748-66d5c9c969-b95kl\" (UID: \"025470d4-6fca-4ab3-b8f2-0067cb018b5f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-60d748-66d5c9c969-b95kl"
Oct 31 17:49:13.701778 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:13.701759    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/025470d4-6fca-4ab3-b8f2-0067cb018b5f-openshift-service-ca\") pod \"maas-api-e2e-worker-single-main-60d748-66d5c9c969-b95kl\" (UID: \"025470d4-6fca-4ab3-b8f2-0067cb018b5f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-60d748-66d5c9c969-b95kl"
Oct 31 17:49:13.796995 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:13.796966    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:49:13.803184 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:13.803161    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/025470d4-6fca-4ab3-b8f2-0067cb018b5f-metrics-certs\") pod \"maas-api-e2e-worker-single-main-60d748-66d5c9c969-b95kl\" (UID: \"025470d4-6fca-4ab3-b8f2-0067cb018b5f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-60d748-66d5c9c969-b95kl"
Oct 31 17:49:13.803281 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:13.803260    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/025470d4-6fca-4ab3-b8f2-0067cb018b5f-openshift-service-ca\") pod \"maas-api-e2e-worker-single-main-60d748-66d5c9c969-b95kl\" (UID: \"025470d4-6fca-4ab3-b8f2-0067cb018b5f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-60d748-66d5c9c969-b95kl"
Oct 31 17:49:13.803331 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:13.803292    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zpwql\" (UniqueName: \"kubernetes.io/projected/025470d4-6fca-4ab3-b8f2-0067cb018b5f-kube-api-access-zpwql\") pod \"maas-api-e2e-worker-single-main-60d748-66d5c9c969-b95kl\" (UID: \"025470d4-6fca-4ab3-b8f2-0067cb018b5f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-60d748-66d5c9c969-b95kl"
Oct 31 17:49:13.803331 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:13.803316    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/025470d4-6fca-4ab3-b8f2-0067cb018b5f-maas-api-tls\") pod \"maas-api-e2e-worker-single-main-60d748-66d5c9c969-b95kl\" (UID: \"025470d4-6fca-4ab3-b8f2-0067cb018b5f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-60d748-66d5c9c969-b95kl"
Oct 31 17:49:13.803934 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:13.803909    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/025470d4-6fca-4ab3-b8f2-0067cb018b5f-openshift-service-ca\") pod \"maas-api-e2e-worker-single-main-60d748-66d5c9c969-b95kl\" (UID: \"025470d4-6fca-4ab3-b8f2-0067cb018b5f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-60d748-66d5c9c969-b95kl"
Oct 31 17:49:13.805551 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:13.805530    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/025470d4-6fca-4ab3-b8f2-0067cb018b5f-metrics-certs\") pod \"maas-api-e2e-worker-single-main-60d748-66d5c9c969-b95kl\" (UID: \"025470d4-6fca-4ab3-b8f2-0067cb018b5f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-60d748-66d5c9c969-b95kl"
Oct 31 17:49:13.805668 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:13.805571    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/025470d4-6fca-4ab3-b8f2-0067cb018b5f-maas-api-tls\") pod \"maas-api-e2e-worker-single-main-60d748-66d5c9c969-b95kl\" (UID: \"025470d4-6fca-4ab3-b8f2-0067cb018b5f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-60d748-66d5c9c969-b95kl"
Oct 31 17:49:13.823219 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:13.823191    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zpwql\" (UniqueName: \"kubernetes.io/projected/025470d4-6fca-4ab3-b8f2-0067cb018b5f-kube-api-access-zpwql\") pod \"maas-api-e2e-worker-single-main-60d748-66d5c9c969-b95kl\" (UID: \"025470d4-6fca-4ab3-b8f2-0067cb018b5f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-60d748-66d5c9c969-b95kl"
Oct 31 17:49:13.899404 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:13.899310    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-60d748-66d5c9c969-b95kl"
Oct 31 17:49:13.906393 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:13.906366    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:49:14.196584 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:14.196560    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-60d748-66d5c9c969-b95kl"]
Oct 31 17:49:14.197889 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:49:14.197864    4276 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod025470d4_6fca_4ab3_b8f2_0067cb018b5f.slice/crio-cf89d73684783d196ba557b002ca2b5806c06379890cc4c57e61694ed228b486 WatchSource:0}: Error finding container cf89d73684783d196ba557b002ca2b5806c06379890cc4c57e61694ed228b486: Status 404 returned error can't find the container with id cf89d73684783d196ba557b002ca2b5806c06379890cc4c57e61694ed228b486
Oct 31 17:49:14.199286 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:14.199270    4276 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 31 17:49:14.255423 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:14.255394    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:49:15.165455 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:15.165414    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-60d748-66d5c9c969-b95kl" event={"ID":"025470d4-6fca-4ab3-b8f2-0067cb018b5f","Type":"ContainerStarted","Data":"5b98e63b5e923373ebd91f786274a534a1cf9892f45a2fb0a6cccd7064bcc108"}
Oct 31 17:49:15.165654 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:15.165499    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-60d748-66d5c9c969-b95kl" event={"ID":"025470d4-6fca-4ab3-b8f2-0067cb018b5f","Type":"ContainerStarted","Data":"cf89d73684783d196ba557b002ca2b5806c06379890cc4c57e61694ed228b486"}
Oct 31 17:49:15.165654 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:15.165528    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-60d748-66d5c9c969-b95kl"
Oct 31 17:49:15.209757 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:15.209699    4276 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-60d748-66d5c9c969-b95kl" podStartSLOduration=1.8132433780000001 podStartE2EDuration="2.209680485s" totalImagesPullingTime="396.437107ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 17:49:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 17:49:15.207170299 +0000 UTC m=+1318.004915227" watchObservedRunningTime="2026-10-31 17:49:15.209680485 +0000 UTC m=+1318.007425414"
Oct 31 17:49:15.515530 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:15.515430    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:49:17.242036 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:17.241995    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:49:17.671214 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:17.671182    4276 scope.go:122] "RemoveContainer" containerID="3dc6908f46c76cdd2f58bb838c9aa84727471f93069a53ce36c5a8b0d9cfc74a"
Oct 31 17:49:17.681313 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:17.681294    4276 scope.go:122] "RemoveContainer" containerID="823176f79415fffcda43732788e03905152faf95571fd7a2f72569d59ccde231"
Oct 31 17:49:19.037182 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:19.037147    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:49:20.870896 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:20.870862    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:49:21.181469 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:21.181373    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-60d748-66d5c9c969-b95kl"
Oct 31 17:49:22.770470 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:22.766146    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:49:24.162009 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:24.161969    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-60d748-648dfj5bj4"
Oct 31 17:49:24.162512 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:24.162047    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-single-main-60d748-58bcc565b56sk2"
Oct 31 17:49:24.632718 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:24.632684    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:49:25.173664 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:25.173623    4276 patch_prober.go:29] interesting pod/e2e-worker-single-main-60d748-openshift-default-7859bd475cskdsz container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.81:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 17:49:25.174055 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:25.173688    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-single-main-60d748-openshift-default-7859bd475cskdsz" podUID="5e5b8fc2-0092-4c1b-b689-cce7439e1c2e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.81:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 17:49:25.297697 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:25.297616    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:49:25.868886 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:25.868837    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:49:25.985182 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:25.985128    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:49:26.101003 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:26.100965    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:49:26.333528 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:26.333498    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:49:27.225465 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:27.225418    4276 generic.go:356] "Generic (PLEG): container finished" podID="5e5b8fc2-0092-4c1b-b689-cce7439e1c2e" containerID="6d9bb44b2d86f98bfd6eac3c363dc0dfbc6018c3c23066cb64f814434caddcfd" exitCode=0
Oct 31 17:49:27.225702 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:27.225480    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-60d748-openshift-default-7859bd475cskdsz" event={"ID":"5e5b8fc2-0092-4c1b-b689-cce7439e1c2e","Type":"ContainerDied","Data":"6d9bb44b2d86f98bfd6eac3c363dc0dfbc6018c3c23066cb64f814434caddcfd"}
Oct 31 17:49:27.225834 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:27.225820    4276 scope.go:122] "RemoveContainer" containerID="6d9bb44b2d86f98bfd6eac3c363dc0dfbc6018c3c23066cb64f814434caddcfd"
Oct 31 17:49:27.227006 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:27.226977    4276 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 17:49:27.227067 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:27.227051    4276 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 17:49:27.227102 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:27.227082    4276 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 17:49:28.139217 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:28.139183    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:49:28.232669 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:28.232638    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-60d748-openshift-default-7859bd475cskdsz" event={"ID":"5e5b8fc2-0092-4c1b-b689-cce7439e1c2e","Type":"ContainerStarted","Data":"e15b20a4da030c3b984cbaaa17ebc50d9fa0fd26c27ccd784d1b04a3c8b22a2b"}
Oct 31 17:49:28.746167 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:28.746136    4276 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-single-main-60d748/simulator-single-main-60d748-kserve-67859fd6cc-cg9g2"]
Oct 31 17:49:28.749718 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:28.749698    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-single-main-60d748/simulator-single-main-60d748-kserve-67859fd6cc-cg9g2"
Oct 31 17:49:28.778475 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:28.768777    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-single-main-60d748/simulator-single-main-60d748-kserve-67859fd6cc-cg9g2"]
Oct 31 17:49:28.778698 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:28.776740    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-single-main-60d748\"/\"simulator-single-main-60d748-kserve-self-signed-certs\""
Oct 31 17:49:28.779599 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:28.779577    4276 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-single-main-60d748\"/\"kube-root-ca.crt\""
Oct 31 17:49:28.779780 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:28.779628    4276 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-single-main-60d748\"/\"openshift-service-ca.crt\""
Oct 31 17:49:28.779863 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:28.779680    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-single-main-60d748\"/\"default-dockercfg-xdhw2\""
Oct 31 17:49:28.842908 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:28.842881    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:49:28.874795 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:28.874771    4276 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-single-main-60d748-openshift-default-7859bd475cskdsz"
Oct 31 17:49:28.875125 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:28.875042    4276 patch_prober.go:29] interesting pod/e2e-worker-single-main-60d748-openshift-default-7859bd475cskdsz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused" start-of-body=
Oct 31 17:49:28.875287 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:28.875264    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-60d748-openshift-default-7859bd475cskdsz" podUID="5e5b8fc2-0092-4c1b-b689-cce7439e1c2e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused"
Oct 31 17:49:28.885584 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:28.885560    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/397a73f5-6e20-46fa-80e6-5fb061a196f2-dshm\") pod \"simulator-single-main-60d748-kserve-67859fd6cc-cg9g2\" (UID: \"397a73f5-6e20-46fa-80e6-5fb061a196f2\") " pod="e2e-models-e2e-worker-single-main-60d748/simulator-single-main-60d748-kserve-67859fd6cc-cg9g2"
Oct 31 17:49:28.885691 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:28.885600    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/397a73f5-6e20-46fa-80e6-5fb061a196f2-home\") pod \"simulator-single-main-60d748-kserve-67859fd6cc-cg9g2\" (UID: \"397a73f5-6e20-46fa-80e6-5fb061a196f2\") " pod="e2e-models-e2e-worker-single-main-60d748/simulator-single-main-60d748-kserve-67859fd6cc-cg9g2"
Oct 31 17:49:28.885736 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:28.885684    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/397a73f5-6e20-46fa-80e6-5fb061a196f2-tls-certs\") pod \"simulator-single-main-60d748-kserve-67859fd6cc-cg9g2\" (UID: \"397a73f5-6e20-46fa-80e6-5fb061a196f2\") " pod="e2e-models-e2e-worker-single-main-60d748/simulator-single-main-60d748-kserve-67859fd6cc-cg9g2"
Oct 31 17:49:28.885795 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:28.885780    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/397a73f5-6e20-46fa-80e6-5fb061a196f2-tmp-dir\") pod \"simulator-single-main-60d748-kserve-67859fd6cc-cg9g2\" (UID: \"397a73f5-6e20-46fa-80e6-5fb061a196f2\") " pod="e2e-models-e2e-worker-single-main-60d748/simulator-single-main-60d748-kserve-67859fd6cc-cg9g2"
Oct 31 17:49:28.885834 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:28.885820    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vzbll\" (UniqueName: \"kubernetes.io/projected/397a73f5-6e20-46fa-80e6-5fb061a196f2-kube-api-access-vzbll\") pod \"simulator-single-main-60d748-kserve-67859fd6cc-cg9g2\" (UID: \"397a73f5-6e20-46fa-80e6-5fb061a196f2\") " pod="e2e-models-e2e-worker-single-main-60d748/simulator-single-main-60d748-kserve-67859fd6cc-cg9g2"
Oct 31 17:49:28.885933 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:28.885915    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/397a73f5-6e20-46fa-80e6-5fb061a196f2-model-cache\") pod \"simulator-single-main-60d748-kserve-67859fd6cc-cg9g2\" (UID: \"397a73f5-6e20-46fa-80e6-5fb061a196f2\") " pod="e2e-models-e2e-worker-single-main-60d748/simulator-single-main-60d748-kserve-67859fd6cc-cg9g2"
Oct 31 17:49:28.987465 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:28.986690    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/397a73f5-6e20-46fa-80e6-5fb061a196f2-model-cache\") pod \"simulator-single-main-60d748-kserve-67859fd6cc-cg9g2\" (UID: \"397a73f5-6e20-46fa-80e6-5fb061a196f2\") " pod="e2e-models-e2e-worker-single-main-60d748/simulator-single-main-60d748-kserve-67859fd6cc-cg9g2"
Oct 31 17:49:28.987465 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:28.986748    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/397a73f5-6e20-46fa-80e6-5fb061a196f2-dshm\") pod \"simulator-single-main-60d748-kserve-67859fd6cc-cg9g2\" (UID: \"397a73f5-6e20-46fa-80e6-5fb061a196f2\") " pod="e2e-models-e2e-worker-single-main-60d748/simulator-single-main-60d748-kserve-67859fd6cc-cg9g2"
Oct 31 17:49:28.987465 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:28.986790    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/397a73f5-6e20-46fa-80e6-5fb061a196f2-home\") pod \"simulator-single-main-60d748-kserve-67859fd6cc-cg9g2\" (UID: \"397a73f5-6e20-46fa-80e6-5fb061a196f2\") " pod="e2e-models-e2e-worker-single-main-60d748/simulator-single-main-60d748-kserve-67859fd6cc-cg9g2"
Oct 31 17:49:28.987465 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:28.986833    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/397a73f5-6e20-46fa-80e6-5fb061a196f2-tls-certs\") pod \"simulator-single-main-60d748-kserve-67859fd6cc-cg9g2\" (UID: \"397a73f5-6e20-46fa-80e6-5fb061a196f2\") " pod="e2e-models-e2e-worker-single-main-60d748/simulator-single-main-60d748-kserve-67859fd6cc-cg9g2"
Oct 31 17:49:28.987465 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:28.986889    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/397a73f5-6e20-46fa-80e6-5fb061a196f2-tmp-dir\") pod \"simulator-single-main-60d748-kserve-67859fd6cc-cg9g2\" (UID: \"397a73f5-6e20-46fa-80e6-5fb061a196f2\") " pod="e2e-models-e2e-worker-single-main-60d748/simulator-single-main-60d748-kserve-67859fd6cc-cg9g2"
Oct 31 17:49:28.987465 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:28.986940    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vzbll\" (UniqueName: \"kubernetes.io/projected/397a73f5-6e20-46fa-80e6-5fb061a196f2-kube-api-access-vzbll\") pod \"simulator-single-main-60d748-kserve-67859fd6cc-cg9g2\" (UID: \"397a73f5-6e20-46fa-80e6-5fb061a196f2\") " pod="e2e-models-e2e-worker-single-main-60d748/simulator-single-main-60d748-kserve-67859fd6cc-cg9g2"
Oct 31 17:49:28.988032 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:28.987747    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/397a73f5-6e20-46fa-80e6-5fb061a196f2-model-cache\") pod \"simulator-single-main-60d748-kserve-67859fd6cc-cg9g2\" (UID: \"397a73f5-6e20-46fa-80e6-5fb061a196f2\") " pod="e2e-models-e2e-worker-single-main-60d748/simulator-single-main-60d748-kserve-67859fd6cc-cg9g2"
Oct 31 17:49:28.992571 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:28.988565    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/397a73f5-6e20-46fa-80e6-5fb061a196f2-home\") pod \"simulator-single-main-60d748-kserve-67859fd6cc-cg9g2\" (UID: \"397a73f5-6e20-46fa-80e6-5fb061a196f2\") " pod="e2e-models-e2e-worker-single-main-60d748/simulator-single-main-60d748-kserve-67859fd6cc-cg9g2"
Oct 31 17:49:28.992571 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:28.988788    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/397a73f5-6e20-46fa-80e6-5fb061a196f2-tmp-dir\") pod \"simulator-single-main-60d748-kserve-67859fd6cc-cg9g2\" (UID: \"397a73f5-6e20-46fa-80e6-5fb061a196f2\") " pod="e2e-models-e2e-worker-single-main-60d748/simulator-single-main-60d748-kserve-67859fd6cc-cg9g2"
Oct 31 17:49:28.992571 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:28.992320    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/397a73f5-6e20-46fa-80e6-5fb061a196f2-tls-certs\") pod \"simulator-single-main-60d748-kserve-67859fd6cc-cg9g2\" (UID: \"397a73f5-6e20-46fa-80e6-5fb061a196f2\") " pod="e2e-models-e2e-worker-single-main-60d748/simulator-single-main-60d748-kserve-67859fd6cc-cg9g2"
Oct 31 17:49:28.992571 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:28.992546    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/397a73f5-6e20-46fa-80e6-5fb061a196f2-dshm\") pod \"simulator-single-main-60d748-kserve-67859fd6cc-cg9g2\" (UID: \"397a73f5-6e20-46fa-80e6-5fb061a196f2\") " pod="e2e-models-e2e-worker-single-main-60d748/simulator-single-main-60d748-kserve-67859fd6cc-cg9g2"
Oct 31 17:49:29.007108 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:29.007045    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vzbll\" (UniqueName: \"kubernetes.io/projected/397a73f5-6e20-46fa-80e6-5fb061a196f2-kube-api-access-vzbll\") pod \"simulator-single-main-60d748-kserve-67859fd6cc-cg9g2\" (UID: \"397a73f5-6e20-46fa-80e6-5fb061a196f2\") " pod="e2e-models-e2e-worker-single-main-60d748/simulator-single-main-60d748-kserve-67859fd6cc-cg9g2"
Oct 31 17:49:29.089978 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:29.089948    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-single-main-60d748/simulator-single-main-60d748-kserve-67859fd6cc-cg9g2"
Oct 31 17:49:29.390580 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:29.390554    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-single-main-60d748/simulator-single-main-60d748-kserve-67859fd6cc-cg9g2"]
Oct 31 17:49:29.392331 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:49:29.392306    4276 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod397a73f5_6e20_46fa_80e6_5fb061a196f2.slice/crio-fea43b3b2fb336d6f02241fd06515fc24b17c3d49b9633c6d3f417f5eb207bbf WatchSource:0}: Error finding container fea43b3b2fb336d6f02241fd06515fc24b17c3d49b9633c6d3f417f5eb207bbf: Status 404 returned error can't find the container with id fea43b3b2fb336d6f02241fd06515fc24b17c3d49b9633c6d3f417f5eb207bbf
Oct 31 17:49:29.863467 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:29.858834    4276 patch_prober.go:29] interesting pod/e2e-worker-single-main-60d748-openshift-default-7859bd475cskdsz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused" start-of-body=
Oct 31 17:49:29.863467 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:29.858896    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-60d748-openshift-default-7859bd475cskdsz" podUID="5e5b8fc2-0092-4c1b-b689-cce7439e1c2e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused"
Oct 31 17:49:29.957898 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:29.957781    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:49:30.243467 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:30.243361    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-single-main-60d748/simulator-single-main-60d748-kserve-67859fd6cc-cg9g2" event={"ID":"397a73f5-6e20-46fa-80e6-5fb061a196f2","Type":"ContainerStarted","Data":"fa251619d36644298978c9c0fdc61b4e80d402663b902dc98f736ea418ba556e"}
Oct 31 17:49:30.243467 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:30.243404    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-single-main-60d748/simulator-single-main-60d748-kserve-67859fd6cc-cg9g2" event={"ID":"397a73f5-6e20-46fa-80e6-5fb061a196f2","Type":"ContainerStarted","Data":"fea43b3b2fb336d6f02241fd06515fc24b17c3d49b9633c6d3f417f5eb207bbf"}
Oct 31 17:49:30.293092 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:30.293039    4276 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-single-main-60d748/simulator-single-main-60d748-kserve-67859fd6cc-cg9g2" podStartSLOduration=2.293023552 podStartE2EDuration="2.293023552s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 17:49:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 17:49:30.291460916 +0000 UTC m=+1333.089205835" watchObservedRunningTime="2026-10-31 17:49:30.293023552 +0000 UTC m=+1333.090768481"
Oct 31 17:49:30.865468 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:30.857779    4276 patch_prober.go:29] interesting pod/e2e-worker-single-main-60d748-openshift-default-7859bd475cskdsz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused" start-of-body=
Oct 31 17:49:30.865468 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:30.857834    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-60d748-openshift-default-7859bd475cskdsz" podUID="5e5b8fc2-0092-4c1b-b689-cce7439e1c2e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused"
Oct 31 17:49:31.091017 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:31.090985    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-single-main-60d748/simulator-single-main-60d748-kserve-67859fd6cc-cg9g2"
Oct 31 17:49:31.874467 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:31.865869    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:49:31.875586 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:31.875554    4276 patch_prober.go:29] interesting pod/e2e-worker-single-main-60d748-openshift-default-7859bd475cskdsz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused" start-of-body=
Oct 31 17:49:31.875706 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:31.875611    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-60d748-openshift-default-7859bd475cskdsz" podUID="5e5b8fc2-0092-4c1b-b689-cce7439e1c2e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused"
Oct 31 17:49:32.861466 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:32.854790    4276 patch_prober.go:29] interesting pod/e2e-worker-single-main-60d748-openshift-default-7859bd475cskdsz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused" start-of-body=
Oct 31 17:49:32.861466 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:32.854860    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-60d748-openshift-default-7859bd475cskdsz" podUID="5e5b8fc2-0092-4c1b-b689-cce7439e1c2e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused"
Oct 31 17:49:33.641989 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:33.641959    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:49:33.854183 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:33.854158    4276 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-single-main-60d748-openshift-default-7859bd475cskdsz"
Oct 31 17:49:33.854482 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:33.854457    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-single-main-60d748-openshift-default-7859bd475cskdsz"
Oct 31 17:49:33.855197 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:33.855182    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-single-main-60d748-openshift-default-7859bd475cskdsz"
Oct 31 17:49:35.315950 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:35.315912    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:49:36.943700 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:36.943666    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:49:38.711768 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:38.711736    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:49:39.091117 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:39.091081    4276 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-single-main-60d748/simulator-single-main-60d748-kserve-67859fd6cc-cg9g2"
Oct 31 17:49:39.108540 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:39.108511    4276 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-single-main-60d748/simulator-single-main-60d748-kserve-67859fd6cc-cg9g2"
Oct 31 17:49:39.324538 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:39.324506    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-single-main-60d748/simulator-single-main-60d748-kserve-67859fd6cc-cg9g2"
Oct 31 17:49:40.412866 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:40.412823    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:49:42.220327 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:42.220295    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:49:42.234185 ip-10-0-129-127 kubenswrapper[4276]: E1031 17:49:42.234152    4276 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tg5qk" containerName="extractor"
Oct 31 17:49:43.918093 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:43.918057    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:49:45.640843 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:45.640809    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:49:47.426267 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:47.426230    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:49:49.118612 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:49.118572    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:49:50.848015 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:50.847984    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:49:52.522987 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:52.522951    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:49:54.299869 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:54.299831    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:49:56.017636 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:56.017604    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:49:57.636979 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:57.636939    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:49:59.427504 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:49:59.427471    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:50:01.129614 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:01.129578    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:50:03.000038 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:03.000006    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:50:04.471420 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:04.471386    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:50:04.608788 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:04.608761    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:50:06.319178 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:06.319147    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:50:08.030900 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:08.030868    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:50:09.378737 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:09.378709    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:50:09.726356 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:09.726265    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:50:10.568675 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:10.568648    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:50:11.596300 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:11.596263    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:50:12.233832 ip-10-0-129-127 kubenswrapper[4276]: E1031 17:50:12.233797    4276 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tg5qk" containerName="extractor"
Oct 31 17:50:15.262518 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:15.262487    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:50:15.465140 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:15.465108    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:50:15.667332 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:15.667297    4276 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-single-main-60d748/simulator-single-main-60d748-kserve-67859fd6cc-cg9g2"]
Oct 31 17:50:15.752740 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:15.752706    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:50:16.923461 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:16.923414    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:50:18.622676 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:18.622624    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:50:20.320613 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:20.320571    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:50:21.252668 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:21.252634    4276 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-60d748-58bcc565b56sk2"]
Oct 31 17:50:21.287607 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:21.287576    4276 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-60d748-648dfj5bj4"]
Oct 31 17:50:21.413191 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:21.413144    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:50:21.521158 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:21.521073    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:50:22.020074 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:22.020040    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:50:23.365953 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:23.365924    4276 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-60d748-66d5c9c969-b95kl"]
Oct 31 17:50:23.366354 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:23.366108    4276 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-60d748-66d5c9c969-b95kl" podUID="025470d4-6fca-4ab3-b8f2-0067cb018b5f" containerName="maas-api" containerID="cri-o://5b98e63b5e923373ebd91f786274a534a1cf9892f45a2fb0a6cccd7064bcc108" gracePeriod=30
Oct 31 17:50:23.445213 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:23.445173    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:50:23.833505 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:23.833476    4276 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-60d748-66d5c9c969-b95kl"
Oct 31 17:50:23.920336 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:23.920247    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/025470d4-6fca-4ab3-b8f2-0067cb018b5f-metrics-certs\" (UniqueName: \"kubernetes.io/secret/025470d4-6fca-4ab3-b8f2-0067cb018b5f-metrics-certs\") pod \"025470d4-6fca-4ab3-b8f2-0067cb018b5f\" (UID: \"025470d4-6fca-4ab3-b8f2-0067cb018b5f\") "
Oct 31 17:50:23.920523 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:23.920341    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/025470d4-6fca-4ab3-b8f2-0067cb018b5f-kube-api-access-zpwql\" (UniqueName: \"kubernetes.io/projected/025470d4-6fca-4ab3-b8f2-0067cb018b5f-kube-api-access-zpwql\") pod \"025470d4-6fca-4ab3-b8f2-0067cb018b5f\" (UID: \"025470d4-6fca-4ab3-b8f2-0067cb018b5f\") "
Oct 31 17:50:23.920523 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:23.920407    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/025470d4-6fca-4ab3-b8f2-0067cb018b5f-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/025470d4-6fca-4ab3-b8f2-0067cb018b5f-maas-api-tls\") pod \"025470d4-6fca-4ab3-b8f2-0067cb018b5f\" (UID: \"025470d4-6fca-4ab3-b8f2-0067cb018b5f\") "
Oct 31 17:50:23.920656 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:23.920522    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/025470d4-6fca-4ab3-b8f2-0067cb018b5f-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/025470d4-6fca-4ab3-b8f2-0067cb018b5f-openshift-service-ca\") pod \"025470d4-6fca-4ab3-b8f2-0067cb018b5f\" (UID: \"025470d4-6fca-4ab3-b8f2-0067cb018b5f\") "
Oct 31 17:50:23.920916 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:23.920888    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/025470d4-6fca-4ab3-b8f2-0067cb018b5f-openshift-service-ca" pod "025470d4-6fca-4ab3-b8f2-0067cb018b5f" (UID: "025470d4-6fca-4ab3-b8f2-0067cb018b5f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 17:50:23.922688 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:23.922658    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/025470d4-6fca-4ab3-b8f2-0067cb018b5f-maas-api-tls" pod "025470d4-6fca-4ab3-b8f2-0067cb018b5f" (UID: "025470d4-6fca-4ab3-b8f2-0067cb018b5f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 17:50:23.922823 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:23.922805    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/025470d4-6fca-4ab3-b8f2-0067cb018b5f-metrics-certs" pod "025470d4-6fca-4ab3-b8f2-0067cb018b5f" (UID: "025470d4-6fca-4ab3-b8f2-0067cb018b5f"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 17:50:23.922871 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:23.922826    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/025470d4-6fca-4ab3-b8f2-0067cb018b5f-kube-api-access-zpwql" pod "025470d4-6fca-4ab3-b8f2-0067cb018b5f" (UID: "025470d4-6fca-4ab3-b8f2-0067cb018b5f"). InnerVolumeSpecName "kube-api-access-zpwql". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 17:50:23.995974 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:23.995939    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:50:24.022327 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:24.022294    4276 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/025470d4-6fca-4ab3-b8f2-0067cb018b5f-openshift-service-ca\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:50:24.022327 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:24.022322    4276 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/025470d4-6fca-4ab3-b8f2-0067cb018b5f-metrics-certs\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:50:24.022327 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:24.022332    4276 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zpwql\" (UniqueName: \"kubernetes.io/projected/025470d4-6fca-4ab3-b8f2-0067cb018b5f-kube-api-access-zpwql\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:50:24.022594 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:24.022343    4276 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/025470d4-6fca-4ab3-b8f2-0067cb018b5f-maas-api-tls\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:50:24.123391 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:24.123358    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:50:24.528482 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:24.528437    4276 generic.go:356] "Generic (PLEG): container finished" podID="025470d4-6fca-4ab3-b8f2-0067cb018b5f" containerID="5b98e63b5e923373ebd91f786274a534a1cf9892f45a2fb0a6cccd7064bcc108" exitCode=0
Oct 31 17:50:24.528899 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:24.528503    4276 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-60d748-66d5c9c969-b95kl"
Oct 31 17:50:24.528899 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:24.528543    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-60d748-66d5c9c969-b95kl" event={"ID":"025470d4-6fca-4ab3-b8f2-0067cb018b5f","Type":"ContainerDied","Data":"5b98e63b5e923373ebd91f786274a534a1cf9892f45a2fb0a6cccd7064bcc108"}
Oct 31 17:50:24.528899 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:24.528596    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-60d748-66d5c9c969-b95kl" event={"ID":"025470d4-6fca-4ab3-b8f2-0067cb018b5f","Type":"ContainerDied","Data":"cf89d73684783d196ba557b002ca2b5806c06379890cc4c57e61694ed228b486"}
Oct 31 17:50:24.528899 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:24.528621    4276 scope.go:122] "RemoveContainer" containerID="5b98e63b5e923373ebd91f786274a534a1cf9892f45a2fb0a6cccd7064bcc108"
Oct 31 17:50:24.538281 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:24.538264    4276 scope.go:122] "RemoveContainer" containerID="5b98e63b5e923373ebd91f786274a534a1cf9892f45a2fb0a6cccd7064bcc108"
Oct 31 17:50:24.538611 ip-10-0-129-127 kubenswrapper[4276]: E1031 17:50:24.538593    4276 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5b98e63b5e923373ebd91f786274a534a1cf9892f45a2fb0a6cccd7064bcc108\": container with ID starting with 5b98e63b5e923373ebd91f786274a534a1cf9892f45a2fb0a6cccd7064bcc108 not found: ID does not exist" containerID="5b98e63b5e923373ebd91f786274a534a1cf9892f45a2fb0a6cccd7064bcc108"
Oct 31 17:50:24.538835 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:24.538618    4276 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5b98e63b5e923373ebd91f786274a534a1cf9892f45a2fb0a6cccd7064bcc108"} err="failed to get container status \"5b98e63b5e923373ebd91f786274a534a1cf9892f45a2fb0a6cccd7064bcc108\": rpc error: code = NotFound desc = could not find container \"5b98e63b5e923373ebd91f786274a534a1cf9892f45a2fb0a6cccd7064bcc108\": container with ID starting with 5b98e63b5e923373ebd91f786274a534a1cf9892f45a2fb0a6cccd7064bcc108 not found: ID does not exist"
Oct 31 17:50:24.582459 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:24.582418    4276 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-60d748-66d5c9c969-b95kl"]
Oct 31 17:50:24.591791 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:24.591762    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:50:24.596029 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:24.596004    4276 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-60d748-66d5c9c969-b95kl"]
Oct 31 17:50:25.367404 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:25.367373    4276 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="025470d4-6fca-4ab3-b8f2-0067cb018b5f" path="/var/lib/kubelet/pods/025470d4-6fca-4ab3-b8f2-0067cb018b5f/volumes"
Oct 31 17:50:26.253720 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:26.253667    4276 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-single-main-60d748-58bcc565b56sk2" podUID="29854a6b-4cf9-47c8-a434-e9b3b3603f9b" containerName="payload-processing" containerID="cri-o://839f7e489eceab005ebdd9279feff8eed11d70e7780ba7dfe95f29949af292e7" gracePeriod=25
Oct 31 17:50:26.288732 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:26.288696    4276 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-60d748-648dfj5bj4" podUID="4b2afdc1-536d-4b96-a234-eb71ce244a5e" containerName="payload-pre-processing" containerID="cri-o://630765ab4c8087841d5e911a89055fe4703f46b5229bbb119a8ac0c3166a108a" gracePeriod=25
Oct 31 17:50:30.525097 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:30.525066    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:50:30.653777 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:30.653702    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:50:30.687719 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:30.687681    4276 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-single-main-60d748/simulator-single-main-60d748-kserve-67859fd6cc-cg9g2" podUID="397a73f5-6e20-46fa-80e6-5fb061a196f2" containerName="main" containerID="cri-o://fa251619d36644298978c9c0fdc61b4e80d402663b902dc98f736ea418ba556e" gracePeriod=45
Oct 31 17:50:30.811549 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:30.810288    4276 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-single-main-60d748-openshift-default-7859bd475cskdsz"]
Oct 31 17:50:30.811549 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:30.810512    4276 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-single-main-60d748-openshift-default-7859bd475cskdsz" podUID="5e5b8fc2-0092-4c1b-b689-cce7439e1c2e" containerName="istio-proxy" containerID="cri-o://e15b20a4da030c3b984cbaaa17ebc50d9fa0fd26c27ccd784d1b04a3c8b22a2b" gracePeriod=30
Oct 31 17:50:30.865726 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:30.865692    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:50:30.973766 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:30.973743    4276 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-single-main-60d748/simulator-single-main-60d748-kserve-67859fd6cc-cg9g2"
Oct 31 17:50:31.090084 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:31.089994    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/397a73f5-6e20-46fa-80e6-5fb061a196f2-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/397a73f5-6e20-46fa-80e6-5fb061a196f2-tmp-dir\") pod \"397a73f5-6e20-46fa-80e6-5fb061a196f2\" (UID: \"397a73f5-6e20-46fa-80e6-5fb061a196f2\") "
Oct 31 17:50:31.090084 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:31.090076    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/397a73f5-6e20-46fa-80e6-5fb061a196f2-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/397a73f5-6e20-46fa-80e6-5fb061a196f2-model-cache\") pod \"397a73f5-6e20-46fa-80e6-5fb061a196f2\" (UID: \"397a73f5-6e20-46fa-80e6-5fb061a196f2\") "
Oct 31 17:50:31.090313 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:31.090153    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/397a73f5-6e20-46fa-80e6-5fb061a196f2-home\" (UniqueName: \"kubernetes.io/empty-dir/397a73f5-6e20-46fa-80e6-5fb061a196f2-home\") pod \"397a73f5-6e20-46fa-80e6-5fb061a196f2\" (UID: \"397a73f5-6e20-46fa-80e6-5fb061a196f2\") "
Oct 31 17:50:31.090313 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:31.090264    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/397a73f5-6e20-46fa-80e6-5fb061a196f2-kube-api-access-vzbll\" (UniqueName: \"kubernetes.io/projected/397a73f5-6e20-46fa-80e6-5fb061a196f2-kube-api-access-vzbll\") pod \"397a73f5-6e20-46fa-80e6-5fb061a196f2\" (UID: \"397a73f5-6e20-46fa-80e6-5fb061a196f2\") "
Oct 31 17:50:31.090432 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:31.090311    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/397a73f5-6e20-46fa-80e6-5fb061a196f2-tls-certs\" (UniqueName: \"kubernetes.io/secret/397a73f5-6e20-46fa-80e6-5fb061a196f2-tls-certs\") pod \"397a73f5-6e20-46fa-80e6-5fb061a196f2\" (UID: \"397a73f5-6e20-46fa-80e6-5fb061a196f2\") "
Oct 31 17:50:31.090432 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:31.090305    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/397a73f5-6e20-46fa-80e6-5fb061a196f2-model-cache" pod "397a73f5-6e20-46fa-80e6-5fb061a196f2" (UID: "397a73f5-6e20-46fa-80e6-5fb061a196f2"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:50:31.090432 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:31.090311    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/397a73f5-6e20-46fa-80e6-5fb061a196f2-tmp-dir" pod "397a73f5-6e20-46fa-80e6-5fb061a196f2" (UID: "397a73f5-6e20-46fa-80e6-5fb061a196f2"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:50:31.090432 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:31.090352    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/397a73f5-6e20-46fa-80e6-5fb061a196f2-dshm\" (UniqueName: \"kubernetes.io/empty-dir/397a73f5-6e20-46fa-80e6-5fb061a196f2-dshm\") pod \"397a73f5-6e20-46fa-80e6-5fb061a196f2\" (UID: \"397a73f5-6e20-46fa-80e6-5fb061a196f2\") "
Oct 31 17:50:31.090668 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:31.090469    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/397a73f5-6e20-46fa-80e6-5fb061a196f2-home" pod "397a73f5-6e20-46fa-80e6-5fb061a196f2" (UID: "397a73f5-6e20-46fa-80e6-5fb061a196f2"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:50:31.090750 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:31.090732    4276 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/397a73f5-6e20-46fa-80e6-5fb061a196f2-tmp-dir\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:50:31.090805 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:31.090754    4276 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/397a73f5-6e20-46fa-80e6-5fb061a196f2-model-cache\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:50:31.090805 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:31.090767    4276 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/397a73f5-6e20-46fa-80e6-5fb061a196f2-home\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:50:31.092767 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:31.092735    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/397a73f5-6e20-46fa-80e6-5fb061a196f2-dshm" pod "397a73f5-6e20-46fa-80e6-5fb061a196f2" (UID: "397a73f5-6e20-46fa-80e6-5fb061a196f2"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:50:31.092971 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:31.092820    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/397a73f5-6e20-46fa-80e6-5fb061a196f2-kube-api-access-vzbll" pod "397a73f5-6e20-46fa-80e6-5fb061a196f2" (UID: "397a73f5-6e20-46fa-80e6-5fb061a196f2"). InnerVolumeSpecName "kube-api-access-vzbll". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 17:50:31.093301 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:31.093283    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/397a73f5-6e20-46fa-80e6-5fb061a196f2-tls-certs" pod "397a73f5-6e20-46fa-80e6-5fb061a196f2" (UID: "397a73f5-6e20-46fa-80e6-5fb061a196f2"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 17:50:31.191284 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:31.191254    4276 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vzbll\" (UniqueName: \"kubernetes.io/projected/397a73f5-6e20-46fa-80e6-5fb061a196f2-kube-api-access-vzbll\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:50:31.191284 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:31.191280    4276 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/397a73f5-6e20-46fa-80e6-5fb061a196f2-tls-certs\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:50:31.191284 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:31.191291    4276 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/397a73f5-6e20-46fa-80e6-5fb061a196f2-dshm\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:50:31.572580 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:31.572534    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-single-main-60d748/simulator-single-main-60d748-kserve-67859fd6cc-cg9g2" event={"ID":"397a73f5-6e20-46fa-80e6-5fb061a196f2","Type":"ContainerDied","Data":"fa251619d36644298978c9c0fdc61b4e80d402663b902dc98f736ea418ba556e"}
Oct 31 17:50:31.573038 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:31.572599    4276 scope.go:122] "RemoveContainer" containerID="fa251619d36644298978c9c0fdc61b4e80d402663b902dc98f736ea418ba556e"
Oct 31 17:50:31.573038 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:31.572597    4276 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-single-main-60d748/simulator-single-main-60d748-kserve-67859fd6cc-cg9g2"
Oct 31 17:50:31.573038 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:31.572536    4276 generic.go:356] "Generic (PLEG): container finished" podID="397a73f5-6e20-46fa-80e6-5fb061a196f2" containerID="fa251619d36644298978c9c0fdc61b4e80d402663b902dc98f736ea418ba556e" exitCode=0
Oct 31 17:50:31.573038 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:31.572791    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-single-main-60d748/simulator-single-main-60d748-kserve-67859fd6cc-cg9g2" event={"ID":"397a73f5-6e20-46fa-80e6-5fb061a196f2","Type":"ContainerDied","Data":"fea43b3b2fb336d6f02241fd06515fc24b17c3d49b9633c6d3f417f5eb207bbf"}
Oct 31 17:50:31.580828 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:31.580808    4276 scope.go:122] "RemoveContainer" containerID="fa251619d36644298978c9c0fdc61b4e80d402663b902dc98f736ea418ba556e"
Oct 31 17:50:31.581108 ip-10-0-129-127 kubenswrapper[4276]: E1031 17:50:31.581079    4276 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fa251619d36644298978c9c0fdc61b4e80d402663b902dc98f736ea418ba556e\": container with ID starting with fa251619d36644298978c9c0fdc61b4e80d402663b902dc98f736ea418ba556e not found: ID does not exist" containerID="fa251619d36644298978c9c0fdc61b4e80d402663b902dc98f736ea418ba556e"
Oct 31 17:50:31.581214 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:31.581111    4276 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fa251619d36644298978c9c0fdc61b4e80d402663b902dc98f736ea418ba556e"} err="failed to get container status \"fa251619d36644298978c9c0fdc61b4e80d402663b902dc98f736ea418ba556e\": rpc error: code = NotFound desc = could not find container \"fa251619d36644298978c9c0fdc61b4e80d402663b902dc98f736ea418ba556e\": container with ID starting with fa251619d36644298978c9c0fdc61b4e80d402663b902dc98f736ea418ba556e not found: ID does not exist"
Oct 31 17:50:31.626276 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:31.626241    4276 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-single-main-60d748/simulator-single-main-60d748-kserve-67859fd6cc-cg9g2"]
Oct 31 17:50:31.639912 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:31.639846    4276 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-single-main-60d748/simulator-single-main-60d748-kserve-67859fd6cc-cg9g2"]
Oct 31 17:50:33.367310 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:33.367273    4276 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="397a73f5-6e20-46fa-80e6-5fb061a196f2" path="/var/lib/kubelet/pods/397a73f5-6e20-46fa-80e6-5fb061a196f2/volumes"
Oct 31 17:50:33.857795 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:33.857763    4276 patch_prober.go:29] interesting pod/e2e-worker-single-main-60d748-openshift-default-7859bd475cskdsz container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 31 17:50:33.857960 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:33.857825    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-single-main-60d748-openshift-default-7859bd475cskdsz" podUID="5e5b8fc2-0092-4c1b-b689-cce7439e1c2e" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 17:50:34.160257 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:34.160172    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-single-main-60d748-58bcc565b56sk2" podUID="29854a6b-4cf9-47c8-a434-e9b3b3603f9b" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 31 17:50:34.160418 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:34.160174    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-60d748-648dfj5bj4" podUID="4b2afdc1-536d-4b96-a234-eb71ce244a5e" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 31 17:50:36.065076 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:36.065051    4276 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-single-main-60d748-openshift-default-7859bd475cskdsz"
Oct 31 17:50:36.141995 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:36.141919    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5e5b8fc2-0092-4c1b-b689-cce7439e1c2e-kube-api-access-tqf8w\" (UniqueName: \"kubernetes.io/projected/5e5b8fc2-0092-4c1b-b689-cce7439e1c2e-kube-api-access-tqf8w\") pod \"5e5b8fc2-0092-4c1b-b689-cce7439e1c2e\" (UID: \"5e5b8fc2-0092-4c1b-b689-cce7439e1c2e\") "
Oct 31 17:50:36.141995 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:36.141967    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5e5b8fc2-0092-4c1b-b689-cce7439e1c2e-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5e5b8fc2-0092-4c1b-b689-cce7439e1c2e-istio-data\") pod \"5e5b8fc2-0092-4c1b-b689-cce7439e1c2e\" (UID: \"5e5b8fc2-0092-4c1b-b689-cce7439e1c2e\") "
Oct 31 17:50:36.141995 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:36.141990    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5e5b8fc2-0092-4c1b-b689-cce7439e1c2e-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5e5b8fc2-0092-4c1b-b689-cce7439e1c2e-workload-certs\") pod \"5e5b8fc2-0092-4c1b-b689-cce7439e1c2e\" (UID: \"5e5b8fc2-0092-4c1b-b689-cce7439e1c2e\") "
Oct 31 17:50:36.142224 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:36.142013    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/5e5b8fc2-0092-4c1b-b689-cce7439e1c2e-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5e5b8fc2-0092-4c1b-b689-cce7439e1c2e-istio-podinfo\") pod \"5e5b8fc2-0092-4c1b-b689-cce7439e1c2e\" (UID: \"5e5b8fc2-0092-4c1b-b689-cce7439e1c2e\") "
Oct 31 17:50:36.142224 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:36.142046    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5e5b8fc2-0092-4c1b-b689-cce7439e1c2e-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e5b8fc2-0092-4c1b-b689-cce7439e1c2e-workload-socket\") pod \"5e5b8fc2-0092-4c1b-b689-cce7439e1c2e\" (UID: \"5e5b8fc2-0092-4c1b-b689-cce7439e1c2e\") "
Oct 31 17:50:36.142224 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:36.142100    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5e5b8fc2-0092-4c1b-b689-cce7439e1c2e-istio-token\" (UniqueName: \"kubernetes.io/projected/5e5b8fc2-0092-4c1b-b689-cce7439e1c2e-istio-token\") pod \"5e5b8fc2-0092-4c1b-b689-cce7439e1c2e\" (UID: \"5e5b8fc2-0092-4c1b-b689-cce7439e1c2e\") "
Oct 31 17:50:36.142224 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:36.142146    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5e5b8fc2-0092-4c1b-b689-cce7439e1c2e-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5e5b8fc2-0092-4c1b-b689-cce7439e1c2e-istiod-ca-cert\") pod \"5e5b8fc2-0092-4c1b-b689-cce7439e1c2e\" (UID: \"5e5b8fc2-0092-4c1b-b689-cce7439e1c2e\") "
Oct 31 17:50:36.142224 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:36.142195    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5e5b8fc2-0092-4c1b-b689-cce7439e1c2e-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e5b8fc2-0092-4c1b-b689-cce7439e1c2e-credential-socket\") pod \"5e5b8fc2-0092-4c1b-b689-cce7439e1c2e\" (UID: \"5e5b8fc2-0092-4c1b-b689-cce7439e1c2e\") "
Oct 31 17:50:36.142224 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:36.142222    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5e5b8fc2-0092-4c1b-b689-cce7439e1c2e-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5e5b8fc2-0092-4c1b-b689-cce7439e1c2e-istio-envoy\") pod \"5e5b8fc2-0092-4c1b-b689-cce7439e1c2e\" (UID: \"5e5b8fc2-0092-4c1b-b689-cce7439e1c2e\") "
Oct 31 17:50:36.142516 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:36.142330    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e5b8fc2-0092-4c1b-b689-cce7439e1c2e-istio-data" pod "5e5b8fc2-0092-4c1b-b689-cce7439e1c2e" (UID: "5e5b8fc2-0092-4c1b-b689-cce7439e1c2e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:50:36.142601 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:36.142583    4276 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5e5b8fc2-0092-4c1b-b689-cce7439e1c2e-istio-data\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:50:36.142686 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:36.142652    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e5b8fc2-0092-4c1b-b689-cce7439e1c2e-workload-certs" pod "5e5b8fc2-0092-4c1b-b689-cce7439e1c2e" (UID: "5e5b8fc2-0092-4c1b-b689-cce7439e1c2e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:50:36.142774 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:36.142692    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e5b8fc2-0092-4c1b-b689-cce7439e1c2e-credential-socket" pod "5e5b8fc2-0092-4c1b-b689-cce7439e1c2e" (UID: "5e5b8fc2-0092-4c1b-b689-cce7439e1c2e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:50:36.142774 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:36.142733    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5e5b8fc2-0092-4c1b-b689-cce7439e1c2e-istiod-ca-cert" pod "5e5b8fc2-0092-4c1b-b689-cce7439e1c2e" (UID: "5e5b8fc2-0092-4c1b-b689-cce7439e1c2e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 17:50:36.142774 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:36.142752    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e5b8fc2-0092-4c1b-b689-cce7439e1c2e-workload-socket" pod "5e5b8fc2-0092-4c1b-b689-cce7439e1c2e" (UID: "5e5b8fc2-0092-4c1b-b689-cce7439e1c2e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:50:36.144634 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:36.144610    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5e5b8fc2-0092-4c1b-b689-cce7439e1c2e-istio-podinfo" pod "5e5b8fc2-0092-4c1b-b689-cce7439e1c2e" (UID: "5e5b8fc2-0092-4c1b-b689-cce7439e1c2e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 17:50:36.145105 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:36.144978    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5e5b8fc2-0092-4c1b-b689-cce7439e1c2e-kube-api-access-tqf8w" pod "5e5b8fc2-0092-4c1b-b689-cce7439e1c2e" (UID: "5e5b8fc2-0092-4c1b-b689-cce7439e1c2e"). InnerVolumeSpecName "kube-api-access-tqf8w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 17:50:36.145105 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:36.145062    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e5b8fc2-0092-4c1b-b689-cce7439e1c2e-istio-envoy" pod "5e5b8fc2-0092-4c1b-b689-cce7439e1c2e" (UID: "5e5b8fc2-0092-4c1b-b689-cce7439e1c2e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:50:36.145105 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:36.145081    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5e5b8fc2-0092-4c1b-b689-cce7439e1c2e-istio-token" pod "5e5b8fc2-0092-4c1b-b689-cce7439e1c2e" (UID: "5e5b8fc2-0092-4c1b-b689-cce7439e1c2e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 17:50:36.243844 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:36.243810    4276 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5e5b8fc2-0092-4c1b-b689-cce7439e1c2e-istio-token\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:50:36.243844 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:36.243841    4276 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5e5b8fc2-0092-4c1b-b689-cce7439e1c2e-istiod-ca-cert\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:50:36.244120 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:36.243856    4276 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e5b8fc2-0092-4c1b-b689-cce7439e1c2e-credential-socket\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:50:36.244120 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:36.243867    4276 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5e5b8fc2-0092-4c1b-b689-cce7439e1c2e-istio-envoy\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:50:36.244120 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:36.243882    4276 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tqf8w\" (UniqueName: \"kubernetes.io/projected/5e5b8fc2-0092-4c1b-b689-cce7439e1c2e-kube-api-access-tqf8w\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:50:36.244120 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:36.243893    4276 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5e5b8fc2-0092-4c1b-b689-cce7439e1c2e-workload-certs\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:50:36.244120 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:36.243903    4276 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5e5b8fc2-0092-4c1b-b689-cce7439e1c2e-istio-podinfo\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:50:36.244120 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:36.243913    4276 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e5b8fc2-0092-4c1b-b689-cce7439e1c2e-workload-socket\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:50:36.598915 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:36.598878    4276 generic.go:356] "Generic (PLEG): container finished" podID="5e5b8fc2-0092-4c1b-b689-cce7439e1c2e" containerID="e15b20a4da030c3b984cbaaa17ebc50d9fa0fd26c27ccd784d1b04a3c8b22a2b" exitCode=0
Oct 31 17:50:36.599136 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:36.598948    4276 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-single-main-60d748-openshift-default-7859bd475cskdsz"
Oct 31 17:50:36.599136 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:36.598972    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-60d748-openshift-default-7859bd475cskdsz" event={"ID":"5e5b8fc2-0092-4c1b-b689-cce7439e1c2e","Type":"ContainerDied","Data":"e15b20a4da030c3b984cbaaa17ebc50d9fa0fd26c27ccd784d1b04a3c8b22a2b"}
Oct 31 17:50:36.599136 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:36.599038    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-60d748-openshift-default-7859bd475cskdsz" event={"ID":"5e5b8fc2-0092-4c1b-b689-cce7439e1c2e","Type":"ContainerDied","Data":"4f2a76d3a620caf972f21348b37aa32b0f36819511b713bee38ab1618866ab11"}
Oct 31 17:50:36.599136 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:36.599061    4276 scope.go:122] "RemoveContainer" containerID="e15b20a4da030c3b984cbaaa17ebc50d9fa0fd26c27ccd784d1b04a3c8b22a2b"
Oct 31 17:50:36.608062 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:36.608044    4276 scope.go:122] "RemoveContainer" containerID="6d9bb44b2d86f98bfd6eac3c363dc0dfbc6018c3c23066cb64f814434caddcfd"
Oct 31 17:50:36.616494 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:36.616476    4276 scope.go:122] "RemoveContainer" containerID="e15b20a4da030c3b984cbaaa17ebc50d9fa0fd26c27ccd784d1b04a3c8b22a2b"
Oct 31 17:50:36.616798 ip-10-0-129-127 kubenswrapper[4276]: E1031 17:50:36.616778    4276 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e15b20a4da030c3b984cbaaa17ebc50d9fa0fd26c27ccd784d1b04a3c8b22a2b\": container with ID starting with e15b20a4da030c3b984cbaaa17ebc50d9fa0fd26c27ccd784d1b04a3c8b22a2b not found: ID does not exist" containerID="e15b20a4da030c3b984cbaaa17ebc50d9fa0fd26c27ccd784d1b04a3c8b22a2b"
Oct 31 17:50:36.616873 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:36.616808    4276 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e15b20a4da030c3b984cbaaa17ebc50d9fa0fd26c27ccd784d1b04a3c8b22a2b"} err="failed to get container status \"e15b20a4da030c3b984cbaaa17ebc50d9fa0fd26c27ccd784d1b04a3c8b22a2b\": rpc error: code = NotFound desc = could not find container \"e15b20a4da030c3b984cbaaa17ebc50d9fa0fd26c27ccd784d1b04a3c8b22a2b\": container with ID starting with e15b20a4da030c3b984cbaaa17ebc50d9fa0fd26c27ccd784d1b04a3c8b22a2b not found: ID does not exist"
Oct 31 17:50:36.616873 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:36.616833    4276 scope.go:122] "RemoveContainer" containerID="6d9bb44b2d86f98bfd6eac3c363dc0dfbc6018c3c23066cb64f814434caddcfd"
Oct 31 17:50:36.617069 ip-10-0-129-127 kubenswrapper[4276]: E1031 17:50:36.617051    4276 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6d9bb44b2d86f98bfd6eac3c363dc0dfbc6018c3c23066cb64f814434caddcfd\": container with ID starting with 6d9bb44b2d86f98bfd6eac3c363dc0dfbc6018c3c23066cb64f814434caddcfd not found: ID does not exist" containerID="6d9bb44b2d86f98bfd6eac3c363dc0dfbc6018c3c23066cb64f814434caddcfd"
Oct 31 17:50:36.617121 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:36.617078    4276 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6d9bb44b2d86f98bfd6eac3c363dc0dfbc6018c3c23066cb64f814434caddcfd"} err="failed to get container status \"6d9bb44b2d86f98bfd6eac3c363dc0dfbc6018c3c23066cb64f814434caddcfd\": rpc error: code = NotFound desc = could not find container \"6d9bb44b2d86f98bfd6eac3c363dc0dfbc6018c3c23066cb64f814434caddcfd\": container with ID starting with 6d9bb44b2d86f98bfd6eac3c363dc0dfbc6018c3c23066cb64f814434caddcfd not found: ID does not exist"
Oct 31 17:50:36.668600 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:36.668572    4276 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-single-main-60d748-openshift-default-7859bd475cskdsz"]
Oct 31 17:50:36.682164 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:36.682133    4276 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-single-main-60d748-openshift-default-7859bd475cskdsz"]
Oct 31 17:50:37.366070 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:37.366035    4276 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5e5b8fc2-0092-4c1b-b689-cce7439e1c2e" path="/var/lib/kubelet/pods/5e5b8fc2-0092-4c1b-b689-cce7439e1c2e/volumes"
Oct 31 17:50:42.235335 ip-10-0-129-127 kubenswrapper[4276]: E1031 17:50:42.235301    4276 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tg5qk" containerName="extractor"
Oct 31 17:50:44.160151 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:44.160112    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-single-main-60d748-58bcc565b56sk2" podUID="29854a6b-4cf9-47c8-a434-e9b3b3603f9b" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 31 17:50:44.160665 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:44.160112    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-60d748-648dfj5bj4" podUID="4b2afdc1-536d-4b96-a234-eb71ce244a5e" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 31 17:50:46.579986 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:46.579949    4276 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-single-main-60d748-58bcc565b56sk2"
Oct 31 17:50:46.623574 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:46.623550    4276 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-60d748-648dfj5bj4"
Oct 31 17:50:46.667263 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:46.667224    4276 generic.go:356] "Generic (PLEG): container finished" podID="4b2afdc1-536d-4b96-a234-eb71ce244a5e" containerID="630765ab4c8087841d5e911a89055fe4703f46b5229bbb119a8ac0c3166a108a" exitCode=0
Oct 31 17:50:46.667458 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:46.667273    4276 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-60d748-648dfj5bj4"
Oct 31 17:50:46.667458 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:46.667283    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-60d748-648dfj5bj4" event={"ID":"4b2afdc1-536d-4b96-a234-eb71ce244a5e","Type":"ContainerDied","Data":"630765ab4c8087841d5e911a89055fe4703f46b5229bbb119a8ac0c3166a108a"}
Oct 31 17:50:46.667458 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:46.667340    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-60d748-648dfj5bj4" event={"ID":"4b2afdc1-536d-4b96-a234-eb71ce244a5e","Type":"ContainerDied","Data":"b4413bb73764171285c5120c789450d62e01696eefc332acfe360a2dff87d1ca"}
Oct 31 17:50:46.667458 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:46.667371    4276 scope.go:122] "RemoveContainer" containerID="630765ab4c8087841d5e911a89055fe4703f46b5229bbb119a8ac0c3166a108a"
Oct 31 17:50:46.669025 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:46.669000    4276 generic.go:356] "Generic (PLEG): container finished" podID="29854a6b-4cf9-47c8-a434-e9b3b3603f9b" containerID="839f7e489eceab005ebdd9279feff8eed11d70e7780ba7dfe95f29949af292e7" exitCode=0
Oct 31 17:50:46.669149 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:46.669045    4276 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-single-main-60d748-58bcc565b56sk2"
Oct 31 17:50:46.669149 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:46.669082    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-single-main-60d748-58bcc565b56sk2" event={"ID":"29854a6b-4cf9-47c8-a434-e9b3b3603f9b","Type":"ContainerDied","Data":"839f7e489eceab005ebdd9279feff8eed11d70e7780ba7dfe95f29949af292e7"}
Oct 31 17:50:46.669149 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:46.669130    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-single-main-60d748-58bcc565b56sk2" event={"ID":"29854a6b-4cf9-47c8-a434-e9b3b3603f9b","Type":"ContainerDied","Data":"9c821af7931911368087259b7ed8fb5e3a5a24678f0b29fd847582fde2901eb5"}
Oct 31 17:50:46.677775 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:46.677751    4276 scope.go:122] "RemoveContainer" containerID="630765ab4c8087841d5e911a89055fe4703f46b5229bbb119a8ac0c3166a108a"
Oct 31 17:50:46.678054 ip-10-0-129-127 kubenswrapper[4276]: E1031 17:50:46.678035    4276 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"630765ab4c8087841d5e911a89055fe4703f46b5229bbb119a8ac0c3166a108a\": container with ID starting with 630765ab4c8087841d5e911a89055fe4703f46b5229bbb119a8ac0c3166a108a not found: ID does not exist" containerID="630765ab4c8087841d5e911a89055fe4703f46b5229bbb119a8ac0c3166a108a"
Oct 31 17:50:46.678121 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:46.678061    4276 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"630765ab4c8087841d5e911a89055fe4703f46b5229bbb119a8ac0c3166a108a"} err="failed to get container status \"630765ab4c8087841d5e911a89055fe4703f46b5229bbb119a8ac0c3166a108a\": rpc error: code = NotFound desc = could not find container \"630765ab4c8087841d5e911a89055fe4703f46b5229bbb119a8ac0c3166a108a\": container with ID starting with 630765ab4c8087841d5e911a89055fe4703f46b5229bbb119a8ac0c3166a108a not found: ID does not exist"
Oct 31 17:50:46.678121 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:46.678077    4276 scope.go:122] "RemoveContainer" containerID="839f7e489eceab005ebdd9279feff8eed11d70e7780ba7dfe95f29949af292e7"
Oct 31 17:50:46.687260 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:46.687244    4276 scope.go:122] "RemoveContainer" containerID="839f7e489eceab005ebdd9279feff8eed11d70e7780ba7dfe95f29949af292e7"
Oct 31 17:50:46.687565 ip-10-0-129-127 kubenswrapper[4276]: E1031 17:50:46.687547    4276 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"839f7e489eceab005ebdd9279feff8eed11d70e7780ba7dfe95f29949af292e7\": container with ID starting with 839f7e489eceab005ebdd9279feff8eed11d70e7780ba7dfe95f29949af292e7 not found: ID does not exist" containerID="839f7e489eceab005ebdd9279feff8eed11d70e7780ba7dfe95f29949af292e7"
Oct 31 17:50:46.687637 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:46.687576    4276 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"839f7e489eceab005ebdd9279feff8eed11d70e7780ba7dfe95f29949af292e7"} err="failed to get container status \"839f7e489eceab005ebdd9279feff8eed11d70e7780ba7dfe95f29949af292e7\": rpc error: code = NotFound desc = could not find container \"839f7e489eceab005ebdd9279feff8eed11d70e7780ba7dfe95f29949af292e7\": container with ID starting with 839f7e489eceab005ebdd9279feff8eed11d70e7780ba7dfe95f29949af292e7 not found: ID does not exist"
Oct 31 17:50:46.768828 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:46.768794    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4b2afdc1-536d-4b96-a234-eb71ce244a5e-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4b2afdc1-536d-4b96-a234-eb71ce244a5e-plugins-config-volume\") pod \"4b2afdc1-536d-4b96-a234-eb71ce244a5e\" (UID: \"4b2afdc1-536d-4b96-a234-eb71ce244a5e\") "
Oct 31 17:50:46.769026 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:46.768849    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/29854a6b-4cf9-47c8-a434-e9b3b3603f9b-kube-api-access-blwwt\" (UniqueName: \"kubernetes.io/projected/29854a6b-4cf9-47c8-a434-e9b3b3603f9b-kube-api-access-blwwt\") pod \"29854a6b-4cf9-47c8-a434-e9b3b3603f9b\" (UID: \"29854a6b-4cf9-47c8-a434-e9b3b3603f9b\") "
Oct 31 17:50:46.769026 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:46.768983    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4b2afdc1-536d-4b96-a234-eb71ce244a5e-kube-api-access-h4b6c\" (UniqueName: \"kubernetes.io/projected/4b2afdc1-536d-4b96-a234-eb71ce244a5e-kube-api-access-h4b6c\") pod \"4b2afdc1-536d-4b96-a234-eb71ce244a5e\" (UID: \"4b2afdc1-536d-4b96-a234-eb71ce244a5e\") "
Oct 31 17:50:46.769155 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:46.769028    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/29854a6b-4cf9-47c8-a434-e9b3b3603f9b-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/29854a6b-4cf9-47c8-a434-e9b3b3603f9b-plugins-config-volume\") pod \"29854a6b-4cf9-47c8-a434-e9b3b3603f9b\" (UID: \"29854a6b-4cf9-47c8-a434-e9b3b3603f9b\") "
Oct 31 17:50:46.769253 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:46.769231    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4b2afdc1-536d-4b96-a234-eb71ce244a5e-plugins-config-volume" pod "4b2afdc1-536d-4b96-a234-eb71ce244a5e" (UID: "4b2afdc1-536d-4b96-a234-eb71ce244a5e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 17:50:46.769420 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:46.769403    4276 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4b2afdc1-536d-4b96-a234-eb71ce244a5e-plugins-config-volume\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:50:46.769515 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:46.769465    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/29854a6b-4cf9-47c8-a434-e9b3b3603f9b-plugins-config-volume" pod "29854a6b-4cf9-47c8-a434-e9b3b3603f9b" (UID: "29854a6b-4cf9-47c8-a434-e9b3b3603f9b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 17:50:46.771067 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:46.771046    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/29854a6b-4cf9-47c8-a434-e9b3b3603f9b-kube-api-access-blwwt" pod "29854a6b-4cf9-47c8-a434-e9b3b3603f9b" (UID: "29854a6b-4cf9-47c8-a434-e9b3b3603f9b"). InnerVolumeSpecName "kube-api-access-blwwt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 17:50:46.771164 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:46.771074    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4b2afdc1-536d-4b96-a234-eb71ce244a5e-kube-api-access-h4b6c" pod "4b2afdc1-536d-4b96-a234-eb71ce244a5e" (UID: "4b2afdc1-536d-4b96-a234-eb71ce244a5e"). InnerVolumeSpecName "kube-api-access-h4b6c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 17:50:46.869940 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:46.869866    4276 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h4b6c\" (UniqueName: \"kubernetes.io/projected/4b2afdc1-536d-4b96-a234-eb71ce244a5e-kube-api-access-h4b6c\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:50:46.869940 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:46.869892    4276 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/29854a6b-4cf9-47c8-a434-e9b3b3603f9b-plugins-config-volume\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:50:46.869940 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:46.869901    4276 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-blwwt\" (UniqueName: \"kubernetes.io/projected/29854a6b-4cf9-47c8-a434-e9b3b3603f9b-kube-api-access-blwwt\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:50:47.014096 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:47.014061    4276 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-60d748-648dfj5bj4"]
Oct 31 17:50:47.027933 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:47.027897    4276 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-60d748-648dfj5bj4"]
Oct 31 17:50:47.054032 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:47.054004    4276 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-60d748-58bcc565b56sk2"]
Oct 31 17:50:47.056823 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:47.056798    4276 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-60d748-58bcc565b56sk2"]
Oct 31 17:50:47.366090 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:47.366058    4276 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="29854a6b-4cf9-47c8-a434-e9b3b3603f9b" path="/var/lib/kubelet/pods/29854a6b-4cf9-47c8-a434-e9b3b3603f9b/volumes"
Oct 31 17:50:47.366659 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:47.366637    4276 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4b2afdc1-536d-4b96-a234-eb71ce244a5e" path="/var/lib/kubelet/pods/4b2afdc1-536d-4b96-a234-eb71ce244a5e/volumes"
Oct 31 17:50:52.372259 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:52.372223    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:50:52.935521 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:52.935489    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:50:58.529228 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:50:58.529196    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:51:04.573254 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:51:04.573218    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:51:09.500876 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:51:09.500844    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:51:12.233521 ip-10-0-129-127 kubenswrapper[4276]: E1031 17:51:12.233487    4276 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tg5qk" containerName="extractor"
Oct 31 17:51:14.473766 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:51:14.473732    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:51:42.233804 ip-10-0-129-127 kubenswrapper[4276]: E1031 17:51:42.233768    4276 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tg5qk" containerName="extractor"
Oct 31 17:52:04.705767 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:52:04.705678    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:52:09.617061 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:52:09.617024    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:52:12.233525 ip-10-0-129-127 kubenswrapper[4276]: E1031 17:52:12.233491    4276 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tg5qk" containerName="extractor"
Oct 31 17:52:14.545663 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:52:14.545609    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:52:17.501483 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:52:17.501454    4276 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zpf4d_5819fd64-9ac0-4436-b8fe-dcd3a27827ac/ovn-acl-logging/0.log"
Oct 31 17:52:17.506004 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:52:17.505985    4276 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zpf4d_5819fd64-9ac0-4436-b8fe-dcd3a27827ac/ovn-acl-logging/0.log"
Oct 31 17:52:35.155350 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:52:35.155305    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:52:37.086670 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:52:37.086619    4276 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7d9897b54c-b8gmm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.34:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 17:52:37.087423 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:52:37.087375    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-b8gmm" podUID="5530ee8f-be36-43d3-a268-051794343d8b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.34:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 17:52:42.233132 ip-10-0-129-127 kubenswrapper[4276]: E1031 17:52:42.233096    4276 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tg5qk" containerName="extractor"
Oct 31 17:52:53.758392 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:52:53.758362    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:52:54.101436 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:52:54.101400    4276 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-single-main-78301f-openshift-default-867d4565f77l69h"]
Oct 31 17:52:54.101922 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:52:54.101906    4276 cpu_manager.go:414] "containerMap: removing container" podUID="4b2afdc1-536d-4b96-a234-eb71ce244a5e" containerName="payload-pre-processing"
Oct 31 17:52:54.101975 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:52:54.101924    4276 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4b2afdc1-536d-4b96-a234-eb71ce244a5e" containerName="payload-pre-processing"
Oct 31 17:52:54.101975 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:52:54.101941    4276 cpu_manager.go:414] "containerMap: removing container" podUID="025470d4-6fca-4ab3-b8f2-0067cb018b5f" containerName="maas-api"
Oct 31 17:52:54.101975 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:52:54.101949    4276 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="025470d4-6fca-4ab3-b8f2-0067cb018b5f" containerName="maas-api"
Oct 31 17:52:54.101975 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:52:54.101966    4276 cpu_manager.go:414] "containerMap: removing container" podUID="397a73f5-6e20-46fa-80e6-5fb061a196f2" containerName="main"
Oct 31 17:52:54.101975 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:52:54.101973    4276 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="397a73f5-6e20-46fa-80e6-5fb061a196f2" containerName="main"
Oct 31 17:52:54.102134 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:52:54.101989    4276 cpu_manager.go:414] "containerMap: removing container" podUID="29854a6b-4cf9-47c8-a434-e9b3b3603f9b" containerName="payload-processing"
Oct 31 17:52:54.102134 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:52:54.101996    4276 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="29854a6b-4cf9-47c8-a434-e9b3b3603f9b" containerName="payload-processing"
Oct 31 17:52:54.102134 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:52:54.102005    4276 cpu_manager.go:414] "containerMap: removing container" podUID="5e5b8fc2-0092-4c1b-b689-cce7439e1c2e" containerName="istio-proxy"
Oct 31 17:52:54.102134 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:52:54.102012    4276 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5e5b8fc2-0092-4c1b-b689-cce7439e1c2e" containerName="istio-proxy"
Oct 31 17:52:54.102134 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:52:54.102050    4276 cpu_manager.go:414] "containerMap: removing container" podUID="5e5b8fc2-0092-4c1b-b689-cce7439e1c2e" containerName="istio-proxy"
Oct 31 17:52:54.102134 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:52:54.102057    4276 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5e5b8fc2-0092-4c1b-b689-cce7439e1c2e" containerName="istio-proxy"
Oct 31 17:52:54.102134 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:52:54.102126    4276 memory_manager.go:365] "RemoveStaleState removing state" podUID="29854a6b-4cf9-47c8-a434-e9b3b3603f9b" containerName="payload-processing"
Oct 31 17:52:54.102343 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:52:54.102138    4276 memory_manager.go:365] "RemoveStaleState removing state" podUID="5e5b8fc2-0092-4c1b-b689-cce7439e1c2e" containerName="istio-proxy"
Oct 31 17:52:54.102343 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:52:54.102153    4276 memory_manager.go:365] "RemoveStaleState removing state" podUID="4b2afdc1-536d-4b96-a234-eb71ce244a5e" containerName="payload-pre-processing"
Oct 31 17:52:54.102343 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:52:54.102161    4276 memory_manager.go:365] "RemoveStaleState removing state" podUID="025470d4-6fca-4ab3-b8f2-0067cb018b5f" containerName="maas-api"
Oct 31 17:52:54.102343 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:52:54.102170    4276 memory_manager.go:365] "RemoveStaleState removing state" podUID="397a73f5-6e20-46fa-80e6-5fb061a196f2" containerName="main"
Oct 31 17:52:54.105571 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:52:54.105549    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-single-main-78301f-openshift-default-867d4565f77l69h"
Oct 31 17:52:54.125416 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:52:54.125390    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-single-main-78301f-openshift-default-dockercfg-jk5xm\""
Oct 31 17:52:54.140788 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:52:54.140750    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-single-main-78301f-openshift-default-867d4565f77l69h"]
Oct 31 17:52:54.155127 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:52:54.155099    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c6b1253-75fe-4ebd-8849-79cbe110cf50-workload-socket\") pod \"e2e-worker-single-main-78301f-openshift-default-867d4565f77l69h\" (UID: \"8c6b1253-75fe-4ebd-8849-79cbe110cf50\") " pod="openshift-ingress/e2e-worker-single-main-78301f-openshift-default-867d4565f77l69h"
Oct 31 17:52:54.155254 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:52:54.155134    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8c6b1253-75fe-4ebd-8849-79cbe110cf50-istiod-ca-cert\") pod \"e2e-worker-single-main-78301f-openshift-default-867d4565f77l69h\" (UID: \"8c6b1253-75fe-4ebd-8849-79cbe110cf50\") " pod="openshift-ingress/e2e-worker-single-main-78301f-openshift-default-867d4565f77l69h"
Oct 31 17:52:54.155254 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:52:54.155152    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gwmzm\" (UniqueName: \"kubernetes.io/projected/8c6b1253-75fe-4ebd-8849-79cbe110cf50-kube-api-access-gwmzm\") pod \"e2e-worker-single-main-78301f-openshift-default-867d4565f77l69h\" (UID: \"8c6b1253-75fe-4ebd-8849-79cbe110cf50\") " pod="openshift-ingress/e2e-worker-single-main-78301f-openshift-default-867d4565f77l69h"
Oct 31 17:52:54.155254 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:52:54.155173    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8c6b1253-75fe-4ebd-8849-79cbe110cf50-istio-podinfo\") pod \"e2e-worker-single-main-78301f-openshift-default-867d4565f77l69h\" (UID: \"8c6b1253-75fe-4ebd-8849-79cbe110cf50\") " pod="openshift-ingress/e2e-worker-single-main-78301f-openshift-default-867d4565f77l69h"
Oct 31 17:52:54.155365 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:52:54.155334    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8c6b1253-75fe-4ebd-8849-79cbe110cf50-istio-envoy\") pod \"e2e-worker-single-main-78301f-openshift-default-867d4565f77l69h\" (UID: \"8c6b1253-75fe-4ebd-8849-79cbe110cf50\") " pod="openshift-ingress/e2e-worker-single-main-78301f-openshift-default-867d4565f77l69h"
Oct 31 17:52:54.155412 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:52:54.155371    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8c6b1253-75fe-4ebd-8849-79cbe110cf50-workload-certs\") pod \"e2e-worker-single-main-78301f-openshift-default-867d4565f77l69h\" (UID: \"8c6b1253-75fe-4ebd-8849-79cbe110cf50\") " pod="openshift-ingress/e2e-worker-single-main-78301f-openshift-default-867d4565f77l69h"
Oct 31 17:52:54.155540 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:52:54.155521    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8c6b1253-75fe-4ebd-8849-79cbe110cf50-istio-token\") pod \"e2e-worker-single-main-78301f-openshift-default-867d4565f77l69h\" (UID: \"8c6b1253-75fe-4ebd-8849-79cbe110cf50\") " pod="openshift-ingress/e2e-worker-single-main-78301f-openshift-default-867d4565f77l69h"
Oct 31 17:52:54.155613 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:52:54.155570    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c6b1253-75fe-4ebd-8849-79cbe110cf50-credential-socket\") pod \"e2e-worker-single-main-78301f-openshift-default-867d4565f77l69h\" (UID: \"8c6b1253-75fe-4ebd-8849-79cbe110cf50\") " pod="openshift-ingress/e2e-worker-single-main-78301f-openshift-default-867d4565f77l69h"
Oct 31 17:52:54.155667 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:52:54.155645    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8c6b1253-75fe-4ebd-8849-79cbe110cf50-istio-data\") pod \"e2e-worker-single-main-78301f-openshift-default-867d4565f77l69h\" (UID: \"8c6b1253-75fe-4ebd-8849-79cbe110cf50\") " pod="openshift-ingress/e2e-worker-single-main-78301f-openshift-default-867d4565f77l69h"
Oct 31 17:52:54.170297 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:52:54.170274    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:52:54.256568 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:52:54.256530    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8c6b1253-75fe-4ebd-8849-79cbe110cf50-istio-token\") pod \"e2e-worker-single-main-78301f-openshift-default-867d4565f77l69h\" (UID: \"8c6b1253-75fe-4ebd-8849-79cbe110cf50\") " pod="openshift-ingress/e2e-worker-single-main-78301f-openshift-default-867d4565f77l69h"
Oct 31 17:52:54.256767 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:52:54.256583    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c6b1253-75fe-4ebd-8849-79cbe110cf50-credential-socket\") pod \"e2e-worker-single-main-78301f-openshift-default-867d4565f77l69h\" (UID: \"8c6b1253-75fe-4ebd-8849-79cbe110cf50\") " pod="openshift-ingress/e2e-worker-single-main-78301f-openshift-default-867d4565f77l69h"
Oct 31 17:52:54.256767 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:52:54.256619    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8c6b1253-75fe-4ebd-8849-79cbe110cf50-istio-data\") pod \"e2e-worker-single-main-78301f-openshift-default-867d4565f77l69h\" (UID: \"8c6b1253-75fe-4ebd-8849-79cbe110cf50\") " pod="openshift-ingress/e2e-worker-single-main-78301f-openshift-default-867d4565f77l69h"
Oct 31 17:52:54.256767 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:52:54.256683    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c6b1253-75fe-4ebd-8849-79cbe110cf50-workload-socket\") pod \"e2e-worker-single-main-78301f-openshift-default-867d4565f77l69h\" (UID: \"8c6b1253-75fe-4ebd-8849-79cbe110cf50\") " pod="openshift-ingress/e2e-worker-single-main-78301f-openshift-default-867d4565f77l69h"
Oct 31 17:52:54.256767 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:52:54.256718    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8c6b1253-75fe-4ebd-8849-79cbe110cf50-istiod-ca-cert\") pod \"e2e-worker-single-main-78301f-openshift-default-867d4565f77l69h\" (UID: \"8c6b1253-75fe-4ebd-8849-79cbe110cf50\") " pod="openshift-ingress/e2e-worker-single-main-78301f-openshift-default-867d4565f77l69h"
Oct 31 17:52:54.256767 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:52:54.256745    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gwmzm\" (UniqueName: \"kubernetes.io/projected/8c6b1253-75fe-4ebd-8849-79cbe110cf50-kube-api-access-gwmzm\") pod \"e2e-worker-single-main-78301f-openshift-default-867d4565f77l69h\" (UID: \"8c6b1253-75fe-4ebd-8849-79cbe110cf50\") " pod="openshift-ingress/e2e-worker-single-main-78301f-openshift-default-867d4565f77l69h"
Oct 31 17:52:54.257064 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:52:54.256776    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8c6b1253-75fe-4ebd-8849-79cbe110cf50-istio-podinfo\") pod \"e2e-worker-single-main-78301f-openshift-default-867d4565f77l69h\" (UID: \"8c6b1253-75fe-4ebd-8849-79cbe110cf50\") " pod="openshift-ingress/e2e-worker-single-main-78301f-openshift-default-867d4565f77l69h"
Oct 31 17:52:54.257064 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:52:54.256825    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8c6b1253-75fe-4ebd-8849-79cbe110cf50-istio-envoy\") pod \"e2e-worker-single-main-78301f-openshift-default-867d4565f77l69h\" (UID: \"8c6b1253-75fe-4ebd-8849-79cbe110cf50\") " pod="openshift-ingress/e2e-worker-single-main-78301f-openshift-default-867d4565f77l69h"
Oct 31 17:52:54.257064 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:52:54.256849    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8c6b1253-75fe-4ebd-8849-79cbe110cf50-workload-certs\") pod \"e2e-worker-single-main-78301f-openshift-default-867d4565f77l69h\" (UID: \"8c6b1253-75fe-4ebd-8849-79cbe110cf50\") " pod="openshift-ingress/e2e-worker-single-main-78301f-openshift-default-867d4565f77l69h"
Oct 31 17:52:54.257064 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:52:54.257046    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c6b1253-75fe-4ebd-8849-79cbe110cf50-credential-socket\") pod \"e2e-worker-single-main-78301f-openshift-default-867d4565f77l69h\" (UID: \"8c6b1253-75fe-4ebd-8849-79cbe110cf50\") " pod="openshift-ingress/e2e-worker-single-main-78301f-openshift-default-867d4565f77l69h"
Oct 31 17:52:54.257064 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:52:54.257059    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8c6b1253-75fe-4ebd-8849-79cbe110cf50-istio-data\") pod \"e2e-worker-single-main-78301f-openshift-default-867d4565f77l69h\" (UID: \"8c6b1253-75fe-4ebd-8849-79cbe110cf50\") " pod="openshift-ingress/e2e-worker-single-main-78301f-openshift-default-867d4565f77l69h"
Oct 31 17:52:54.257333 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:52:54.257118    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c6b1253-75fe-4ebd-8849-79cbe110cf50-workload-socket\") pod \"e2e-worker-single-main-78301f-openshift-default-867d4565f77l69h\" (UID: \"8c6b1253-75fe-4ebd-8849-79cbe110cf50\") " pod="openshift-ingress/e2e-worker-single-main-78301f-openshift-default-867d4565f77l69h"
Oct 31 17:52:54.257392 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:52:54.257333    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8c6b1253-75fe-4ebd-8849-79cbe110cf50-workload-certs\") pod \"e2e-worker-single-main-78301f-openshift-default-867d4565f77l69h\" (UID: \"8c6b1253-75fe-4ebd-8849-79cbe110cf50\") " pod="openshift-ingress/e2e-worker-single-main-78301f-openshift-default-867d4565f77l69h"
Oct 31 17:52:54.257520 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:52:54.257500    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8c6b1253-75fe-4ebd-8849-79cbe110cf50-istiod-ca-cert\") pod \"e2e-worker-single-main-78301f-openshift-default-867d4565f77l69h\" (UID: \"8c6b1253-75fe-4ebd-8849-79cbe110cf50\") " pod="openshift-ingress/e2e-worker-single-main-78301f-openshift-default-867d4565f77l69h"
Oct 31 17:52:54.259321 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:52:54.259298    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8c6b1253-75fe-4ebd-8849-79cbe110cf50-istio-envoy\") pod \"e2e-worker-single-main-78301f-openshift-default-867d4565f77l69h\" (UID: \"8c6b1253-75fe-4ebd-8849-79cbe110cf50\") " pod="openshift-ingress/e2e-worker-single-main-78301f-openshift-default-867d4565f77l69h"
Oct 31 17:52:54.259405 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:52:54.259395    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8c6b1253-75fe-4ebd-8849-79cbe110cf50-istio-podinfo\") pod \"e2e-worker-single-main-78301f-openshift-default-867d4565f77l69h\" (UID: \"8c6b1253-75fe-4ebd-8849-79cbe110cf50\") " pod="openshift-ingress/e2e-worker-single-main-78301f-openshift-default-867d4565f77l69h"
Oct 31 17:52:54.277393 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:52:54.277372    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gwmzm\" (UniqueName: \"kubernetes.io/projected/8c6b1253-75fe-4ebd-8849-79cbe110cf50-kube-api-access-gwmzm\") pod \"e2e-worker-single-main-78301f-openshift-default-867d4565f77l69h\" (UID: \"8c6b1253-75fe-4ebd-8849-79cbe110cf50\") " pod="openshift-ingress/e2e-worker-single-main-78301f-openshift-default-867d4565f77l69h"
Oct 31 17:52:54.277393 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:52:54.277381    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8c6b1253-75fe-4ebd-8849-79cbe110cf50-istio-token\") pod \"e2e-worker-single-main-78301f-openshift-default-867d4565f77l69h\" (UID: \"8c6b1253-75fe-4ebd-8849-79cbe110cf50\") " pod="openshift-ingress/e2e-worker-single-main-78301f-openshift-default-867d4565f77l69h"
Oct 31 17:52:54.420353 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:52:54.420286    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-single-main-78301f-openshift-default-867d4565f77l69h"
Oct 31 17:52:54.511328 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:52:54.511299    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:52:54.732682 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:52:54.732659    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-single-main-78301f-openshift-default-867d4565f77l69h"]
Oct 31 17:52:54.733695 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:52:54.733670    4276 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c6b1253_75fe_4ebd_8849_79cbe110cf50.slice/crio-03fbfd2dad1268b2dbf101d35b26ec61fa6418ea42aa348976f7f934eae31593 WatchSource:0}: Error finding container 03fbfd2dad1268b2dbf101d35b26ec61fa6418ea42aa348976f7f934eae31593: Status 404 returned error can't find the container with id 03fbfd2dad1268b2dbf101d35b26ec61fa6418ea42aa348976f7f934eae31593
Oct 31 17:52:54.735866 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:52:54.735835    4276 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 17:52:54.735978 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:52:54.735908    4276 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 17:52:54.735978 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:52:54.735950    4276 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 17:52:55.292494 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:52:55.292463    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-78301f-openshift-default-867d4565f77l69h" event={"ID":"8c6b1253-75fe-4ebd-8849-79cbe110cf50","Type":"ContainerStarted","Data":"8206aefa0b1f92a92d921f37e05e813d4485b5ea71f53914063a9bec1f810e65"}
Oct 31 17:52:55.292898 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:52:55.292515    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-78301f-openshift-default-867d4565f77l69h" event={"ID":"8c6b1253-75fe-4ebd-8849-79cbe110cf50","Type":"ContainerStarted","Data":"03fbfd2dad1268b2dbf101d35b26ec61fa6418ea42aa348976f7f934eae31593"}
Oct 31 17:52:55.337540 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:52:55.337496    4276 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-single-main-78301f-openshift-default-867d4565f77l69h" podStartSLOduration=1.337483109 podStartE2EDuration="1.337483109s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 17:52:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 17:52:55.335470198 +0000 UTC m=+1538.133215123" watchObservedRunningTime="2026-10-31 17:52:55.337483109 +0000 UTC m=+1538.135228037"
Oct 31 17:52:55.420961 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:52:55.420925    4276 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-single-main-78301f-openshift-default-867d4565f77l69h"
Oct 31 17:52:55.426102 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:52:55.426077    4276 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-single-main-78301f-openshift-default-867d4565f77l69h"
Oct 31 17:52:56.298057 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:52:56.298020    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-single-main-78301f-openshift-default-867d4565f77l69h"
Oct 31 17:52:56.303793 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:52:56.303769    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-single-main-78301f-openshift-default-867d4565f77l69h"
Oct 31 17:52:57.721871 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:52:57.721837    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:52:57.988783 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:52:57.988704    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:52:59.166714 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:52:59.166682    4276 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-78301f-567555f6cc-lscgk"]
Oct 31 17:52:59.167211 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:52:59.167194    4276 memory_manager.go:365] "RemoveStaleState removing state" podUID="5e5b8fc2-0092-4c1b-b689-cce7439e1c2e" containerName="istio-proxy"
Oct 31 17:52:59.170457 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:52:59.170420    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-78301f-567555f6cc-lscgk"
Oct 31 17:52:59.186224 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:52:59.186203    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-single-main-78301f-serving-cert\""
Oct 31 17:52:59.202581 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:52:59.202555    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-78301f-567555f6cc-lscgk"]
Oct 31 17:52:59.204569 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:52:59.204545    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xtf67\" (UniqueName: \"kubernetes.io/projected/1f51254c-4a54-4c60-b654-7658abec8fa8-kube-api-access-xtf67\") pod \"maas-api-e2e-worker-single-main-78301f-567555f6cc-lscgk\" (UID: \"1f51254c-4a54-4c60-b654-7658abec8fa8\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-78301f-567555f6cc-lscgk"
Oct 31 17:52:59.204702 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:52:59.204592    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1f51254c-4a54-4c60-b654-7658abec8fa8-openshift-service-ca\") pod \"maas-api-e2e-worker-single-main-78301f-567555f6cc-lscgk\" (UID: \"1f51254c-4a54-4c60-b654-7658abec8fa8\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-78301f-567555f6cc-lscgk"
Oct 31 17:52:59.204769 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:52:59.204742    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1f51254c-4a54-4c60-b654-7658abec8fa8-metrics-certs\") pod \"maas-api-e2e-worker-single-main-78301f-567555f6cc-lscgk\" (UID: \"1f51254c-4a54-4c60-b654-7658abec8fa8\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-78301f-567555f6cc-lscgk"
Oct 31 17:52:59.204825 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:52:59.204807    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1f51254c-4a54-4c60-b654-7658abec8fa8-maas-api-tls\") pod \"maas-api-e2e-worker-single-main-78301f-567555f6cc-lscgk\" (UID: \"1f51254c-4a54-4c60-b654-7658abec8fa8\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-78301f-567555f6cc-lscgk"
Oct 31 17:52:59.279302 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:52:59.279270    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:52:59.305457 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:52:59.305420    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1f51254c-4a54-4c60-b654-7658abec8fa8-metrics-certs\") pod \"maas-api-e2e-worker-single-main-78301f-567555f6cc-lscgk\" (UID: \"1f51254c-4a54-4c60-b654-7658abec8fa8\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-78301f-567555f6cc-lscgk"
Oct 31 17:52:59.305578 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:52:59.305491    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1f51254c-4a54-4c60-b654-7658abec8fa8-maas-api-tls\") pod \"maas-api-e2e-worker-single-main-78301f-567555f6cc-lscgk\" (UID: \"1f51254c-4a54-4c60-b654-7658abec8fa8\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-78301f-567555f6cc-lscgk"
Oct 31 17:52:59.305578 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:52:59.305536    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xtf67\" (UniqueName: \"kubernetes.io/projected/1f51254c-4a54-4c60-b654-7658abec8fa8-kube-api-access-xtf67\") pod \"maas-api-e2e-worker-single-main-78301f-567555f6cc-lscgk\" (UID: \"1f51254c-4a54-4c60-b654-7658abec8fa8\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-78301f-567555f6cc-lscgk"
Oct 31 17:52:59.305578 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:52:59.305573    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1f51254c-4a54-4c60-b654-7658abec8fa8-openshift-service-ca\") pod \"maas-api-e2e-worker-single-main-78301f-567555f6cc-lscgk\" (UID: \"1f51254c-4a54-4c60-b654-7658abec8fa8\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-78301f-567555f6cc-lscgk"
Oct 31 17:52:59.305738 ip-10-0-129-127 kubenswrapper[4276]: E1031 17:52:59.305610    4276 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-single-main-78301f-serving-cert: secret "maas-api-e2e-worker-single-main-78301f-serving-cert" not found
Oct 31 17:52:59.305738 ip-10-0-129-127 kubenswrapper[4276]: E1031 17:52:59.305675    4276 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1f51254c-4a54-4c60-b654-7658abec8fa8-maas-api-tls podName:1f51254c-4a54-4c60-b654-7658abec8fa8 nodeName:}" failed. No retries permitted until 2026-10-31 17:52:59.805654681 +0000 UTC m=+1542.603399600 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/1f51254c-4a54-4c60-b654-7658abec8fa8-maas-api-tls") pod "maas-api-e2e-worker-single-main-78301f-567555f6cc-lscgk" (UID: "1f51254c-4a54-4c60-b654-7658abec8fa8") : secret "maas-api-e2e-worker-single-main-78301f-serving-cert" not found
Oct 31 17:52:59.306251 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:52:59.306233    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1f51254c-4a54-4c60-b654-7658abec8fa8-openshift-service-ca\") pod \"maas-api-e2e-worker-single-main-78301f-567555f6cc-lscgk\" (UID: \"1f51254c-4a54-4c60-b654-7658abec8fa8\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-78301f-567555f6cc-lscgk"
Oct 31 17:52:59.307749 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:52:59.307729    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1f51254c-4a54-4c60-b654-7658abec8fa8-metrics-certs\") pod \"maas-api-e2e-worker-single-main-78301f-567555f6cc-lscgk\" (UID: \"1f51254c-4a54-4c60-b654-7658abec8fa8\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-78301f-567555f6cc-lscgk"
Oct 31 17:52:59.328335 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:52:59.328316    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xtf67\" (UniqueName: \"kubernetes.io/projected/1f51254c-4a54-4c60-b654-7658abec8fa8-kube-api-access-xtf67\") pod \"maas-api-e2e-worker-single-main-78301f-567555f6cc-lscgk\" (UID: \"1f51254c-4a54-4c60-b654-7658abec8fa8\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-78301f-567555f6cc-lscgk"
Oct 31 17:52:59.391422 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:52:59.391386    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:52:59.501366 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:52:59.501282    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:52:59.612958 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:52:59.612926    4276 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-78301f-848442tw52"]
Oct 31 17:52:59.616577 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:52:59.616559    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-78301f-848442tw52"
Oct 31 17:52:59.628727 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:52:59.628695    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:52:59.633316 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:52:59.633296    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-78301f-848442tw52"]
Oct 31 17:52:59.638271 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:52:59.638248    4276 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-single-main-78301f\""
Oct 31 17:52:59.639282 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:52:59.639253    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-single-m-fd50ff72-dockercfg-gwd5x\""
Oct 31 17:52:59.701610 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:52:59.701573    4276 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-78301f-5db98f554cjjtc"]
Oct 31 17:52:59.705110 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:52:59.705090    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-single-main-78301f-5db98f554cjjtc"
Oct 31 17:52:59.710099 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:52:59.710065    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ce1ffbaa-b297-4c0b-b598-552d28abd520-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-single-main-78301f-848442tw52\" (UID: \"ce1ffbaa-b297-4c0b-b598-552d28abd520\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-78301f-848442tw52"
Oct 31 17:52:59.710233 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:52:59.710156    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vpncg\" (UniqueName: \"kubernetes.io/projected/ce1ffbaa-b297-4c0b-b598-552d28abd520-kube-api-access-vpncg\") pod \"payload-pre-processing-e2e-worker-single-main-78301f-848442tw52\" (UID: \"ce1ffbaa-b297-4c0b-b598-552d28abd520\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-78301f-848442tw52"
Oct 31 17:52:59.721314 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:52:59.721291    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-78301f-5db98f554cjjtc"]
Oct 31 17:52:59.810859 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:52:59.810830    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1f51254c-4a54-4c60-b654-7658abec8fa8-maas-api-tls\") pod \"maas-api-e2e-worker-single-main-78301f-567555f6cc-lscgk\" (UID: \"1f51254c-4a54-4c60-b654-7658abec8fa8\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-78301f-567555f6cc-lscgk"
Oct 31 17:52:59.811025 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:52:59.810885    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f4g6t\" (UniqueName: \"kubernetes.io/projected/7a22298b-5d98-45db-9831-f02ba58d0eac-kube-api-access-f4g6t\") pod \"payload-processing-e2e-worker-single-main-78301f-5db98f554cjjtc\" (UID: \"7a22298b-5d98-45db-9831-f02ba58d0eac\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-78301f-5db98f554cjjtc"
Oct 31 17:52:59.811025 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:52:59.810909    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ce1ffbaa-b297-4c0b-b598-552d28abd520-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-single-main-78301f-848442tw52\" (UID: \"ce1ffbaa-b297-4c0b-b598-552d28abd520\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-78301f-848442tw52"
Oct 31 17:52:59.811120 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:52:59.811019    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7a22298b-5d98-45db-9831-f02ba58d0eac-plugins-config-volume\") pod \"payload-processing-e2e-worker-single-main-78301f-5db98f554cjjtc\" (UID: \"7a22298b-5d98-45db-9831-f02ba58d0eac\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-78301f-5db98f554cjjtc"
Oct 31 17:52:59.811120 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:52:59.811060    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vpncg\" (UniqueName: \"kubernetes.io/projected/ce1ffbaa-b297-4c0b-b598-552d28abd520-kube-api-access-vpncg\") pod \"payload-pre-processing-e2e-worker-single-main-78301f-848442tw52\" (UID: \"ce1ffbaa-b297-4c0b-b598-552d28abd520\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-78301f-848442tw52"
Oct 31 17:52:59.811666 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:52:59.811641    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ce1ffbaa-b297-4c0b-b598-552d28abd520-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-single-main-78301f-848442tw52\" (UID: \"ce1ffbaa-b297-4c0b-b598-552d28abd520\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-78301f-848442tw52"
Oct 31 17:52:59.813519 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:52:59.813499    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1f51254c-4a54-4c60-b654-7658abec8fa8-maas-api-tls\") pod \"maas-api-e2e-worker-single-main-78301f-567555f6cc-lscgk\" (UID: \"1f51254c-4a54-4c60-b654-7658abec8fa8\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-78301f-567555f6cc-lscgk"
Oct 31 17:52:59.834070 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:52:59.834043    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vpncg\" (UniqueName: \"kubernetes.io/projected/ce1ffbaa-b297-4c0b-b598-552d28abd520-kube-api-access-vpncg\") pod \"payload-pre-processing-e2e-worker-single-main-78301f-848442tw52\" (UID: \"ce1ffbaa-b297-4c0b-b598-552d28abd520\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-78301f-848442tw52"
Oct 31 17:52:59.912308 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:52:59.912282    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7a22298b-5d98-45db-9831-f02ba58d0eac-plugins-config-volume\") pod \"payload-processing-e2e-worker-single-main-78301f-5db98f554cjjtc\" (UID: \"7a22298b-5d98-45db-9831-f02ba58d0eac\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-78301f-5db98f554cjjtc"
Oct 31 17:52:59.912500 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:52:59.912364    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f4g6t\" (UniqueName: \"kubernetes.io/projected/7a22298b-5d98-45db-9831-f02ba58d0eac-kube-api-access-f4g6t\") pod \"payload-processing-e2e-worker-single-main-78301f-5db98f554cjjtc\" (UID: \"7a22298b-5d98-45db-9831-f02ba58d0eac\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-78301f-5db98f554cjjtc"
Oct 31 17:52:59.913016 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:52:59.912991    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7a22298b-5d98-45db-9831-f02ba58d0eac-plugins-config-volume\") pod \"payload-processing-e2e-worker-single-main-78301f-5db98f554cjjtc\" (UID: \"7a22298b-5d98-45db-9831-f02ba58d0eac\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-78301f-5db98f554cjjtc"
Oct 31 17:52:59.930095 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:52:59.930073    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-78301f-848442tw52"
Oct 31 17:52:59.932290 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:52:59.932269    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f4g6t\" (UniqueName: \"kubernetes.io/projected/7a22298b-5d98-45db-9831-f02ba58d0eac-kube-api-access-f4g6t\") pod \"payload-processing-e2e-worker-single-main-78301f-5db98f554cjjtc\" (UID: \"7a22298b-5d98-45db-9831-f02ba58d0eac\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-78301f-5db98f554cjjtc"
Oct 31 17:53:00.018876 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:53:00.018848    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-single-main-78301f-5db98f554cjjtc"
Oct 31 17:53:00.085280 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:53:00.085250    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-78301f-567555f6cc-lscgk"
Oct 31 17:53:00.227480 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:53:00.227457    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-78301f-848442tw52"]
Oct 31 17:53:00.228993 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:53:00.228969    4276 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce1ffbaa_b297_4c0b_b598_552d28abd520.slice/crio-572b76f904cc6a10a0272d9822a142ce591ca423713dd6b2c9714873f19a5a3c WatchSource:0}: Error finding container 572b76f904cc6a10a0272d9822a142ce591ca423713dd6b2c9714873f19a5a3c: Status 404 returned error can't find the container with id 572b76f904cc6a10a0272d9822a142ce591ca423713dd6b2c9714873f19a5a3c
Oct 31 17:53:00.317150 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:53:00.317124    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-78301f-5db98f554cjjtc"]
Oct 31 17:53:00.319339 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:53:00.319280    4276 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a22298b_5d98_45db_9831_f02ba58d0eac.slice/crio-80d4312a42daf485d38c6e8da86cfe0b84504739769104c88ba50800609b490a WatchSource:0}: Error finding container 80d4312a42daf485d38c6e8da86cfe0b84504739769104c88ba50800609b490a: Status 404 returned error can't find the container with id 80d4312a42daf485d38c6e8da86cfe0b84504739769104c88ba50800609b490a
Oct 31 17:53:00.321430 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:53:00.321403    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-78301f-848442tw52" event={"ID":"ce1ffbaa-b297-4c0b-b598-552d28abd520","Type":"ContainerStarted","Data":"2583ef262fd8b0fa0db843110f2342536769d06e50b2b21022b85a3976095a64"}
Oct 31 17:53:00.321541 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:53:00.321469    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-78301f-848442tw52" event={"ID":"ce1ffbaa-b297-4c0b-b598-552d28abd520","Type":"ContainerStarted","Data":"572b76f904cc6a10a0272d9822a142ce591ca423713dd6b2c9714873f19a5a3c"}
Oct 31 17:53:00.321541 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:53:00.321502    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-78301f-848442tw52"
Oct 31 17:53:00.363500 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:53:00.363460    4276 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-78301f-848442tw52" podStartSLOduration=1.36342849 podStartE2EDuration="1.36342849s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 17:52:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 17:53:00.361945013 +0000 UTC m=+1543.159689941" watchObservedRunningTime="2026-10-31 17:53:00.36342849 +0000 UTC m=+1543.161173408"
Oct 31 17:53:00.394193 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:53:00.394169    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-78301f-567555f6cc-lscgk"]
Oct 31 17:53:00.394764 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:53:00.394739    4276 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f51254c_4a54_4c60_b654_7658abec8fa8.slice/crio-a24a346f2b49de6348b432aa66bc0e419f729023ccbb8f9f4dbbe3bdcd1dcc8e WatchSource:0}: Error finding container a24a346f2b49de6348b432aa66bc0e419f729023ccbb8f9f4dbbe3bdcd1dcc8e: Status 404 returned error can't find the container with id a24a346f2b49de6348b432aa66bc0e419f729023ccbb8f9f4dbbe3bdcd1dcc8e
Oct 31 17:53:01.205371 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:53:01.205296    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:53:01.327049 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:53:01.327018    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-single-main-78301f-5db98f554cjjtc" event={"ID":"7a22298b-5d98-45db-9831-f02ba58d0eac","Type":"ContainerStarted","Data":"038e73c95de56087274f70b2009396f5256e6fc58857eb24ca1bf9fd29bdb632"}
Oct 31 17:53:01.327522 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:53:01.327067    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-single-main-78301f-5db98f554cjjtc" event={"ID":"7a22298b-5d98-45db-9831-f02ba58d0eac","Type":"ContainerStarted","Data":"80d4312a42daf485d38c6e8da86cfe0b84504739769104c88ba50800609b490a"}
Oct 31 17:53:01.327522 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:53:01.327155    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-single-main-78301f-5db98f554cjjtc"
Oct 31 17:53:01.328751 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:53:01.328731    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-78301f-567555f6cc-lscgk" event={"ID":"1f51254c-4a54-4c60-b654-7658abec8fa8","Type":"ContainerStarted","Data":"46793c9e46a28be949323f26705f729d02a1ffdda147b3b7c66090c819991f7d"}
Oct 31 17:53:01.328830 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:53:01.328775    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-78301f-567555f6cc-lscgk"
Oct 31 17:53:01.328830 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:53:01.328793    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-78301f-567555f6cc-lscgk" event={"ID":"1f51254c-4a54-4c60-b654-7658abec8fa8","Type":"ContainerStarted","Data":"a24a346f2b49de6348b432aa66bc0e419f729023ccbb8f9f4dbbe3bdcd1dcc8e"}
Oct 31 17:53:01.369301 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:53:01.369254    4276 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-single-main-78301f-5db98f554cjjtc" podStartSLOduration=2.369243218 podStartE2EDuration="2.369243218s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 17:52:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 17:53:01.369207488 +0000 UTC m=+1544.166952415" watchObservedRunningTime="2026-10-31 17:53:01.369243218 +0000 UTC m=+1544.166988146"
Oct 31 17:53:01.411738 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:53:01.411694    4276 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-78301f-567555f6cc-lscgk" podStartSLOduration=2.010070277 podStartE2EDuration="2.411682389s" totalImagesPullingTime="401.612112ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 17:52:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 17:53:01.410509902 +0000 UTC m=+1544.208254830" watchObservedRunningTime="2026-10-31 17:53:01.411682389 +0000 UTC m=+1544.209427316"
Oct 31 17:53:02.917042 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:53:02.917012    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:53:04.638857 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:53:04.638823    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:53:04.807922 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:53:04.807891    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:53:06.577469 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:53:06.574877    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:53:07.340070 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:53:07.340042    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-78301f-567555f6cc-lscgk"
Oct 31 17:53:08.376681 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:53:08.376651    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:53:09.726057 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:53:09.726016    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:53:10.327098 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:53:10.327054    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:53:11.331850 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:53:11.331821    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-78301f-848442tw52"
Oct 31 17:53:12.013642 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:53:12.013605    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:53:12.234477 ip-10-0-129-127 kubenswrapper[4276]: E1031 17:53:12.234422    4276 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tg5qk" containerName="extractor"
Oct 31 17:53:12.337959 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:53:12.337934    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-single-main-78301f-5db98f554cjjtc"
Oct 31 17:53:12.339578 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:53:12.339544    4276 patch_prober.go:29] interesting pod/e2e-worker-single-main-78301f-openshift-default-867d4565f77l69h container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.86:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 17:53:12.339700 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:53:12.339614    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-single-main-78301f-openshift-default-867d4565f77l69h" podUID="8c6b1253-75fe-4ebd-8849-79cbe110cf50" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.86:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 17:53:12.487750 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:53:12.487709    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:53:13.039513 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:53:13.039431    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:53:13.159281 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:53:13.159209    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:53:13.281472 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:53:13.281416    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:53:13.410821 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:53:13.410740    4276 generic.go:356] "Generic (PLEG): container finished" podID="8c6b1253-75fe-4ebd-8849-79cbe110cf50" containerID="8206aefa0b1f92a92d921f37e05e813d4485b5ea71f53914063a9bec1f810e65" exitCode=0
Oct 31 17:53:13.410821 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:53:13.410808    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-78301f-openshift-default-867d4565f77l69h" event={"ID":"8c6b1253-75fe-4ebd-8849-79cbe110cf50","Type":"ContainerDied","Data":"8206aefa0b1f92a92d921f37e05e813d4485b5ea71f53914063a9bec1f810e65"}
Oct 31 17:53:13.411285 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:53:13.411184    4276 scope.go:122] "RemoveContainer" containerID="8206aefa0b1f92a92d921f37e05e813d4485b5ea71f53914063a9bec1f810e65"
Oct 31 17:53:13.412378 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:53:13.412352    4276 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 17:53:13.412434 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:53:13.412418    4276 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 17:53:13.412495 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:53:13.412470    4276 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 17:53:13.830471 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:53:13.830428    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:53:14.035315 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:53:14.035278    4276 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-single-main-78301f/simulator-single-main-78301f-kserve-764d4cdf6f-8qzhv"]
Oct 31 17:53:14.040404 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:53:14.040375    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-single-main-78301f/simulator-single-main-78301f-kserve-764d4cdf6f-8qzhv"
Oct 31 17:53:14.085810 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:53:14.085667    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-single-main-78301f/simulator-single-main-78301f-kserve-764d4cdf6f-8qzhv"]
Oct 31 17:53:14.099498 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:53:14.096266    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-single-main-78301f\"/\"simulator-single-main-78301f-kserve-self-signed-certs\""
Oct 31 17:53:14.099498 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:53:14.096357    4276 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-single-main-78301f\"/\"kube-root-ca.crt\""
Oct 31 17:53:14.099498 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:53:14.096414    4276 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-single-main-78301f\"/\"openshift-service-ca.crt\""
Oct 31 17:53:14.099498 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:53:14.096527    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-single-main-78301f\"/\"default-dockercfg-w2db5\""
Oct 31 17:53:14.116294 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:53:14.116264    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:53:14.182547 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:53:14.182512    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b6df8bc9-9cb4-4a3f-a53c-a0004fb26107-model-cache\") pod \"simulator-single-main-78301f-kserve-764d4cdf6f-8qzhv\" (UID: \"b6df8bc9-9cb4-4a3f-a53c-a0004fb26107\") " pod="e2e-models-e2e-worker-single-main-78301f/simulator-single-main-78301f-kserve-764d4cdf6f-8qzhv"
Oct 31 17:53:14.182765 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:53:14.182568    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b6df8bc9-9cb4-4a3f-a53c-a0004fb26107-tls-certs\") pod \"simulator-single-main-78301f-kserve-764d4cdf6f-8qzhv\" (UID: \"b6df8bc9-9cb4-4a3f-a53c-a0004fb26107\") " pod="e2e-models-e2e-worker-single-main-78301f/simulator-single-main-78301f-kserve-764d4cdf6f-8qzhv"
Oct 31 17:53:14.182765 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:53:14.182638    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9ph72\" (UniqueName: \"kubernetes.io/projected/b6df8bc9-9cb4-4a3f-a53c-a0004fb26107-kube-api-access-9ph72\") pod \"simulator-single-main-78301f-kserve-764d4cdf6f-8qzhv\" (UID: \"b6df8bc9-9cb4-4a3f-a53c-a0004fb26107\") " pod="e2e-models-e2e-worker-single-main-78301f/simulator-single-main-78301f-kserve-764d4cdf6f-8qzhv"
Oct 31 17:53:14.182882 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:53:14.182768    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b6df8bc9-9cb4-4a3f-a53c-a0004fb26107-dshm\") pod \"simulator-single-main-78301f-kserve-764d4cdf6f-8qzhv\" (UID: \"b6df8bc9-9cb4-4a3f-a53c-a0004fb26107\") " pod="e2e-models-e2e-worker-single-main-78301f/simulator-single-main-78301f-kserve-764d4cdf6f-8qzhv"
Oct 31 17:53:14.182882 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:53:14.182850    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b6df8bc9-9cb4-4a3f-a53c-a0004fb26107-tmp-dir\") pod \"simulator-single-main-78301f-kserve-764d4cdf6f-8qzhv\" (UID: \"b6df8bc9-9cb4-4a3f-a53c-a0004fb26107\") " pod="e2e-models-e2e-worker-single-main-78301f/simulator-single-main-78301f-kserve-764d4cdf6f-8qzhv"
Oct 31 17:53:14.183012 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:53:14.182884    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b6df8bc9-9cb4-4a3f-a53c-a0004fb26107-home\") pod \"simulator-single-main-78301f-kserve-764d4cdf6f-8qzhv\" (UID: \"b6df8bc9-9cb4-4a3f-a53c-a0004fb26107\") " pod="e2e-models-e2e-worker-single-main-78301f/simulator-single-main-78301f-kserve-764d4cdf6f-8qzhv"
Oct 31 17:53:14.284250 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:53:14.284216    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b6df8bc9-9cb4-4a3f-a53c-a0004fb26107-model-cache\") pod \"simulator-single-main-78301f-kserve-764d4cdf6f-8qzhv\" (UID: \"b6df8bc9-9cb4-4a3f-a53c-a0004fb26107\") " pod="e2e-models-e2e-worker-single-main-78301f/simulator-single-main-78301f-kserve-764d4cdf6f-8qzhv"
Oct 31 17:53:14.284572 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:53:14.284535    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b6df8bc9-9cb4-4a3f-a53c-a0004fb26107-tls-certs\") pod \"simulator-single-main-78301f-kserve-764d4cdf6f-8qzhv\" (UID: \"b6df8bc9-9cb4-4a3f-a53c-a0004fb26107\") " pod="e2e-models-e2e-worker-single-main-78301f/simulator-single-main-78301f-kserve-764d4cdf6f-8qzhv"
Oct 31 17:53:14.284725 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:53:14.284708    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9ph72\" (UniqueName: \"kubernetes.io/projected/b6df8bc9-9cb4-4a3f-a53c-a0004fb26107-kube-api-access-9ph72\") pod \"simulator-single-main-78301f-kserve-764d4cdf6f-8qzhv\" (UID: \"b6df8bc9-9cb4-4a3f-a53c-a0004fb26107\") " pod="e2e-models-e2e-worker-single-main-78301f/simulator-single-main-78301f-kserve-764d4cdf6f-8qzhv"
Oct 31 17:53:14.284924 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:53:14.284910    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b6df8bc9-9cb4-4a3f-a53c-a0004fb26107-dshm\") pod \"simulator-single-main-78301f-kserve-764d4cdf6f-8qzhv\" (UID: \"b6df8bc9-9cb4-4a3f-a53c-a0004fb26107\") " pod="e2e-models-e2e-worker-single-main-78301f/simulator-single-main-78301f-kserve-764d4cdf6f-8qzhv"
Oct 31 17:53:14.285181 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:53:14.285125    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b6df8bc9-9cb4-4a3f-a53c-a0004fb26107-tmp-dir\") pod \"simulator-single-main-78301f-kserve-764d4cdf6f-8qzhv\" (UID: \"b6df8bc9-9cb4-4a3f-a53c-a0004fb26107\") " pod="e2e-models-e2e-worker-single-main-78301f/simulator-single-main-78301f-kserve-764d4cdf6f-8qzhv"
Oct 31 17:53:14.285374 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:53:14.285356    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b6df8bc9-9cb4-4a3f-a53c-a0004fb26107-home\") pod \"simulator-single-main-78301f-kserve-764d4cdf6f-8qzhv\" (UID: \"b6df8bc9-9cb4-4a3f-a53c-a0004fb26107\") " pod="e2e-models-e2e-worker-single-main-78301f/simulator-single-main-78301f-kserve-764d4cdf6f-8qzhv"
Oct 31 17:53:14.285876 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:53:14.284997    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b6df8bc9-9cb4-4a3f-a53c-a0004fb26107-model-cache\") pod \"simulator-single-main-78301f-kserve-764d4cdf6f-8qzhv\" (UID: \"b6df8bc9-9cb4-4a3f-a53c-a0004fb26107\") " pod="e2e-models-e2e-worker-single-main-78301f/simulator-single-main-78301f-kserve-764d4cdf6f-8qzhv"
Oct 31 17:53:14.286288 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:53:14.286269    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b6df8bc9-9cb4-4a3f-a53c-a0004fb26107-tmp-dir\") pod \"simulator-single-main-78301f-kserve-764d4cdf6f-8qzhv\" (UID: \"b6df8bc9-9cb4-4a3f-a53c-a0004fb26107\") " pod="e2e-models-e2e-worker-single-main-78301f/simulator-single-main-78301f-kserve-764d4cdf6f-8qzhv"
Oct 31 17:53:14.292139 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:53:14.292108    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b6df8bc9-9cb4-4a3f-a53c-a0004fb26107-home\") pod \"simulator-single-main-78301f-kserve-764d4cdf6f-8qzhv\" (UID: \"b6df8bc9-9cb4-4a3f-a53c-a0004fb26107\") " pod="e2e-models-e2e-worker-single-main-78301f/simulator-single-main-78301f-kserve-764d4cdf6f-8qzhv"
Oct 31 17:53:14.292321 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:53:14.292224    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b6df8bc9-9cb4-4a3f-a53c-a0004fb26107-dshm\") pod \"simulator-single-main-78301f-kserve-764d4cdf6f-8qzhv\" (UID: \"b6df8bc9-9cb4-4a3f-a53c-a0004fb26107\") " pod="e2e-models-e2e-worker-single-main-78301f/simulator-single-main-78301f-kserve-764d4cdf6f-8qzhv"
Oct 31 17:53:14.293696 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:53:14.293664    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b6df8bc9-9cb4-4a3f-a53c-a0004fb26107-tls-certs\") pod \"simulator-single-main-78301f-kserve-764d4cdf6f-8qzhv\" (UID: \"b6df8bc9-9cb4-4a3f-a53c-a0004fb26107\") " pod="e2e-models-e2e-worker-single-main-78301f/simulator-single-main-78301f-kserve-764d4cdf6f-8qzhv"
Oct 31 17:53:14.306821 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:53:14.306789    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9ph72\" (UniqueName: \"kubernetes.io/projected/b6df8bc9-9cb4-4a3f-a53c-a0004fb26107-kube-api-access-9ph72\") pod \"simulator-single-main-78301f-kserve-764d4cdf6f-8qzhv\" (UID: \"b6df8bc9-9cb4-4a3f-a53c-a0004fb26107\") " pod="e2e-models-e2e-worker-single-main-78301f/simulator-single-main-78301f-kserve-764d4cdf6f-8qzhv"
Oct 31 17:53:14.389930 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:53:14.389842    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-single-main-78301f/simulator-single-main-78301f-kserve-764d4cdf6f-8qzhv"
Oct 31 17:53:14.417375 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:53:14.417343    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-78301f-openshift-default-867d4565f77l69h" event={"ID":"8c6b1253-75fe-4ebd-8849-79cbe110cf50","Type":"ContainerStarted","Data":"c5d0a42a935edd8342f69a360b90306c0df0a50d25109c434cb72c831facd5f7"}
Oct 31 17:53:14.420874 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:53:14.420852    4276 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-single-main-78301f-openshift-default-867d4565f77l69h"
Oct 31 17:53:14.421060 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:53:14.421042    4276 patch_prober.go:29] interesting pod/e2e-worker-single-main-78301f-openshift-default-867d4565f77l69h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused" start-of-body=
Oct 31 17:53:14.421104 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:53:14.421083    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-78301f-openshift-default-867d4565f77l69h" podUID="8c6b1253-75fe-4ebd-8849-79cbe110cf50" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused"
Oct 31 17:53:14.685699 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:53:14.685605    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:53:14.690975 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:53:14.690953    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-single-main-78301f/simulator-single-main-78301f-kserve-764d4cdf6f-8qzhv"]
Oct 31 17:53:14.692838 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:53:14.692814    4276 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb6df8bc9_9cb4_4a3f_a53c_a0004fb26107.slice/crio-9ebba97fcd5c4d34771ccab58caea5f05eb8fc8b3233ddace5e3b4e12e76fda6 WatchSource:0}: Error finding container 9ebba97fcd5c4d34771ccab58caea5f05eb8fc8b3233ddace5e3b4e12e76fda6: Status 404 returned error can't find the container with id 9ebba97fcd5c4d34771ccab58caea5f05eb8fc8b3233ddace5e3b4e12e76fda6
Oct 31 17:53:15.420757 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:53:15.420715    4276 patch_prober.go:29] interesting pod/e2e-worker-single-main-78301f-openshift-default-867d4565f77l69h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused" start-of-body=
Oct 31 17:53:15.421262 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:53:15.420774    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-78301f-openshift-default-867d4565f77l69h" podUID="8c6b1253-75fe-4ebd-8849-79cbe110cf50" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused"
Oct 31 17:53:15.423809 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:53:15.423778    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-single-main-78301f/simulator-single-main-78301f-kserve-764d4cdf6f-8qzhv" event={"ID":"b6df8bc9-9cb4-4a3f-a53c-a0004fb26107","Type":"ContainerStarted","Data":"4458582bb2d3462528e95f5b07b263407966c3f19488af1184689e5c97f56d54"}
Oct 31 17:53:15.423927 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:53:15.423823    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-single-main-78301f/simulator-single-main-78301f-kserve-764d4cdf6f-8qzhv" event={"ID":"b6df8bc9-9cb4-4a3f-a53c-a0004fb26107","Type":"ContainerStarted","Data":"9ebba97fcd5c4d34771ccab58caea5f05eb8fc8b3233ddace5e3b4e12e76fda6"}
Oct 31 17:53:15.499768 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:53:15.499720    4276 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-single-main-78301f/simulator-single-main-78301f-kserve-764d4cdf6f-8qzhv" podStartSLOduration=1.499705709 podStartE2EDuration="1.499705709s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 17:53:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 17:53:15.497966451 +0000 UTC m=+1558.295711381" watchObservedRunningTime="2026-10-31 17:53:15.499705709 +0000 UTC m=+1558.297450637"
Oct 31 17:53:15.638343 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:53:15.638307    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:53:16.390067 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:53:16.390034    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-single-main-78301f/simulator-single-main-78301f-kserve-764d4cdf6f-8qzhv"
Oct 31 17:53:16.420780 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:53:16.420739    4276 patch_prober.go:29] interesting pod/e2e-worker-single-main-78301f-openshift-default-867d4565f77l69h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused" start-of-body=
Oct 31 17:53:16.421228 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:53:16.420803    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-78301f-openshift-default-867d4565f77l69h" podUID="8c6b1253-75fe-4ebd-8849-79cbe110cf50" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused"
Oct 31 17:53:17.420626 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:53:17.420592    4276 patch_prober.go:29] interesting pod/e2e-worker-single-main-78301f-openshift-default-867d4565f77l69h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused" start-of-body=
Oct 31 17:53:17.420791 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:53:17.420650    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-78301f-openshift-default-867d4565f77l69h" podUID="8c6b1253-75fe-4ebd-8849-79cbe110cf50" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused"
Oct 31 17:53:17.538334 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:53:17.538296    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:53:18.421097 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:53:18.421063    4276 patch_prober.go:29] interesting pod/e2e-worker-single-main-78301f-openshift-default-867d4565f77l69h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused" start-of-body=
Oct 31 17:53:18.421307 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:53:18.421119    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-78301f-openshift-default-867d4565f77l69h" podUID="8c6b1253-75fe-4ebd-8849-79cbe110cf50" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused"
Oct 31 17:53:19.350435 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:53:19.350404    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:53:19.875375 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:53:19.875340    4276 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-single-main-78301f-openshift-default-867d4565f77l69h"
Oct 31 17:53:19.876083 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:53:19.876051    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-single-main-78301f-openshift-default-867d4565f77l69h"
Oct 31 17:53:19.877141 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:53:19.877117    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-single-main-78301f-openshift-default-867d4565f77l69h"
Oct 31 17:53:21.025167 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:53:21.025135    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:53:22.703998 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:53:22.703965    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:53:24.390758 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:53:24.390726    4276 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-single-main-78301f/simulator-single-main-78301f-kserve-764d4cdf6f-8qzhv"
Oct 31 17:53:24.403181 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:53:24.403154    4276 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-single-main-78301f/simulator-single-main-78301f-kserve-764d4cdf6f-8qzhv"
Oct 31 17:53:24.406735 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:53:24.406716    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:53:24.476491 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:53:24.476462    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-single-main-78301f/simulator-single-main-78301f-kserve-764d4cdf6f-8qzhv"
Oct 31 17:53:26.115952 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:53:26.115921    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:53:27.954782 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:53:27.954748    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:53:29.620349 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:53:29.620320    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:53:31.424775 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:53:31.424695    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:53:33.232433 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:53:33.232399    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:53:35.021986 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:53:35.021953    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:53:36.728846 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:53:36.728807    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:53:38.511841 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:53:38.511789    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:53:40.234560 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:53:40.234518    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:53:42.025548 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:53:42.025515    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:53:42.234328 ip-10-0-129-127 kubenswrapper[4276]: E1031 17:53:42.234287    4276 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tg5qk" containerName="extractor"
Oct 31 17:53:43.711658 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:53:43.711622    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:53:45.424150 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:53:45.424115    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:53:47.117422 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:53:47.117382    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:53:48.948802 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:53:48.948770    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:53:50.614476 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:53:50.614435    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:53:51.759343 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:53:51.759306    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:53:51.888189 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:53:51.888161    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:53:52.421840 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:53:52.421802    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:53:54.130611 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:53:54.130580    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:53:55.820475 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:53:55.820436    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:53:57.488234 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:53:57.488154    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:53:57.612762 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:53:57.612732    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:53:59.524560 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:53:59.524528    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:54:01.215666 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:01.215628    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:54:02.427938 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:02.427904    4276 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-single-main-78301f/simulator-single-main-78301f-kserve-764d4cdf6f-8qzhv"]
Oct 31 17:54:02.512157 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:02.512128    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:54:03.005876 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:03.005845    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:54:04.707512 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:04.707479    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:54:04.929784 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:04.929754    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:54:06.413693 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:06.413657    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:54:08.121511 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:08.121478    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:54:08.179271 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:08.179236    4276 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-78301f-5db98f554cjjtc"]
Oct 31 17:54:08.214381 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:08.214350    4276 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-78301f-848442tw52"]
Oct 31 17:54:08.343139 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:08.343112    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:54:08.449506 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:08.449382    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:54:09.473927 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:09.473890    4276 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-78301f-567555f6cc-lscgk"]
Oct 31 17:54:09.474340 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:09.474120    4276 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-78301f-567555f6cc-lscgk" podUID="1f51254c-4a54-4c60-b654-7658abec8fa8" containerName="maas-api" containerID="cri-o://46793c9e46a28be949323f26705f729d02a1ffdda147b3b7c66090c819991f7d" gracePeriod=30
Oct 31 17:54:09.555159 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:09.555092    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:54:09.816517 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:09.816488    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:54:09.950390 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:09.950366    4276 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-78301f-567555f6cc-lscgk"
Oct 31 17:54:10.030120 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:10.030046    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1f51254c-4a54-4c60-b654-7658abec8fa8-kube-api-access-xtf67\" (UniqueName: \"kubernetes.io/projected/1f51254c-4a54-4c60-b654-7658abec8fa8-kube-api-access-xtf67\") pod \"1f51254c-4a54-4c60-b654-7658abec8fa8\" (UID: \"1f51254c-4a54-4c60-b654-7658abec8fa8\") "
Oct 31 17:54:10.030263 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:10.030181    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1f51254c-4a54-4c60-b654-7658abec8fa8-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1f51254c-4a54-4c60-b654-7658abec8fa8-openshift-service-ca\") pod \"1f51254c-4a54-4c60-b654-7658abec8fa8\" (UID: \"1f51254c-4a54-4c60-b654-7658abec8fa8\") "
Oct 31 17:54:10.030263 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:10.030209    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1f51254c-4a54-4c60-b654-7658abec8fa8-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1f51254c-4a54-4c60-b654-7658abec8fa8-maas-api-tls\") pod \"1f51254c-4a54-4c60-b654-7658abec8fa8\" (UID: \"1f51254c-4a54-4c60-b654-7658abec8fa8\") "
Oct 31 17:54:10.030263 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:10.030226    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1f51254c-4a54-4c60-b654-7658abec8fa8-metrics-certs\" (UniqueName: \"kubernetes.io/secret/1f51254c-4a54-4c60-b654-7658abec8fa8-metrics-certs\") pod \"1f51254c-4a54-4c60-b654-7658abec8fa8\" (UID: \"1f51254c-4a54-4c60-b654-7658abec8fa8\") "
Oct 31 17:54:10.030540 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:10.030518    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1f51254c-4a54-4c60-b654-7658abec8fa8-openshift-service-ca" pod "1f51254c-4a54-4c60-b654-7658abec8fa8" (UID: "1f51254c-4a54-4c60-b654-7658abec8fa8"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 17:54:10.032189 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:10.032158    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1f51254c-4a54-4c60-b654-7658abec8fa8-kube-api-access-xtf67" pod "1f51254c-4a54-4c60-b654-7658abec8fa8" (UID: "1f51254c-4a54-4c60-b654-7658abec8fa8"). InnerVolumeSpecName "kube-api-access-xtf67". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 17:54:10.032321 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:10.032297    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1f51254c-4a54-4c60-b654-7658abec8fa8-maas-api-tls" pod "1f51254c-4a54-4c60-b654-7658abec8fa8" (UID: "1f51254c-4a54-4c60-b654-7658abec8fa8"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 17:54:10.032548 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:10.032426    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1f51254c-4a54-4c60-b654-7658abec8fa8-metrics-certs" pod "1f51254c-4a54-4c60-b654-7658abec8fa8" (UID: "1f51254c-4a54-4c60-b654-7658abec8fa8"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 17:54:10.088785 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:10.088759    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:54:10.130933 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:10.130900    4276 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xtf67\" (UniqueName: \"kubernetes.io/projected/1f51254c-4a54-4c60-b654-7658abec8fa8-kube-api-access-xtf67\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:54:10.130933 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:10.130932    4276 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1f51254c-4a54-4c60-b654-7658abec8fa8-openshift-service-ca\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:54:10.131098 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:10.130943    4276 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1f51254c-4a54-4c60-b654-7658abec8fa8-maas-api-tls\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:54:10.131098 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:10.130952    4276 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1f51254c-4a54-4c60-b654-7658abec8fa8-metrics-certs\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:54:10.204598 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:10.204573    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:54:10.678387 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:10.678353    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:54:10.683301 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:10.683277    4276 generic.go:356] "Generic (PLEG): container finished" podID="1f51254c-4a54-4c60-b654-7658abec8fa8" containerID="46793c9e46a28be949323f26705f729d02a1ffdda147b3b7c66090c819991f7d" exitCode=0
Oct 31 17:54:10.683423 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:10.683321    4276 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-78301f-567555f6cc-lscgk"
Oct 31 17:54:10.683423 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:10.683336    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-78301f-567555f6cc-lscgk" event={"ID":"1f51254c-4a54-4c60-b654-7658abec8fa8","Type":"ContainerDied","Data":"46793c9e46a28be949323f26705f729d02a1ffdda147b3b7c66090c819991f7d"}
Oct 31 17:54:10.683423 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:10.683375    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-78301f-567555f6cc-lscgk" event={"ID":"1f51254c-4a54-4c60-b654-7658abec8fa8","Type":"ContainerDied","Data":"a24a346f2b49de6348b432aa66bc0e419f729023ccbb8f9f4dbbe3bdcd1dcc8e"}
Oct 31 17:54:10.683423 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:10.683396    4276 scope.go:122] "RemoveContainer" containerID="46793c9e46a28be949323f26705f729d02a1ffdda147b3b7c66090c819991f7d"
Oct 31 17:54:10.692927 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:10.692899    4276 scope.go:122] "RemoveContainer" containerID="46793c9e46a28be949323f26705f729d02a1ffdda147b3b7c66090c819991f7d"
Oct 31 17:54:10.693184 ip-10-0-129-127 kubenswrapper[4276]: E1031 17:54:10.693166    4276 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"46793c9e46a28be949323f26705f729d02a1ffdda147b3b7c66090c819991f7d\": container with ID starting with 46793c9e46a28be949323f26705f729d02a1ffdda147b3b7c66090c819991f7d not found: ID does not exist" containerID="46793c9e46a28be949323f26705f729d02a1ffdda147b3b7c66090c819991f7d"
Oct 31 17:54:10.693241 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:10.693191    4276 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"46793c9e46a28be949323f26705f729d02a1ffdda147b3b7c66090c819991f7d"} err="failed to get container status \"46793c9e46a28be949323f26705f729d02a1ffdda147b3b7c66090c819991f7d\": rpc error: code = NotFound desc = could not find container \"46793c9e46a28be949323f26705f729d02a1ffdda147b3b7c66090c819991f7d\": container with ID starting with 46793c9e46a28be949323f26705f729d02a1ffdda147b3b7c66090c819991f7d not found: ID does not exist"
Oct 31 17:54:10.737412 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:10.737388    4276 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-78301f-567555f6cc-lscgk"]
Oct 31 17:54:10.750936 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:10.750909    4276 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-78301f-567555f6cc-lscgk"]
Oct 31 17:54:11.366896 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:11.366862    4276 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1f51254c-4a54-4c60-b654-7658abec8fa8" path="/var/lib/kubelet/pods/1f51254c-4a54-4c60-b654-7658abec8fa8/volumes"
Oct 31 17:54:12.234471 ip-10-0-129-127 kubenswrapper[4276]: E1031 17:54:12.234427    4276 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tg5qk" containerName="extractor"
Oct 31 17:54:13.179657 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:13.179621    4276 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-single-main-78301f-5db98f554cjjtc" podUID="7a22298b-5d98-45db-9831-f02ba58d0eac" containerName="payload-processing" containerID="cri-o://038e73c95de56087274f70b2009396f5256e6fc58857eb24ca1bf9fd29bdb632" gracePeriod=25
Oct 31 17:54:13.215177 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:13.215144    4276 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-78301f-848442tw52" podUID="ce1ffbaa-b297-4c0b-b598-552d28abd520" containerName="payload-pre-processing" containerID="cri-o://2583ef262fd8b0fa0db843110f2342536769d06e50b2b21022b85a3976095a64" gracePeriod=25
Oct 31 17:54:14.790838 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:14.790802    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:54:16.408171 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:16.408141    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:54:16.523612 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:16.523582    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:54:16.673381 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:16.673296    4276 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-single-main-78301f-openshift-default-867d4565f77l69h"]
Oct 31 17:54:16.673872 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:16.673843    4276 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-single-main-78301f-openshift-default-867d4565f77l69h" podUID="8c6b1253-75fe-4ebd-8849-79cbe110cf50" containerName="istio-proxy" containerID="cri-o://c5d0a42a935edd8342f69a360b90306c0df0a50d25109c434cb72c831facd5f7" gracePeriod=30
Oct 31 17:54:16.723884 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:16.723847    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:54:17.447533 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:17.447501    4276 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-single-main-78301f/simulator-single-main-78301f-kserve-764d4cdf6f-8qzhv" podUID="b6df8bc9-9cb4-4a3f-a53c-a0004fb26107" containerName="main" containerID="cri-o://4458582bb2d3462528e95f5b07b263407966c3f19488af1184689e5c97f56d54" gracePeriod=45
Oct 31 17:54:17.465743 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:17.465712    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-single-main-78301f/simulator-single-main-78301f-kserve-764d4cdf6f-8qzhv" podUID="b6df8bc9-9cb4-4a3f-a53c-a0004fb26107" containerName="main" probeResult="failure" output="Get \"https://10.133.0.90:8000/ready\": dial tcp 10.133.0.90:8000: connect: connection refused"
Oct 31 17:54:17.686265 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:17.686215    4276 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-single-main-78301f/simulator-single-main-78301f-kserve-764d4cdf6f-8qzhv"
Oct 31 17:54:17.718566 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:17.718492    4276 generic.go:356] "Generic (PLEG): container finished" podID="b6df8bc9-9cb4-4a3f-a53c-a0004fb26107" containerID="4458582bb2d3462528e95f5b07b263407966c3f19488af1184689e5c97f56d54" exitCode=0
Oct 31 17:54:17.718566 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:17.718528    4276 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-single-main-78301f/simulator-single-main-78301f-kserve-764d4cdf6f-8qzhv"
Oct 31 17:54:17.718566 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:17.718562    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-single-main-78301f/simulator-single-main-78301f-kserve-764d4cdf6f-8qzhv" event={"ID":"b6df8bc9-9cb4-4a3f-a53c-a0004fb26107","Type":"ContainerDied","Data":"4458582bb2d3462528e95f5b07b263407966c3f19488af1184689e5c97f56d54"}
Oct 31 17:54:17.718799 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:17.718600    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-single-main-78301f/simulator-single-main-78301f-kserve-764d4cdf6f-8qzhv" event={"ID":"b6df8bc9-9cb4-4a3f-a53c-a0004fb26107","Type":"ContainerDied","Data":"9ebba97fcd5c4d34771ccab58caea5f05eb8fc8b3233ddace5e3b4e12e76fda6"}
Oct 31 17:54:17.718799 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:17.718625    4276 scope.go:122] "RemoveContainer" containerID="4458582bb2d3462528e95f5b07b263407966c3f19488af1184689e5c97f56d54"
Oct 31 17:54:17.727269 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:17.727251    4276 scope.go:122] "RemoveContainer" containerID="4458582bb2d3462528e95f5b07b263407966c3f19488af1184689e5c97f56d54"
Oct 31 17:54:17.727542 ip-10-0-129-127 kubenswrapper[4276]: E1031 17:54:17.727525    4276 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4458582bb2d3462528e95f5b07b263407966c3f19488af1184689e5c97f56d54\": container with ID starting with 4458582bb2d3462528e95f5b07b263407966c3f19488af1184689e5c97f56d54 not found: ID does not exist" containerID="4458582bb2d3462528e95f5b07b263407966c3f19488af1184689e5c97f56d54"
Oct 31 17:54:17.727612 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:17.727550    4276 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4458582bb2d3462528e95f5b07b263407966c3f19488af1184689e5c97f56d54"} err="failed to get container status \"4458582bb2d3462528e95f5b07b263407966c3f19488af1184689e5c97f56d54\": rpc error: code = NotFound desc = could not find container \"4458582bb2d3462528e95f5b07b263407966c3f19488af1184689e5c97f56d54\": container with ID starting with 4458582bb2d3462528e95f5b07b263407966c3f19488af1184689e5c97f56d54 not found: ID does not exist"
Oct 31 17:54:17.806381 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:17.806356    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b6df8bc9-9cb4-4a3f-a53c-a0004fb26107-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b6df8bc9-9cb4-4a3f-a53c-a0004fb26107-model-cache\") pod \"b6df8bc9-9cb4-4a3f-a53c-a0004fb26107\" (UID: \"b6df8bc9-9cb4-4a3f-a53c-a0004fb26107\") "
Oct 31 17:54:17.806523 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:17.806406    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b6df8bc9-9cb4-4a3f-a53c-a0004fb26107-tls-certs\" (UniqueName: \"kubernetes.io/secret/b6df8bc9-9cb4-4a3f-a53c-a0004fb26107-tls-certs\") pod \"b6df8bc9-9cb4-4a3f-a53c-a0004fb26107\" (UID: \"b6df8bc9-9cb4-4a3f-a53c-a0004fb26107\") "
Oct 31 17:54:17.806523 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:17.806436    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b6df8bc9-9cb4-4a3f-a53c-a0004fb26107-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b6df8bc9-9cb4-4a3f-a53c-a0004fb26107-tmp-dir\") pod \"b6df8bc9-9cb4-4a3f-a53c-a0004fb26107\" (UID: \"b6df8bc9-9cb4-4a3f-a53c-a0004fb26107\") "
Oct 31 17:54:17.806653 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:17.806606    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b6df8bc9-9cb4-4a3f-a53c-a0004fb26107-kube-api-access-9ph72\" (UniqueName: \"kubernetes.io/projected/b6df8bc9-9cb4-4a3f-a53c-a0004fb26107-kube-api-access-9ph72\") pod \"b6df8bc9-9cb4-4a3f-a53c-a0004fb26107\" (UID: \"b6df8bc9-9cb4-4a3f-a53c-a0004fb26107\") "
Oct 31 17:54:17.806653 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:17.806618    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b6df8bc9-9cb4-4a3f-a53c-a0004fb26107-model-cache" pod "b6df8bc9-9cb4-4a3f-a53c-a0004fb26107" (UID: "b6df8bc9-9cb4-4a3f-a53c-a0004fb26107"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:54:17.806653 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:17.806639    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b6df8bc9-9cb4-4a3f-a53c-a0004fb26107-dshm\" (UniqueName: \"kubernetes.io/empty-dir/b6df8bc9-9cb4-4a3f-a53c-a0004fb26107-dshm\") pod \"b6df8bc9-9cb4-4a3f-a53c-a0004fb26107\" (UID: \"b6df8bc9-9cb4-4a3f-a53c-a0004fb26107\") "
Oct 31 17:54:17.806820 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:17.806698    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b6df8bc9-9cb4-4a3f-a53c-a0004fb26107-home\" (UniqueName: \"kubernetes.io/empty-dir/b6df8bc9-9cb4-4a3f-a53c-a0004fb26107-home\") pod \"b6df8bc9-9cb4-4a3f-a53c-a0004fb26107\" (UID: \"b6df8bc9-9cb4-4a3f-a53c-a0004fb26107\") "
Oct 31 17:54:17.806820 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:17.806726    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b6df8bc9-9cb4-4a3f-a53c-a0004fb26107-tmp-dir" pod "b6df8bc9-9cb4-4a3f-a53c-a0004fb26107" (UID: "b6df8bc9-9cb4-4a3f-a53c-a0004fb26107"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:54:17.807005 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:17.806983    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b6df8bc9-9cb4-4a3f-a53c-a0004fb26107-home" pod "b6df8bc9-9cb4-4a3f-a53c-a0004fb26107" (UID: "b6df8bc9-9cb4-4a3f-a53c-a0004fb26107"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:54:17.807092 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:17.807078    4276 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b6df8bc9-9cb4-4a3f-a53c-a0004fb26107-tmp-dir\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:54:17.807134 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:17.807094    4276 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b6df8bc9-9cb4-4a3f-a53c-a0004fb26107-home\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:54:17.807134 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:17.807103    4276 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b6df8bc9-9cb4-4a3f-a53c-a0004fb26107-model-cache\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:54:17.808745 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:17.808726    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b6df8bc9-9cb4-4a3f-a53c-a0004fb26107-kube-api-access-9ph72" pod "b6df8bc9-9cb4-4a3f-a53c-a0004fb26107" (UID: "b6df8bc9-9cb4-4a3f-a53c-a0004fb26107"). InnerVolumeSpecName "kube-api-access-9ph72". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 17:54:17.808965 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:17.808939    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b6df8bc9-9cb4-4a3f-a53c-a0004fb26107-dshm" pod "b6df8bc9-9cb4-4a3f-a53c-a0004fb26107" (UID: "b6df8bc9-9cb4-4a3f-a53c-a0004fb26107"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:54:17.809062 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:17.808944    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b6df8bc9-9cb4-4a3f-a53c-a0004fb26107-tls-certs" pod "b6df8bc9-9cb4-4a3f-a53c-a0004fb26107" (UID: "b6df8bc9-9cb4-4a3f-a53c-a0004fb26107"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 17:54:17.907840 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:17.907812    4276 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b6df8bc9-9cb4-4a3f-a53c-a0004fb26107-tls-certs\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:54:17.907840 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:17.907837    4276 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9ph72\" (UniqueName: \"kubernetes.io/projected/b6df8bc9-9cb4-4a3f-a53c-a0004fb26107-kube-api-access-9ph72\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:54:17.908013 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:17.907847    4276 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b6df8bc9-9cb4-4a3f-a53c-a0004fb26107-dshm\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:54:18.066778 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:18.066746    4276 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-single-main-78301f/simulator-single-main-78301f-kserve-764d4cdf6f-8qzhv"]
Oct 31 17:54:18.069223 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:18.069203    4276 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-single-main-78301f/simulator-single-main-78301f-kserve-764d4cdf6f-8qzhv"]
Oct 31 17:54:19.366795 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:19.366765    4276 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b6df8bc9-9cb4-4a3f-a53c-a0004fb26107" path="/var/lib/kubelet/pods/b6df8bc9-9cb4-4a3f-a53c-a0004fb26107/volumes"
Oct 31 17:54:19.878546 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:19.878513    4276 patch_prober.go:29] interesting pod/e2e-worker-single-main-78301f-openshift-default-867d4565f77l69h container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 31 17:54:19.878712 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:19.878586    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-single-main-78301f-openshift-default-867d4565f77l69h" podUID="8c6b1253-75fe-4ebd-8849-79cbe110cf50" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 17:54:21.111225 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:21.111191    4276 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-f51c0935-kserve-6c8bc48c45-kxhmm"]
Oct 31 17:54:21.111796 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:21.111776    4276 cpu_manager.go:414] "containerMap: removing container" podUID="b6df8bc9-9cb4-4a3f-a53c-a0004fb26107" containerName="main"
Oct 31 17:54:21.111835 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:21.111800    4276 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b6df8bc9-9cb4-4a3f-a53c-a0004fb26107" containerName="main"
Oct 31 17:54:21.111875 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:21.111851    4276 cpu_manager.go:414] "containerMap: removing container" podUID="1f51254c-4a54-4c60-b654-7658abec8fa8" containerName="maas-api"
Oct 31 17:54:21.111875 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:21.111860    4276 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1f51254c-4a54-4c60-b654-7658abec8fa8" containerName="maas-api"
Oct 31 17:54:21.111974 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:21.111962    4276 memory_manager.go:365] "RemoveStaleState removing state" podUID="1f51254c-4a54-4c60-b654-7658abec8fa8" containerName="maas-api"
Oct 31 17:54:21.112017 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:21.111982    4276 memory_manager.go:365] "RemoveStaleState removing state" podUID="b6df8bc9-9cb4-4a3f-a53c-a0004fb26107" containerName="main"
Oct 31 17:54:21.117812 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:21.117788    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-f51c0935-kserve-6c8bc48c45-kxhmm"
Oct 31 17:54:21.131154 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:21.131127    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-f51c0935-kserve-6c8bc48c45-kxhmm"]
Oct 31 17:54:21.143149 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:21.143127    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-f51c0935-kserve-self-signed-certs\""
Oct 31 17:54:21.177820 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:21.177788    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:54:21.239314 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:21.239287    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bdb6fbc3-29ac-449e-90c1-42b0aa4aa1a2-model-cache\") pod \"e2e-same-id-a-f51c0935-kserve-6c8bc48c45-kxhmm\" (UID: \"bdb6fbc3-29ac-449e-90c1-42b0aa4aa1a2\") " pod="llm/e2e-same-id-a-f51c0935-kserve-6c8bc48c45-kxhmm"
Oct 31 17:54:21.239477 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:21.239323    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bdb6fbc3-29ac-449e-90c1-42b0aa4aa1a2-home\") pod \"e2e-same-id-a-f51c0935-kserve-6c8bc48c45-kxhmm\" (UID: \"bdb6fbc3-29ac-449e-90c1-42b0aa4aa1a2\") " pod="llm/e2e-same-id-a-f51c0935-kserve-6c8bc48c45-kxhmm"
Oct 31 17:54:21.239477 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:21.239342    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bdb6fbc3-29ac-449e-90c1-42b0aa4aa1a2-dshm\") pod \"e2e-same-id-a-f51c0935-kserve-6c8bc48c45-kxhmm\" (UID: \"bdb6fbc3-29ac-449e-90c1-42b0aa4aa1a2\") " pod="llm/e2e-same-id-a-f51c0935-kserve-6c8bc48c45-kxhmm"
Oct 31 17:54:21.239477 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:21.239413    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bdb6fbc3-29ac-449e-90c1-42b0aa4aa1a2-tmp-dir\") pod \"e2e-same-id-a-f51c0935-kserve-6c8bc48c45-kxhmm\" (UID: \"bdb6fbc3-29ac-449e-90c1-42b0aa4aa1a2\") " pod="llm/e2e-same-id-a-f51c0935-kserve-6c8bc48c45-kxhmm"
Oct 31 17:54:21.239664 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:21.239573    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r5nfx\" (UniqueName: \"kubernetes.io/projected/bdb6fbc3-29ac-449e-90c1-42b0aa4aa1a2-kube-api-access-r5nfx\") pod \"e2e-same-id-a-f51c0935-kserve-6c8bc48c45-kxhmm\" (UID: \"bdb6fbc3-29ac-449e-90c1-42b0aa4aa1a2\") " pod="llm/e2e-same-id-a-f51c0935-kserve-6c8bc48c45-kxhmm"
Oct 31 17:54:21.239664 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:21.239614    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bdb6fbc3-29ac-449e-90c1-42b0aa4aa1a2-tls-certs\") pod \"e2e-same-id-a-f51c0935-kserve-6c8bc48c45-kxhmm\" (UID: \"bdb6fbc3-29ac-449e-90c1-42b0aa4aa1a2\") " pod="llm/e2e-same-id-a-f51c0935-kserve-6c8bc48c45-kxhmm"
Oct 31 17:54:21.330738 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:21.330703    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-78301f-848442tw52" podUID="ce1ffbaa-b297-4c0b-b598-552d28abd520" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 31 17:54:21.341133 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:21.341108    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r5nfx\" (UniqueName: \"kubernetes.io/projected/bdb6fbc3-29ac-449e-90c1-42b0aa4aa1a2-kube-api-access-r5nfx\") pod \"e2e-same-id-a-f51c0935-kserve-6c8bc48c45-kxhmm\" (UID: \"bdb6fbc3-29ac-449e-90c1-42b0aa4aa1a2\") " pod="llm/e2e-same-id-a-f51c0935-kserve-6c8bc48c45-kxhmm"
Oct 31 17:54:21.341253 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:21.341143    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bdb6fbc3-29ac-449e-90c1-42b0aa4aa1a2-tls-certs\") pod \"e2e-same-id-a-f51c0935-kserve-6c8bc48c45-kxhmm\" (UID: \"bdb6fbc3-29ac-449e-90c1-42b0aa4aa1a2\") " pod="llm/e2e-same-id-a-f51c0935-kserve-6c8bc48c45-kxhmm"
Oct 31 17:54:21.341318 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:21.341262    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bdb6fbc3-29ac-449e-90c1-42b0aa4aa1a2-model-cache\") pod \"e2e-same-id-a-f51c0935-kserve-6c8bc48c45-kxhmm\" (UID: \"bdb6fbc3-29ac-449e-90c1-42b0aa4aa1a2\") " pod="llm/e2e-same-id-a-f51c0935-kserve-6c8bc48c45-kxhmm"
Oct 31 17:54:21.341318 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:21.341306    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bdb6fbc3-29ac-449e-90c1-42b0aa4aa1a2-home\") pod \"e2e-same-id-a-f51c0935-kserve-6c8bc48c45-kxhmm\" (UID: \"bdb6fbc3-29ac-449e-90c1-42b0aa4aa1a2\") " pod="llm/e2e-same-id-a-f51c0935-kserve-6c8bc48c45-kxhmm"
Oct 31 17:54:21.341427 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:21.341326    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bdb6fbc3-29ac-449e-90c1-42b0aa4aa1a2-dshm\") pod \"e2e-same-id-a-f51c0935-kserve-6c8bc48c45-kxhmm\" (UID: \"bdb6fbc3-29ac-449e-90c1-42b0aa4aa1a2\") " pod="llm/e2e-same-id-a-f51c0935-kserve-6c8bc48c45-kxhmm"
Oct 31 17:54:21.341427 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:21.341360    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bdb6fbc3-29ac-449e-90c1-42b0aa4aa1a2-tmp-dir\") pod \"e2e-same-id-a-f51c0935-kserve-6c8bc48c45-kxhmm\" (UID: \"bdb6fbc3-29ac-449e-90c1-42b0aa4aa1a2\") " pod="llm/e2e-same-id-a-f51c0935-kserve-6c8bc48c45-kxhmm"
Oct 31 17:54:21.341740 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:21.341720    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bdb6fbc3-29ac-449e-90c1-42b0aa4aa1a2-model-cache\") pod \"e2e-same-id-a-f51c0935-kserve-6c8bc48c45-kxhmm\" (UID: \"bdb6fbc3-29ac-449e-90c1-42b0aa4aa1a2\") " pod="llm/e2e-same-id-a-f51c0935-kserve-6c8bc48c45-kxhmm"
Oct 31 17:54:21.341784 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:21.341727    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bdb6fbc3-29ac-449e-90c1-42b0aa4aa1a2-home\") pod \"e2e-same-id-a-f51c0935-kserve-6c8bc48c45-kxhmm\" (UID: \"bdb6fbc3-29ac-449e-90c1-42b0aa4aa1a2\") " pod="llm/e2e-same-id-a-f51c0935-kserve-6c8bc48c45-kxhmm"
Oct 31 17:54:21.341819 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:21.341803    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bdb6fbc3-29ac-449e-90c1-42b0aa4aa1a2-tmp-dir\") pod \"e2e-same-id-a-f51c0935-kserve-6c8bc48c45-kxhmm\" (UID: \"bdb6fbc3-29ac-449e-90c1-42b0aa4aa1a2\") " pod="llm/e2e-same-id-a-f51c0935-kserve-6c8bc48c45-kxhmm"
Oct 31 17:54:21.343736 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:21.343711    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bdb6fbc3-29ac-449e-90c1-42b0aa4aa1a2-dshm\") pod \"e2e-same-id-a-f51c0935-kserve-6c8bc48c45-kxhmm\" (UID: \"bdb6fbc3-29ac-449e-90c1-42b0aa4aa1a2\") " pod="llm/e2e-same-id-a-f51c0935-kserve-6c8bc48c45-kxhmm"
Oct 31 17:54:21.343996 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:21.343978    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bdb6fbc3-29ac-449e-90c1-42b0aa4aa1a2-tls-certs\") pod \"e2e-same-id-a-f51c0935-kserve-6c8bc48c45-kxhmm\" (UID: \"bdb6fbc3-29ac-449e-90c1-42b0aa4aa1a2\") " pod="llm/e2e-same-id-a-f51c0935-kserve-6c8bc48c45-kxhmm"
Oct 31 17:54:21.361729 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:21.361674    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r5nfx\" (UniqueName: \"kubernetes.io/projected/bdb6fbc3-29ac-449e-90c1-42b0aa4aa1a2-kube-api-access-r5nfx\") pod \"e2e-same-id-a-f51c0935-kserve-6c8bc48c45-kxhmm\" (UID: \"bdb6fbc3-29ac-449e-90c1-42b0aa4aa1a2\") " pod="llm/e2e-same-id-a-f51c0935-kserve-6c8bc48c45-kxhmm"
Oct 31 17:54:21.430562 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:21.430536    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-f51c0935-kserve-6c8bc48c45-kxhmm"
Oct 31 17:54:21.730389 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:21.730365    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-f51c0935-kserve-6c8bc48c45-kxhmm"]
Oct 31 17:54:21.763267 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:54:21.763240    4276 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbdb6fbc3_29ac_449e_90c1_42b0aa4aa1a2.slice/crio-d090b1047e06a5570924400490c10f9e1a9f8f0c55c2035f7c57199f7f391ae8 WatchSource:0}: Error finding container d090b1047e06a5570924400490c10f9e1a9f8f0c55c2035f7c57199f7f391ae8: Status 404 returned error can't find the container with id d090b1047e06a5570924400490c10f9e1a9f8f0c55c2035f7c57199f7f391ae8
Oct 31 17:54:21.916994 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:21.916970    4276 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-single-main-78301f-openshift-default-867d4565f77l69h"
Oct 31 17:54:22.049873 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:22.049850    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8c6b1253-75fe-4ebd-8849-79cbe110cf50-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8c6b1253-75fe-4ebd-8849-79cbe110cf50-istio-envoy\") pod \"8c6b1253-75fe-4ebd-8849-79cbe110cf50\" (UID: \"8c6b1253-75fe-4ebd-8849-79cbe110cf50\") "
Oct 31 17:54:22.050016 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:22.049930    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8c6b1253-75fe-4ebd-8849-79cbe110cf50-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c6b1253-75fe-4ebd-8849-79cbe110cf50-credential-socket\") pod \"8c6b1253-75fe-4ebd-8849-79cbe110cf50\" (UID: \"8c6b1253-75fe-4ebd-8849-79cbe110cf50\") "
Oct 31 17:54:22.050016 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:22.049954    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8c6b1253-75fe-4ebd-8849-79cbe110cf50-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8c6b1253-75fe-4ebd-8849-79cbe110cf50-istio-data\") pod \"8c6b1253-75fe-4ebd-8849-79cbe110cf50\" (UID: \"8c6b1253-75fe-4ebd-8849-79cbe110cf50\") "
Oct 31 17:54:22.050016 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:22.049979    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8c6b1253-75fe-4ebd-8849-79cbe110cf50-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c6b1253-75fe-4ebd-8849-79cbe110cf50-workload-socket\") pod \"8c6b1253-75fe-4ebd-8849-79cbe110cf50\" (UID: \"8c6b1253-75fe-4ebd-8849-79cbe110cf50\") "
Oct 31 17:54:22.050187 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:22.050017    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8c6b1253-75fe-4ebd-8849-79cbe110cf50-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8c6b1253-75fe-4ebd-8849-79cbe110cf50-workload-certs\") pod \"8c6b1253-75fe-4ebd-8849-79cbe110cf50\" (UID: \"8c6b1253-75fe-4ebd-8849-79cbe110cf50\") "
Oct 31 17:54:22.050187 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:22.050059    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8c6b1253-75fe-4ebd-8849-79cbe110cf50-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8c6b1253-75fe-4ebd-8849-79cbe110cf50-istiod-ca-cert\") pod \"8c6b1253-75fe-4ebd-8849-79cbe110cf50\" (UID: \"8c6b1253-75fe-4ebd-8849-79cbe110cf50\") "
Oct 31 17:54:22.050187 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:22.050076    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8c6b1253-75fe-4ebd-8849-79cbe110cf50-istio-token\" (UniqueName: \"kubernetes.io/projected/8c6b1253-75fe-4ebd-8849-79cbe110cf50-istio-token\") pod \"8c6b1253-75fe-4ebd-8849-79cbe110cf50\" (UID: \"8c6b1253-75fe-4ebd-8849-79cbe110cf50\") "
Oct 31 17:54:22.050187 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:22.050100    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8c6b1253-75fe-4ebd-8849-79cbe110cf50-kube-api-access-gwmzm\" (UniqueName: \"kubernetes.io/projected/8c6b1253-75fe-4ebd-8849-79cbe110cf50-kube-api-access-gwmzm\") pod \"8c6b1253-75fe-4ebd-8849-79cbe110cf50\" (UID: \"8c6b1253-75fe-4ebd-8849-79cbe110cf50\") "
Oct 31 17:54:22.050403 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:22.050228    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8c6b1253-75fe-4ebd-8849-79cbe110cf50-credential-socket" pod "8c6b1253-75fe-4ebd-8849-79cbe110cf50" (UID: "8c6b1253-75fe-4ebd-8849-79cbe110cf50"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:54:22.050403 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:22.050313    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8c6b1253-75fe-4ebd-8849-79cbe110cf50-workload-socket" pod "8c6b1253-75fe-4ebd-8849-79cbe110cf50" (UID: "8c6b1253-75fe-4ebd-8849-79cbe110cf50"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:54:22.050552 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:22.050506    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8c6b1253-75fe-4ebd-8849-79cbe110cf50-workload-certs" pod "8c6b1253-75fe-4ebd-8849-79cbe110cf50" (UID: "8c6b1253-75fe-4ebd-8849-79cbe110cf50"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:54:22.050552 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:22.050514    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/8c6b1253-75fe-4ebd-8849-79cbe110cf50-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8c6b1253-75fe-4ebd-8849-79cbe110cf50-istio-podinfo\") pod \"8c6b1253-75fe-4ebd-8849-79cbe110cf50\" (UID: \"8c6b1253-75fe-4ebd-8849-79cbe110cf50\") "
Oct 31 17:54:22.050552 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:22.050535    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8c6b1253-75fe-4ebd-8849-79cbe110cf50-istiod-ca-cert" pod "8c6b1253-75fe-4ebd-8849-79cbe110cf50" (UID: "8c6b1253-75fe-4ebd-8849-79cbe110cf50"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 17:54:22.050716 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:22.050663    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8c6b1253-75fe-4ebd-8849-79cbe110cf50-istio-data" pod "8c6b1253-75fe-4ebd-8849-79cbe110cf50" (UID: "8c6b1253-75fe-4ebd-8849-79cbe110cf50"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:54:22.050894 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:22.050875    4276 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c6b1253-75fe-4ebd-8849-79cbe110cf50-credential-socket\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:54:22.050977 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:22.050901    4276 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8c6b1253-75fe-4ebd-8849-79cbe110cf50-istio-data\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:54:22.050977 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:22.050916    4276 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c6b1253-75fe-4ebd-8849-79cbe110cf50-workload-socket\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:54:22.050977 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:22.050935    4276 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8c6b1253-75fe-4ebd-8849-79cbe110cf50-workload-certs\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:54:22.050977 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:22.050951    4276 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8c6b1253-75fe-4ebd-8849-79cbe110cf50-istiod-ca-cert\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:54:22.052334 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:22.052311    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8c6b1253-75fe-4ebd-8849-79cbe110cf50-kube-api-access-gwmzm" pod "8c6b1253-75fe-4ebd-8849-79cbe110cf50" (UID: "8c6b1253-75fe-4ebd-8849-79cbe110cf50"). InnerVolumeSpecName "kube-api-access-gwmzm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 17:54:22.052455 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:22.052401    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8c6b1253-75fe-4ebd-8849-79cbe110cf50-istio-podinfo" pod "8c6b1253-75fe-4ebd-8849-79cbe110cf50" (UID: "8c6b1253-75fe-4ebd-8849-79cbe110cf50"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 17:54:22.052508 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:22.052469    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8c6b1253-75fe-4ebd-8849-79cbe110cf50-istio-envoy" pod "8c6b1253-75fe-4ebd-8849-79cbe110cf50" (UID: "8c6b1253-75fe-4ebd-8849-79cbe110cf50"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:54:22.052547 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:22.052520    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8c6b1253-75fe-4ebd-8849-79cbe110cf50-istio-token" pod "8c6b1253-75fe-4ebd-8849-79cbe110cf50" (UID: "8c6b1253-75fe-4ebd-8849-79cbe110cf50"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 17:54:22.152240 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:22.152216    4276 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8c6b1253-75fe-4ebd-8849-79cbe110cf50-istio-envoy\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:54:22.152240 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:22.152241    4276 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8c6b1253-75fe-4ebd-8849-79cbe110cf50-istio-token\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:54:22.152691 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:22.152255    4276 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gwmzm\" (UniqueName: \"kubernetes.io/projected/8c6b1253-75fe-4ebd-8849-79cbe110cf50-kube-api-access-gwmzm\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:54:22.152691 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:22.152268    4276 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8c6b1253-75fe-4ebd-8849-79cbe110cf50-istio-podinfo\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:54:22.335871 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:22.335794    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-single-main-78301f-5db98f554cjjtc" podUID="7a22298b-5d98-45db-9831-f02ba58d0eac" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 31 17:54:22.743550 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:22.743464    4276 generic.go:356] "Generic (PLEG): container finished" podID="8c6b1253-75fe-4ebd-8849-79cbe110cf50" containerID="c5d0a42a935edd8342f69a360b90306c0df0a50d25109c434cb72c831facd5f7" exitCode=0
Oct 31 17:54:22.743550 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:22.743513    4276 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-single-main-78301f-openshift-default-867d4565f77l69h"
Oct 31 17:54:22.743794 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:22.743552    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-78301f-openshift-default-867d4565f77l69h" event={"ID":"8c6b1253-75fe-4ebd-8849-79cbe110cf50","Type":"ContainerDied","Data":"c5d0a42a935edd8342f69a360b90306c0df0a50d25109c434cb72c831facd5f7"}
Oct 31 17:54:22.743794 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:22.743611    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-78301f-openshift-default-867d4565f77l69h" event={"ID":"8c6b1253-75fe-4ebd-8849-79cbe110cf50","Type":"ContainerDied","Data":"03fbfd2dad1268b2dbf101d35b26ec61fa6418ea42aa348976f7f934eae31593"}
Oct 31 17:54:22.743794 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:22.743639    4276 scope.go:122] "RemoveContainer" containerID="c5d0a42a935edd8342f69a360b90306c0df0a50d25109c434cb72c831facd5f7"
Oct 31 17:54:22.745238 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:22.745215    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-f51c0935-kserve-6c8bc48c45-kxhmm" event={"ID":"bdb6fbc3-29ac-449e-90c1-42b0aa4aa1a2","Type":"ContainerStarted","Data":"58c2e100769a7af5db08fce2daeee9c9160bd9911699165322f2e51ac0c9886d"}
Oct 31 17:54:22.745238 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:22.745246    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-f51c0935-kserve-6c8bc48c45-kxhmm" event={"ID":"bdb6fbc3-29ac-449e-90c1-42b0aa4aa1a2","Type":"ContainerStarted","Data":"d090b1047e06a5570924400490c10f9e1a9f8f0c55c2035f7c57199f7f391ae8"}
Oct 31 17:54:22.752974 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:22.752956    4276 scope.go:122] "RemoveContainer" containerID="8206aefa0b1f92a92d921f37e05e813d4485b5ea71f53914063a9bec1f810e65"
Oct 31 17:54:22.760871 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:22.760854    4276 scope.go:122] "RemoveContainer" containerID="c5d0a42a935edd8342f69a360b90306c0df0a50d25109c434cb72c831facd5f7"
Oct 31 17:54:22.761118 ip-10-0-129-127 kubenswrapper[4276]: E1031 17:54:22.761100    4276 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c5d0a42a935edd8342f69a360b90306c0df0a50d25109c434cb72c831facd5f7\": container with ID starting with c5d0a42a935edd8342f69a360b90306c0df0a50d25109c434cb72c831facd5f7 not found: ID does not exist" containerID="c5d0a42a935edd8342f69a360b90306c0df0a50d25109c434cb72c831facd5f7"
Oct 31 17:54:22.761157 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:22.761129    4276 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c5d0a42a935edd8342f69a360b90306c0df0a50d25109c434cb72c831facd5f7"} err="failed to get container status \"c5d0a42a935edd8342f69a360b90306c0df0a50d25109c434cb72c831facd5f7\": rpc error: code = NotFound desc = could not find container \"c5d0a42a935edd8342f69a360b90306c0df0a50d25109c434cb72c831facd5f7\": container with ID starting with c5d0a42a935edd8342f69a360b90306c0df0a50d25109c434cb72c831facd5f7 not found: ID does not exist"
Oct 31 17:54:22.761157 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:22.761144    4276 scope.go:122] "RemoveContainer" containerID="8206aefa0b1f92a92d921f37e05e813d4485b5ea71f53914063a9bec1f810e65"
Oct 31 17:54:22.761372 ip-10-0-129-127 kubenswrapper[4276]: E1031 17:54:22.761357    4276 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8206aefa0b1f92a92d921f37e05e813d4485b5ea71f53914063a9bec1f810e65\": container with ID starting with 8206aefa0b1f92a92d921f37e05e813d4485b5ea71f53914063a9bec1f810e65 not found: ID does not exist" containerID="8206aefa0b1f92a92d921f37e05e813d4485b5ea71f53914063a9bec1f810e65"
Oct 31 17:54:22.761411 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:22.761378    4276 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8206aefa0b1f92a92d921f37e05e813d4485b5ea71f53914063a9bec1f810e65"} err="failed to get container status \"8206aefa0b1f92a92d921f37e05e813d4485b5ea71f53914063a9bec1f810e65\": rpc error: code = NotFound desc = could not find container \"8206aefa0b1f92a92d921f37e05e813d4485b5ea71f53914063a9bec1f810e65\": container with ID starting with 8206aefa0b1f92a92d921f37e05e813d4485b5ea71f53914063a9bec1f810e65 not found: ID does not exist"
Oct 31 17:54:22.815654 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:22.815594    4276 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-same-id-a-f51c0935-kserve-6c8bc48c45-kxhmm" podStartSLOduration=1.815578963 podStartE2EDuration="1.815578963s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 17:54:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 17:54:22.814780872 +0000 UTC m=+1625.612525797" watchObservedRunningTime="2026-10-31 17:54:22.815578963 +0000 UTC m=+1625.613323892"
Oct 31 17:54:22.853485 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:22.853454    4276 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-single-main-78301f-openshift-default-867d4565f77l69h"]
Oct 31 17:54:22.866638 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:22.866613    4276 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-single-main-78301f-openshift-default-867d4565f77l69h"]
Oct 31 17:54:23.366278 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:23.366238    4276 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8c6b1253-75fe-4ebd-8849-79cbe110cf50" path="/var/lib/kubelet/pods/8c6b1253-75fe-4ebd-8849-79cbe110cf50/volumes"
Oct 31 17:54:23.430653 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:23.430633    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-f51c0935-kserve-6c8bc48c45-kxhmm"
Oct 31 17:54:26.938728 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:26.938684    4276 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7d9897b54c-pfv5k container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.30:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 17:54:26.939141 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:26.938741    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-pfv5k" podUID="a0744b4f-5676-4385-b50c-5796ca2365a0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.30:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 17:54:31.330276 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:31.330230    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-78301f-848442tw52" podUID="ce1ffbaa-b297-4c0b-b598-552d28abd520" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 31 17:54:31.431384 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:31.431355    4276 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-f51c0935-kserve-6c8bc48c45-kxhmm"
Oct 31 17:54:31.444063 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:31.444033    4276 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-f51c0935-kserve-6c8bc48c45-kxhmm"
Oct 31 17:54:31.819747 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:31.819719    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-f51c0935-kserve-6c8bc48c45-kxhmm"
Oct 31 17:54:32.336006 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:32.335974    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-single-main-78301f-5db98f554cjjtc" podUID="7a22298b-5d98-45db-9831-f02ba58d0eac" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 31 17:54:32.486780 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:32.486734    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:54:33.468834 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:33.468808    4276 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-single-main-78301f-5db98f554cjjtc"
Oct 31 17:54:33.489011 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:33.488988    4276 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-78301f-848442tw52"
Oct 31 17:54:33.657088 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:33.657060    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ce1ffbaa-b297-4c0b-b598-552d28abd520-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ce1ffbaa-b297-4c0b-b598-552d28abd520-plugins-config-volume\") pod \"ce1ffbaa-b297-4c0b-b598-552d28abd520\" (UID: \"ce1ffbaa-b297-4c0b-b598-552d28abd520\") "
Oct 31 17:54:33.657257 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:33.657094    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7a22298b-5d98-45db-9831-f02ba58d0eac-kube-api-access-f4g6t\" (UniqueName: \"kubernetes.io/projected/7a22298b-5d98-45db-9831-f02ba58d0eac-kube-api-access-f4g6t\") pod \"7a22298b-5d98-45db-9831-f02ba58d0eac\" (UID: \"7a22298b-5d98-45db-9831-f02ba58d0eac\") "
Oct 31 17:54:33.657257 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:33.657142    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7a22298b-5d98-45db-9831-f02ba58d0eac-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7a22298b-5d98-45db-9831-f02ba58d0eac-plugins-config-volume\") pod \"7a22298b-5d98-45db-9831-f02ba58d0eac\" (UID: \"7a22298b-5d98-45db-9831-f02ba58d0eac\") "
Oct 31 17:54:33.657257 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:33.657167    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ce1ffbaa-b297-4c0b-b598-552d28abd520-kube-api-access-vpncg\" (UniqueName: \"kubernetes.io/projected/ce1ffbaa-b297-4c0b-b598-552d28abd520-kube-api-access-vpncg\") pod \"ce1ffbaa-b297-4c0b-b598-552d28abd520\" (UID: \"ce1ffbaa-b297-4c0b-b598-552d28abd520\") "
Oct 31 17:54:33.657557 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:33.657530    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ce1ffbaa-b297-4c0b-b598-552d28abd520-plugins-config-volume" pod "ce1ffbaa-b297-4c0b-b598-552d28abd520" (UID: "ce1ffbaa-b297-4c0b-b598-552d28abd520"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 17:54:33.657622 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:33.657535    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7a22298b-5d98-45db-9831-f02ba58d0eac-plugins-config-volume" pod "7a22298b-5d98-45db-9831-f02ba58d0eac" (UID: "7a22298b-5d98-45db-9831-f02ba58d0eac"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 17:54:33.659243 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:33.659221    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7a22298b-5d98-45db-9831-f02ba58d0eac-kube-api-access-f4g6t" pod "7a22298b-5d98-45db-9831-f02ba58d0eac" (UID: "7a22298b-5d98-45db-9831-f02ba58d0eac"). InnerVolumeSpecName "kube-api-access-f4g6t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 17:54:33.659305 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:33.659251    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ce1ffbaa-b297-4c0b-b598-552d28abd520-kube-api-access-vpncg" pod "ce1ffbaa-b297-4c0b-b598-552d28abd520" (UID: "ce1ffbaa-b297-4c0b-b598-552d28abd520"). InnerVolumeSpecName "kube-api-access-vpncg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 17:54:33.758784 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:33.758725    4276 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7a22298b-5d98-45db-9831-f02ba58d0eac-plugins-config-volume\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:54:33.758784 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:33.758747    4276 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vpncg\" (UniqueName: \"kubernetes.io/projected/ce1ffbaa-b297-4c0b-b598-552d28abd520-kube-api-access-vpncg\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:54:33.758784 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:33.758756    4276 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ce1ffbaa-b297-4c0b-b598-552d28abd520-plugins-config-volume\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:54:33.758784 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:33.758765    4276 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f4g6t\" (UniqueName: \"kubernetes.io/projected/7a22298b-5d98-45db-9831-f02ba58d0eac-kube-api-access-f4g6t\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:54:33.816853 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:33.816825    4276 generic.go:356] "Generic (PLEG): container finished" podID="7a22298b-5d98-45db-9831-f02ba58d0eac" containerID="038e73c95de56087274f70b2009396f5256e6fc58857eb24ca1bf9fd29bdb632" exitCode=0
Oct 31 17:54:33.816963 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:33.816877    4276 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-single-main-78301f-5db98f554cjjtc"
Oct 31 17:54:33.816963 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:33.816912    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-single-main-78301f-5db98f554cjjtc" event={"ID":"7a22298b-5d98-45db-9831-f02ba58d0eac","Type":"ContainerDied","Data":"038e73c95de56087274f70b2009396f5256e6fc58857eb24ca1bf9fd29bdb632"}
Oct 31 17:54:33.817087 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:33.816972    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-single-main-78301f-5db98f554cjjtc" event={"ID":"7a22298b-5d98-45db-9831-f02ba58d0eac","Type":"ContainerDied","Data":"80d4312a42daf485d38c6e8da86cfe0b84504739769104c88ba50800609b490a"}
Oct 31 17:54:33.817087 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:33.817002    4276 scope.go:122] "RemoveContainer" containerID="038e73c95de56087274f70b2009396f5256e6fc58857eb24ca1bf9fd29bdb632"
Oct 31 17:54:33.819048 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:33.819017    4276 generic.go:356] "Generic (PLEG): container finished" podID="ce1ffbaa-b297-4c0b-b598-552d28abd520" containerID="2583ef262fd8b0fa0db843110f2342536769d06e50b2b21022b85a3976095a64" exitCode=0
Oct 31 17:54:33.819108 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:33.819064    4276 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-78301f-848442tw52"
Oct 31 17:54:33.819156 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:33.819067    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-78301f-848442tw52" event={"ID":"ce1ffbaa-b297-4c0b-b598-552d28abd520","Type":"ContainerDied","Data":"2583ef262fd8b0fa0db843110f2342536769d06e50b2b21022b85a3976095a64"}
Oct 31 17:54:33.819206 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:33.819188    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-78301f-848442tw52" event={"ID":"ce1ffbaa-b297-4c0b-b598-552d28abd520","Type":"ContainerDied","Data":"572b76f904cc6a10a0272d9822a142ce591ca423713dd6b2c9714873f19a5a3c"}
Oct 31 17:54:33.829267 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:33.829249    4276 scope.go:122] "RemoveContainer" containerID="038e73c95de56087274f70b2009396f5256e6fc58857eb24ca1bf9fd29bdb632"
Oct 31 17:54:33.829581 ip-10-0-129-127 kubenswrapper[4276]: E1031 17:54:33.829564    4276 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"038e73c95de56087274f70b2009396f5256e6fc58857eb24ca1bf9fd29bdb632\": container with ID starting with 038e73c95de56087274f70b2009396f5256e6fc58857eb24ca1bf9fd29bdb632 not found: ID does not exist" containerID="038e73c95de56087274f70b2009396f5256e6fc58857eb24ca1bf9fd29bdb632"
Oct 31 17:54:33.829643 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:33.829587    4276 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"038e73c95de56087274f70b2009396f5256e6fc58857eb24ca1bf9fd29bdb632"} err="failed to get container status \"038e73c95de56087274f70b2009396f5256e6fc58857eb24ca1bf9fd29bdb632\": rpc error: code = NotFound desc = could not find container \"038e73c95de56087274f70b2009396f5256e6fc58857eb24ca1bf9fd29bdb632\": container with ID starting with 038e73c95de56087274f70b2009396f5256e6fc58857eb24ca1bf9fd29bdb632 not found: ID does not exist"
Oct 31 17:54:33.829643 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:33.829605    4276 scope.go:122] "RemoveContainer" containerID="2583ef262fd8b0fa0db843110f2342536769d06e50b2b21022b85a3976095a64"
Oct 31 17:54:33.837081 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:33.837065    4276 scope.go:122] "RemoveContainer" containerID="2583ef262fd8b0fa0db843110f2342536769d06e50b2b21022b85a3976095a64"
Oct 31 17:54:33.837331 ip-10-0-129-127 kubenswrapper[4276]: E1031 17:54:33.837310    4276 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2583ef262fd8b0fa0db843110f2342536769d06e50b2b21022b85a3976095a64\": container with ID starting with 2583ef262fd8b0fa0db843110f2342536769d06e50b2b21022b85a3976095a64 not found: ID does not exist" containerID="2583ef262fd8b0fa0db843110f2342536769d06e50b2b21022b85a3976095a64"
Oct 31 17:54:33.837398 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:33.837343    4276 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2583ef262fd8b0fa0db843110f2342536769d06e50b2b21022b85a3976095a64"} err="failed to get container status \"2583ef262fd8b0fa0db843110f2342536769d06e50b2b21022b85a3976095a64\": rpc error: code = NotFound desc = could not find container \"2583ef262fd8b0fa0db843110f2342536769d06e50b2b21022b85a3976095a64\": container with ID starting with 2583ef262fd8b0fa0db843110f2342536769d06e50b2b21022b85a3976095a64 not found: ID does not exist"
Oct 31 17:54:33.870095 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:33.870070    4276 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-78301f-5db98f554cjjtc"]
Oct 31 17:54:33.873750 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:33.873734    4276 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-78301f-5db98f554cjjtc"]
Oct 31 17:54:33.910055 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:33.910034    4276 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-78301f-848442tw52"]
Oct 31 17:54:33.913248 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:33.913231    4276 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-78301f-848442tw52"]
Oct 31 17:54:35.365593 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:35.365561    4276 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7a22298b-5d98-45db-9831-f02ba58d0eac" path="/var/lib/kubelet/pods/7a22298b-5d98-45db-9831-f02ba58d0eac/volumes"
Oct 31 17:54:35.365973 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:35.365928    4276 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ce1ffbaa-b297-4c0b-b598-552d28abd520" path="/var/lib/kubelet/pods/ce1ffbaa-b297-4c0b-b598-552d28abd520/volumes"
Oct 31 17:54:37.057295 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:37.057264    4276 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-f51c0935-kserve-7f9df8644f-rxhlw"]
Oct 31 17:54:37.057730 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:37.057714    4276 cpu_manager.go:414] "containerMap: removing container" podUID="7a22298b-5d98-45db-9831-f02ba58d0eac" containerName="payload-processing"
Oct 31 17:54:37.057782 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:37.057731    4276 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7a22298b-5d98-45db-9831-f02ba58d0eac" containerName="payload-processing"
Oct 31 17:54:37.057782 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:37.057748    4276 cpu_manager.go:414] "containerMap: removing container" podUID="8c6b1253-75fe-4ebd-8849-79cbe110cf50" containerName="istio-proxy"
Oct 31 17:54:37.057782 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:37.057755    4276 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8c6b1253-75fe-4ebd-8849-79cbe110cf50" containerName="istio-proxy"
Oct 31 17:54:37.057782 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:37.057766    4276 cpu_manager.go:414] "containerMap: removing container" podUID="8c6b1253-75fe-4ebd-8849-79cbe110cf50" containerName="istio-proxy"
Oct 31 17:54:37.057782 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:37.057771    4276 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8c6b1253-75fe-4ebd-8849-79cbe110cf50" containerName="istio-proxy"
Oct 31 17:54:37.057782 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:37.057778    4276 cpu_manager.go:414] "containerMap: removing container" podUID="ce1ffbaa-b297-4c0b-b598-552d28abd520" containerName="payload-pre-processing"
Oct 31 17:54:37.057955 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:37.057784    4276 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ce1ffbaa-b297-4c0b-b598-552d28abd520" containerName="payload-pre-processing"
Oct 31 17:54:37.057955 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:37.057858    4276 memory_manager.go:365] "RemoveStaleState removing state" podUID="7a22298b-5d98-45db-9831-f02ba58d0eac" containerName="payload-processing"
Oct 31 17:54:37.057955 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:37.057867    4276 memory_manager.go:365] "RemoveStaleState removing state" podUID="8c6b1253-75fe-4ebd-8849-79cbe110cf50" containerName="istio-proxy"
Oct 31 17:54:37.057955 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:37.057874    4276 memory_manager.go:365] "RemoveStaleState removing state" podUID="8c6b1253-75fe-4ebd-8849-79cbe110cf50" containerName="istio-proxy"
Oct 31 17:54:37.057955 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:37.057880    4276 memory_manager.go:365] "RemoveStaleState removing state" podUID="ce1ffbaa-b297-4c0b-b598-552d28abd520" containerName="payload-pre-processing"
Oct 31 17:54:37.062578 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:37.062550    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-f51c0935-kserve-7f9df8644f-rxhlw"
Oct 31 17:54:37.078905 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:37.078857    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-f51c0935-kserve-7f9df8644f-rxhlw"]
Oct 31 17:54:37.087685 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:37.087660    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/577a97f6-3638-4481-9b54-5e239b780118-tmp-dir\") pod \"e2e-same-id-b-f51c0935-kserve-7f9df8644f-rxhlw\" (UID: \"577a97f6-3638-4481-9b54-5e239b780118\") " pod="llm/e2e-same-id-b-f51c0935-kserve-7f9df8644f-rxhlw"
Oct 31 17:54:37.087815 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:37.087694    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/577a97f6-3638-4481-9b54-5e239b780118-home\") pod \"e2e-same-id-b-f51c0935-kserve-7f9df8644f-rxhlw\" (UID: \"577a97f6-3638-4481-9b54-5e239b780118\") " pod="llm/e2e-same-id-b-f51c0935-kserve-7f9df8644f-rxhlw"
Oct 31 17:54:37.087815 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:37.087723    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s7w8x\" (UniqueName: \"kubernetes.io/projected/577a97f6-3638-4481-9b54-5e239b780118-kube-api-access-s7w8x\") pod \"e2e-same-id-b-f51c0935-kserve-7f9df8644f-rxhlw\" (UID: \"577a97f6-3638-4481-9b54-5e239b780118\") " pod="llm/e2e-same-id-b-f51c0935-kserve-7f9df8644f-rxhlw"
Oct 31 17:54:37.087931 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:37.087905    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/577a97f6-3638-4481-9b54-5e239b780118-tls-certs\") pod \"e2e-same-id-b-f51c0935-kserve-7f9df8644f-rxhlw\" (UID: \"577a97f6-3638-4481-9b54-5e239b780118\") " pod="llm/e2e-same-id-b-f51c0935-kserve-7f9df8644f-rxhlw"
Oct 31 17:54:37.088028 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:37.088014    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/577a97f6-3638-4481-9b54-5e239b780118-dshm\") pod \"e2e-same-id-b-f51c0935-kserve-7f9df8644f-rxhlw\" (UID: \"577a97f6-3638-4481-9b54-5e239b780118\") " pod="llm/e2e-same-id-b-f51c0935-kserve-7f9df8644f-rxhlw"
Oct 31 17:54:37.088070 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:37.088042    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/577a97f6-3638-4481-9b54-5e239b780118-model-cache\") pod \"e2e-same-id-b-f51c0935-kserve-7f9df8644f-rxhlw\" (UID: \"577a97f6-3638-4481-9b54-5e239b780118\") " pod="llm/e2e-same-id-b-f51c0935-kserve-7f9df8644f-rxhlw"
Oct 31 17:54:37.090048 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:37.090025    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-f51c0935-kserve-self-signed-certs\""
Oct 31 17:54:37.134756 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:37.134724    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:54:37.188993 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:37.188960    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/577a97f6-3638-4481-9b54-5e239b780118-dshm\") pod \"e2e-same-id-b-f51c0935-kserve-7f9df8644f-rxhlw\" (UID: \"577a97f6-3638-4481-9b54-5e239b780118\") " pod="llm/e2e-same-id-b-f51c0935-kserve-7f9df8644f-rxhlw"
Oct 31 17:54:37.188993 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:37.188994    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/577a97f6-3638-4481-9b54-5e239b780118-model-cache\") pod \"e2e-same-id-b-f51c0935-kserve-7f9df8644f-rxhlw\" (UID: \"577a97f6-3638-4481-9b54-5e239b780118\") " pod="llm/e2e-same-id-b-f51c0935-kserve-7f9df8644f-rxhlw"
Oct 31 17:54:37.189233 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:37.189028    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/577a97f6-3638-4481-9b54-5e239b780118-tmp-dir\") pod \"e2e-same-id-b-f51c0935-kserve-7f9df8644f-rxhlw\" (UID: \"577a97f6-3638-4481-9b54-5e239b780118\") " pod="llm/e2e-same-id-b-f51c0935-kserve-7f9df8644f-rxhlw"
Oct 31 17:54:37.189233 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:37.189046    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/577a97f6-3638-4481-9b54-5e239b780118-home\") pod \"e2e-same-id-b-f51c0935-kserve-7f9df8644f-rxhlw\" (UID: \"577a97f6-3638-4481-9b54-5e239b780118\") " pod="llm/e2e-same-id-b-f51c0935-kserve-7f9df8644f-rxhlw"
Oct 31 17:54:37.189334 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:37.189245    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s7w8x\" (UniqueName: \"kubernetes.io/projected/577a97f6-3638-4481-9b54-5e239b780118-kube-api-access-s7w8x\") pod \"e2e-same-id-b-f51c0935-kserve-7f9df8644f-rxhlw\" (UID: \"577a97f6-3638-4481-9b54-5e239b780118\") " pod="llm/e2e-same-id-b-f51c0935-kserve-7f9df8644f-rxhlw"
Oct 31 17:54:37.189403 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:37.189377    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/577a97f6-3638-4481-9b54-5e239b780118-tls-certs\") pod \"e2e-same-id-b-f51c0935-kserve-7f9df8644f-rxhlw\" (UID: \"577a97f6-3638-4481-9b54-5e239b780118\") " pod="llm/e2e-same-id-b-f51c0935-kserve-7f9df8644f-rxhlw"
Oct 31 17:54:37.189403 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:37.189391    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/577a97f6-3638-4481-9b54-5e239b780118-model-cache\") pod \"e2e-same-id-b-f51c0935-kserve-7f9df8644f-rxhlw\" (UID: \"577a97f6-3638-4481-9b54-5e239b780118\") " pod="llm/e2e-same-id-b-f51c0935-kserve-7f9df8644f-rxhlw"
Oct 31 17:54:37.189535 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:37.189416    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/577a97f6-3638-4481-9b54-5e239b780118-home\") pod \"e2e-same-id-b-f51c0935-kserve-7f9df8644f-rxhlw\" (UID: \"577a97f6-3638-4481-9b54-5e239b780118\") " pod="llm/e2e-same-id-b-f51c0935-kserve-7f9df8644f-rxhlw"
Oct 31 17:54:37.189535 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:37.189496    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/577a97f6-3638-4481-9b54-5e239b780118-tmp-dir\") pod \"e2e-same-id-b-f51c0935-kserve-7f9df8644f-rxhlw\" (UID: \"577a97f6-3638-4481-9b54-5e239b780118\") " pod="llm/e2e-same-id-b-f51c0935-kserve-7f9df8644f-rxhlw"
Oct 31 17:54:37.191271 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:37.191252    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/577a97f6-3638-4481-9b54-5e239b780118-dshm\") pod \"e2e-same-id-b-f51c0935-kserve-7f9df8644f-rxhlw\" (UID: \"577a97f6-3638-4481-9b54-5e239b780118\") " pod="llm/e2e-same-id-b-f51c0935-kserve-7f9df8644f-rxhlw"
Oct 31 17:54:37.191676 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:37.191645    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/577a97f6-3638-4481-9b54-5e239b780118-tls-certs\") pod \"e2e-same-id-b-f51c0935-kserve-7f9df8644f-rxhlw\" (UID: \"577a97f6-3638-4481-9b54-5e239b780118\") " pod="llm/e2e-same-id-b-f51c0935-kserve-7f9df8644f-rxhlw"
Oct 31 17:54:37.209193 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:37.209171    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s7w8x\" (UniqueName: \"kubernetes.io/projected/577a97f6-3638-4481-9b54-5e239b780118-kube-api-access-s7w8x\") pod \"e2e-same-id-b-f51c0935-kserve-7f9df8644f-rxhlw\" (UID: \"577a97f6-3638-4481-9b54-5e239b780118\") " pod="llm/e2e-same-id-b-f51c0935-kserve-7f9df8644f-rxhlw"
Oct 31 17:54:37.379364 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:37.379289    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-f51c0935-kserve-7f9df8644f-rxhlw"
Oct 31 17:54:37.673594 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:37.673567    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-f51c0935-kserve-7f9df8644f-rxhlw"]
Oct 31 17:54:37.674919 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:54:37.674891    4276 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod577a97f6_3638_4481_9b54_5e239b780118.slice/crio-8a31900cdca8d52626892f291485695a2e0e3b442eab2976aa5e9a15d35b01d4 WatchSource:0}: Error finding container 8a31900cdca8d52626892f291485695a2e0e3b442eab2976aa5e9a15d35b01d4: Status 404 returned error can't find the container with id 8a31900cdca8d52626892f291485695a2e0e3b442eab2976aa5e9a15d35b01d4
Oct 31 17:54:37.840492 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:37.840455    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-f51c0935-kserve-7f9df8644f-rxhlw" event={"ID":"577a97f6-3638-4481-9b54-5e239b780118","Type":"ContainerStarted","Data":"d3f88e9ba317b3d95a4f3c98db457fcd1bccab2779b24737bdf27a014c12f164"}
Oct 31 17:54:37.840675 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:37.840504    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-f51c0935-kserve-7f9df8644f-rxhlw" event={"ID":"577a97f6-3638-4481-9b54-5e239b780118","Type":"ContainerStarted","Data":"8a31900cdca8d52626892f291485695a2e0e3b442eab2976aa5e9a15d35b01d4"}
Oct 31 17:54:37.881843 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:37.881795    4276 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-same-id-b-f51c0935-kserve-7f9df8644f-rxhlw" podStartSLOduration=0.881782966 podStartE2EDuration="881.782966ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 17:54:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 17:54:37.879913042 +0000 UTC m=+1640.677657969" watchObservedRunningTime="2026-10-31 17:54:37.881782966 +0000 UTC m=+1640.679527893"
Oct 31 17:54:38.379515 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:38.379489    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-f51c0935-kserve-7f9df8644f-rxhlw"
Oct 31 17:54:39.002031 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:39.002000    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:54:40.210577 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:40.210540    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:54:42.359680 ip-10-0-129-127 kubenswrapper[4276]: E1031 17:54:42.359608    4276 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tg5qk" containerName="extractor"
Oct 31 17:54:42.633669 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:42.633561    4276 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7d9897b54c-98fcj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 17:54:42.633968 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:42.633919    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-98fcj" podUID="9421021e-be39-4906-8fae-da1ce9ba4047" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 17:54:42.749576 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:42.749536    4276 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7d9897b54c-p4v97 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 17:54:42.749851 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:42.749823    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-p4v97" podUID="3ffeb078-cb4d-4293-b7fb-eb6fd305d96b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 17:54:45.997194 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:45.997161    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:54:47.380217 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:47.380185    4276 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-f51c0935-kserve-7f9df8644f-rxhlw"
Oct 31 17:54:47.392527 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:47.392505    4276 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-f51c0935-kserve-7f9df8644f-rxhlw"
Oct 31 17:54:47.920757 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:54:47.920725    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-f51c0935-kserve-7f9df8644f-rxhlw"
Oct 31 17:55:05.037177 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:55:05.037098    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:55:09.925069 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:55:09.925039    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:55:12.234141 ip-10-0-129-127 kubenswrapper[4276]: E1031 17:55:12.234106    4276 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tg5qk" containerName="extractor"
Oct 31 17:55:12.649964 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:55:12.649932    4276 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-f51c0935-kserve-6c8bc48c45-kxhmm"]
Oct 31 17:55:12.699131 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:55:12.699097    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:55:13.951149 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:55:13.951104    4276 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-f51c0935-kserve-7f9df8644f-rxhlw"]
Oct 31 17:55:14.019191 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:55:14.019163    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:55:14.900774 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:55:14.900736    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:55:27.670110 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:55:27.670050    4276 kuberuntime_container.go:898] "Killing container with a grace period" pod="llm/e2e-same-id-a-f51c0935-kserve-6c8bc48c45-kxhmm" podUID="bdb6fbc3-29ac-449e-90c1-42b0aa4aa1a2" containerName="main" containerID="cri-o://58c2e100769a7af5db08fce2daeee9c9160bd9911699165322f2e51ac0c9886d" gracePeriod=45
Oct 31 17:55:27.807615 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:55:27.807574    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/e2e-same-id-a-f51c0935-kserve-6c8bc48c45-kxhmm" podUID="bdb6fbc3-29ac-449e-90c1-42b0aa4aa1a2" containerName="main" probeResult="failure" output="Get \"https://10.133.0.91:8000/ready\": dial tcp 10.133.0.91:8000: connect: connection refused"
Oct 31 17:55:27.923280 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:55:27.923224    4276 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-f51c0935-kserve-6c8bc48c45-kxhmm"
Oct 31 17:55:28.002949 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:55:28.002905    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bdb6fbc3-29ac-449e-90c1-42b0aa4aa1a2-dshm\" (UniqueName: \"kubernetes.io/empty-dir/bdb6fbc3-29ac-449e-90c1-42b0aa4aa1a2-dshm\") pod \"bdb6fbc3-29ac-449e-90c1-42b0aa4aa1a2\" (UID: \"bdb6fbc3-29ac-449e-90c1-42b0aa4aa1a2\") "
Oct 31 17:55:28.003114 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:55:28.002970    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bdb6fbc3-29ac-449e-90c1-42b0aa4aa1a2-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bdb6fbc3-29ac-449e-90c1-42b0aa4aa1a2-tmp-dir\") pod \"bdb6fbc3-29ac-449e-90c1-42b0aa4aa1a2\" (UID: \"bdb6fbc3-29ac-449e-90c1-42b0aa4aa1a2\") "
Oct 31 17:55:28.003114 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:55:28.003036    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bdb6fbc3-29ac-449e-90c1-42b0aa4aa1a2-kube-api-access-r5nfx\" (UniqueName: \"kubernetes.io/projected/bdb6fbc3-29ac-449e-90c1-42b0aa4aa1a2-kube-api-access-r5nfx\") pod \"bdb6fbc3-29ac-449e-90c1-42b0aa4aa1a2\" (UID: \"bdb6fbc3-29ac-449e-90c1-42b0aa4aa1a2\") "
Oct 31 17:55:28.003114 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:55:28.003084    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bdb6fbc3-29ac-449e-90c1-42b0aa4aa1a2-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bdb6fbc3-29ac-449e-90c1-42b0aa4aa1a2-model-cache\") pod \"bdb6fbc3-29ac-449e-90c1-42b0aa4aa1a2\" (UID: \"bdb6fbc3-29ac-449e-90c1-42b0aa4aa1a2\") "
Oct 31 17:55:28.003114 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:55:28.003110    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bdb6fbc3-29ac-449e-90c1-42b0aa4aa1a2-home\" (UniqueName: \"kubernetes.io/empty-dir/bdb6fbc3-29ac-449e-90c1-42b0aa4aa1a2-home\") pod \"bdb6fbc3-29ac-449e-90c1-42b0aa4aa1a2\" (UID: \"bdb6fbc3-29ac-449e-90c1-42b0aa4aa1a2\") "
Oct 31 17:55:28.003311 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:55:28.003154    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/bdb6fbc3-29ac-449e-90c1-42b0aa4aa1a2-tls-certs\" (UniqueName: \"kubernetes.io/secret/bdb6fbc3-29ac-449e-90c1-42b0aa4aa1a2-tls-certs\") pod \"bdb6fbc3-29ac-449e-90c1-42b0aa4aa1a2\" (UID: \"bdb6fbc3-29ac-449e-90c1-42b0aa4aa1a2\") "
Oct 31 17:55:28.003371 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:55:28.003320    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bdb6fbc3-29ac-449e-90c1-42b0aa4aa1a2-tmp-dir" pod "bdb6fbc3-29ac-449e-90c1-42b0aa4aa1a2" (UID: "bdb6fbc3-29ac-449e-90c1-42b0aa4aa1a2"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:55:28.003429 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:55:28.003364    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bdb6fbc3-29ac-449e-90c1-42b0aa4aa1a2-model-cache" pod "bdb6fbc3-29ac-449e-90c1-42b0aa4aa1a2" (UID: "bdb6fbc3-29ac-449e-90c1-42b0aa4aa1a2"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:55:28.003429 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:55:28.003415    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bdb6fbc3-29ac-449e-90c1-42b0aa4aa1a2-home" pod "bdb6fbc3-29ac-449e-90c1-42b0aa4aa1a2" (UID: "bdb6fbc3-29ac-449e-90c1-42b0aa4aa1a2"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:55:28.003549 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:55:28.003536    4276 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bdb6fbc3-29ac-449e-90c1-42b0aa4aa1a2-tmp-dir\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:55:28.003591 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:55:28.003551    4276 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bdb6fbc3-29ac-449e-90c1-42b0aa4aa1a2-model-cache\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:55:28.003591 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:55:28.003562    4276 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bdb6fbc3-29ac-449e-90c1-42b0aa4aa1a2-home\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:55:28.005297 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:55:28.005269    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bdb6fbc3-29ac-449e-90c1-42b0aa4aa1a2-dshm" pod "bdb6fbc3-29ac-449e-90c1-42b0aa4aa1a2" (UID: "bdb6fbc3-29ac-449e-90c1-42b0aa4aa1a2"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:55:28.005403 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:55:28.005275    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bdb6fbc3-29ac-449e-90c1-42b0aa4aa1a2-kube-api-access-r5nfx" pod "bdb6fbc3-29ac-449e-90c1-42b0aa4aa1a2" (UID: "bdb6fbc3-29ac-449e-90c1-42b0aa4aa1a2"). InnerVolumeSpecName "kube-api-access-r5nfx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 17:55:28.005403 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:55:28.005275    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bdb6fbc3-29ac-449e-90c1-42b0aa4aa1a2-tls-certs" pod "bdb6fbc3-29ac-449e-90c1-42b0aa4aa1a2" (UID: "bdb6fbc3-29ac-449e-90c1-42b0aa4aa1a2"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 17:55:28.104047 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:55:28.104016    4276 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r5nfx\" (UniqueName: \"kubernetes.io/projected/bdb6fbc3-29ac-449e-90c1-42b0aa4aa1a2-kube-api-access-r5nfx\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:55:28.104047 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:55:28.104046    4276 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bdb6fbc3-29ac-449e-90c1-42b0aa4aa1a2-tls-certs\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:55:28.104047 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:55:28.104055    4276 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bdb6fbc3-29ac-449e-90c1-42b0aa4aa1a2-dshm\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:55:28.187261 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:55:28.187163    4276 generic.go:356] "Generic (PLEG): container finished" podID="bdb6fbc3-29ac-449e-90c1-42b0aa4aa1a2" containerID="58c2e100769a7af5db08fce2daeee9c9160bd9911699165322f2e51ac0c9886d" exitCode=0
Oct 31 17:55:28.187463 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:55:28.187324    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-f51c0935-kserve-6c8bc48c45-kxhmm" event={"ID":"bdb6fbc3-29ac-449e-90c1-42b0aa4aa1a2","Type":"ContainerDied","Data":"58c2e100769a7af5db08fce2daeee9c9160bd9911699165322f2e51ac0c9886d"}
Oct 31 17:55:28.187463 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:55:28.187362    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-f51c0935-kserve-6c8bc48c45-kxhmm" event={"ID":"bdb6fbc3-29ac-449e-90c1-42b0aa4aa1a2","Type":"ContainerDied","Data":"d090b1047e06a5570924400490c10f9e1a9f8f0c55c2035f7c57199f7f391ae8"}
Oct 31 17:55:28.187463 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:55:28.187393    4276 scope.go:122] "RemoveContainer" containerID="58c2e100769a7af5db08fce2daeee9c9160bd9911699165322f2e51ac0c9886d"
Oct 31 17:55:28.187698 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:55:28.187618    4276 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-f51c0935-kserve-6c8bc48c45-kxhmm"
Oct 31 17:55:28.199132 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:55:28.199114    4276 scope.go:122] "RemoveContainer" containerID="58c2e100769a7af5db08fce2daeee9c9160bd9911699165322f2e51ac0c9886d"
Oct 31 17:55:28.199495 ip-10-0-129-127 kubenswrapper[4276]: E1031 17:55:28.199471    4276 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"58c2e100769a7af5db08fce2daeee9c9160bd9911699165322f2e51ac0c9886d\": container with ID starting with 58c2e100769a7af5db08fce2daeee9c9160bd9911699165322f2e51ac0c9886d not found: ID does not exist" containerID="58c2e100769a7af5db08fce2daeee9c9160bd9911699165322f2e51ac0c9886d"
Oct 31 17:55:28.199554 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:55:28.199505    4276 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"58c2e100769a7af5db08fce2daeee9c9160bd9911699165322f2e51ac0c9886d"} err="failed to get container status \"58c2e100769a7af5db08fce2daeee9c9160bd9911699165322f2e51ac0c9886d\": rpc error: code = NotFound desc = could not find container \"58c2e100769a7af5db08fce2daeee9c9160bd9911699165322f2e51ac0c9886d\": container with ID starting with 58c2e100769a7af5db08fce2daeee9c9160bd9911699165322f2e51ac0c9886d not found: ID does not exist"
Oct 31 17:55:28.242932 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:55:28.242897    4276 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-f51c0935-kserve-6c8bc48c45-kxhmm"]
Oct 31 17:55:28.245906 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:55:28.245886    4276 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-f51c0935-kserve-6c8bc48c45-kxhmm"]
Oct 31 17:55:28.972163 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:55:28.972107    4276 kuberuntime_container.go:898] "Killing container with a grace period" pod="llm/e2e-same-id-b-f51c0935-kserve-7f9df8644f-rxhlw" podUID="577a97f6-3638-4481-9b54-5e239b780118" containerName="main" containerID="cri-o://d3f88e9ba317b3d95a4f3c98db457fcd1bccab2779b24737bdf27a014c12f164" gracePeriod=45
Oct 31 17:55:29.194358 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:55:29.194320    4276 generic.go:356] "Generic (PLEG): container finished" podID="577a97f6-3638-4481-9b54-5e239b780118" containerID="d3f88e9ba317b3d95a4f3c98db457fcd1bccab2779b24737bdf27a014c12f164" exitCode=0
Oct 31 17:55:29.194534 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:55:29.194384    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-f51c0935-kserve-7f9df8644f-rxhlw" event={"ID":"577a97f6-3638-4481-9b54-5e239b780118","Type":"ContainerDied","Data":"d3f88e9ba317b3d95a4f3c98db457fcd1bccab2779b24737bdf27a014c12f164"}
Oct 31 17:55:29.231395 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:55:29.231339    4276 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-f51c0935-kserve-7f9df8644f-rxhlw"
Oct 31 17:55:29.316909 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:55:29.316875    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/577a97f6-3638-4481-9b54-5e239b780118-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/577a97f6-3638-4481-9b54-5e239b780118-tmp-dir\") pod \"577a97f6-3638-4481-9b54-5e239b780118\" (UID: \"577a97f6-3638-4481-9b54-5e239b780118\") "
Oct 31 17:55:29.317096 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:55:29.316919    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/577a97f6-3638-4481-9b54-5e239b780118-tls-certs\" (UniqueName: \"kubernetes.io/secret/577a97f6-3638-4481-9b54-5e239b780118-tls-certs\") pod \"577a97f6-3638-4481-9b54-5e239b780118\" (UID: \"577a97f6-3638-4481-9b54-5e239b780118\") "
Oct 31 17:55:29.317096 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:55:29.316951    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/577a97f6-3638-4481-9b54-5e239b780118-home\" (UniqueName: \"kubernetes.io/empty-dir/577a97f6-3638-4481-9b54-5e239b780118-home\") pod \"577a97f6-3638-4481-9b54-5e239b780118\" (UID: \"577a97f6-3638-4481-9b54-5e239b780118\") "
Oct 31 17:55:29.317096 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:55:29.317027    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/577a97f6-3638-4481-9b54-5e239b780118-kube-api-access-s7w8x\" (UniqueName: \"kubernetes.io/projected/577a97f6-3638-4481-9b54-5e239b780118-kube-api-access-s7w8x\") pod \"577a97f6-3638-4481-9b54-5e239b780118\" (UID: \"577a97f6-3638-4481-9b54-5e239b780118\") "
Oct 31 17:55:29.317096 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:55:29.317056    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/577a97f6-3638-4481-9b54-5e239b780118-dshm\" (UniqueName: \"kubernetes.io/empty-dir/577a97f6-3638-4481-9b54-5e239b780118-dshm\") pod \"577a97f6-3638-4481-9b54-5e239b780118\" (UID: \"577a97f6-3638-4481-9b54-5e239b780118\") "
Oct 31 17:55:29.317376 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:55:29.317119    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/577a97f6-3638-4481-9b54-5e239b780118-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/577a97f6-3638-4481-9b54-5e239b780118-model-cache\") pod \"577a97f6-3638-4481-9b54-5e239b780118\" (UID: \"577a97f6-3638-4481-9b54-5e239b780118\") "
Oct 31 17:55:29.317376 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:55:29.317186    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/577a97f6-3638-4481-9b54-5e239b780118-tmp-dir" pod "577a97f6-3638-4481-9b54-5e239b780118" (UID: "577a97f6-3638-4481-9b54-5e239b780118"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:55:29.317376 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:55:29.317251    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/577a97f6-3638-4481-9b54-5e239b780118-home" pod "577a97f6-3638-4481-9b54-5e239b780118" (UID: "577a97f6-3638-4481-9b54-5e239b780118"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:55:29.317561 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:55:29.317381    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/577a97f6-3638-4481-9b54-5e239b780118-model-cache" pod "577a97f6-3638-4481-9b54-5e239b780118" (UID: "577a97f6-3638-4481-9b54-5e239b780118"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:55:29.317561 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:55:29.317544    4276 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/577a97f6-3638-4481-9b54-5e239b780118-home\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:55:29.317667 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:55:29.317564    4276 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/577a97f6-3638-4481-9b54-5e239b780118-model-cache\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:55:29.317667 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:55:29.317579    4276 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/577a97f6-3638-4481-9b54-5e239b780118-tmp-dir\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:55:29.319101 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:55:29.319072    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/577a97f6-3638-4481-9b54-5e239b780118-tls-certs" pod "577a97f6-3638-4481-9b54-5e239b780118" (UID: "577a97f6-3638-4481-9b54-5e239b780118"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 17:55:29.319229 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:55:29.319198    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/577a97f6-3638-4481-9b54-5e239b780118-kube-api-access-s7w8x" pod "577a97f6-3638-4481-9b54-5e239b780118" (UID: "577a97f6-3638-4481-9b54-5e239b780118"). InnerVolumeSpecName "kube-api-access-s7w8x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 17:55:29.319324 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:55:29.319305    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/577a97f6-3638-4481-9b54-5e239b780118-dshm" pod "577a97f6-3638-4481-9b54-5e239b780118" (UID: "577a97f6-3638-4481-9b54-5e239b780118"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:55:29.366241 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:55:29.366205    4276 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="bdb6fbc3-29ac-449e-90c1-42b0aa4aa1a2" path="/var/lib/kubelet/pods/bdb6fbc3-29ac-449e-90c1-42b0aa4aa1a2/volumes"
Oct 31 17:55:29.418271 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:55:29.418238    4276 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/577a97f6-3638-4481-9b54-5e239b780118-tls-certs\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:55:29.418271 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:55:29.418267    4276 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s7w8x\" (UniqueName: \"kubernetes.io/projected/577a97f6-3638-4481-9b54-5e239b780118-kube-api-access-s7w8x\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:55:29.418271 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:55:29.418278    4276 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/577a97f6-3638-4481-9b54-5e239b780118-dshm\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:55:30.216595 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:55:30.216555    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-f51c0935-kserve-7f9df8644f-rxhlw" event={"ID":"577a97f6-3638-4481-9b54-5e239b780118","Type":"ContainerDied","Data":"8a31900cdca8d52626892f291485695a2e0e3b442eab2976aa5e9a15d35b01d4"}
Oct 31 17:55:30.217182 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:55:30.217161    4276 scope.go:122] "RemoveContainer" containerID="d3f88e9ba317b3d95a4f3c98db457fcd1bccab2779b24737bdf27a014c12f164"
Oct 31 17:55:30.217412 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:55:30.217393    4276 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-f51c0935-kserve-7f9df8644f-rxhlw"
Oct 31 17:55:30.291477 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:55:30.288379    4276 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-f51c0935-kserve-7f9df8644f-rxhlw"]
Oct 31 17:55:30.307485 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:55:30.304832    4276 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-f51c0935-kserve-7f9df8644f-rxhlw"]
Oct 31 17:55:31.393024 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:55:31.392989    4276 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="577a97f6-3638-4481-9b54-5e239b780118" path="/var/lib/kubelet/pods/577a97f6-3638-4481-9b54-5e239b780118/volumes"
Oct 31 17:55:42.233981 ip-10-0-129-127 kubenswrapper[4276]: E1031 17:55:42.233940    4276 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tg5qk" containerName="extractor"
Oct 31 17:55:53.966900 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:55:53.966869    4276 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-f5575fb46-29pdk"]
Oct 31 17:55:53.967308 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:55:53.967294    4276 cpu_manager.go:414] "containerMap: removing container" podUID="bdb6fbc3-29ac-449e-90c1-42b0aa4aa1a2" containerName="main"
Oct 31 17:55:53.967355 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:55:53.967310    4276 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bdb6fbc3-29ac-449e-90c1-42b0aa4aa1a2" containerName="main"
Oct 31 17:55:53.967355 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:55:53.967321    4276 cpu_manager.go:414] "containerMap: removing container" podUID="577a97f6-3638-4481-9b54-5e239b780118" containerName="main"
Oct 31 17:55:53.967355 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:55:53.967329    4276 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="577a97f6-3638-4481-9b54-5e239b780118" containerName="main"
Oct 31 17:55:53.967472 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:55:53.967410    4276 memory_manager.go:365] "RemoveStaleState removing state" podUID="bdb6fbc3-29ac-449e-90c1-42b0aa4aa1a2" containerName="main"
Oct 31 17:55:53.967472 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:55:53.967419    4276 memory_manager.go:365] "RemoveStaleState removing state" podUID="577a97f6-3638-4481-9b54-5e239b780118" containerName="main"
Oct 31 17:55:53.970455 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:55:53.970424    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-f5575fb46-29pdk"
Oct 31 17:55:54.002571 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:55:54.002544    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-f5575fb46-29pdk"]
Oct 31 17:55:54.136542 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:55:54.136513    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cf94ce76-24ab-4767-b43d-3abfff4688f8-metrics-certs\") pod \"maas-api-f5575fb46-29pdk\" (UID: \"cf94ce76-24ab-4767-b43d-3abfff4688f8\") " pod="odh-ai-gateway-infra/maas-api-f5575fb46-29pdk"
Oct 31 17:55:54.136542 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:55:54.136546    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q5kw8\" (UniqueName: \"kubernetes.io/projected/cf94ce76-24ab-4767-b43d-3abfff4688f8-kube-api-access-q5kw8\") pod \"maas-api-f5575fb46-29pdk\" (UID: \"cf94ce76-24ab-4767-b43d-3abfff4688f8\") " pod="odh-ai-gateway-infra/maas-api-f5575fb46-29pdk"
Oct 31 17:55:54.136758 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:55:54.136571    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cf94ce76-24ab-4767-b43d-3abfff4688f8-openshift-service-ca\") pod \"maas-api-f5575fb46-29pdk\" (UID: \"cf94ce76-24ab-4767-b43d-3abfff4688f8\") " pod="odh-ai-gateway-infra/maas-api-f5575fb46-29pdk"
Oct 31 17:55:54.136758 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:55:54.136628    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cf94ce76-24ab-4767-b43d-3abfff4688f8-maas-api-tls\") pod \"maas-api-f5575fb46-29pdk\" (UID: \"cf94ce76-24ab-4767-b43d-3abfff4688f8\") " pod="odh-ai-gateway-infra/maas-api-f5575fb46-29pdk"
Oct 31 17:55:54.237242 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:55:54.237171    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cf94ce76-24ab-4767-b43d-3abfff4688f8-metrics-certs\") pod \"maas-api-f5575fb46-29pdk\" (UID: \"cf94ce76-24ab-4767-b43d-3abfff4688f8\") " pod="odh-ai-gateway-infra/maas-api-f5575fb46-29pdk"
Oct 31 17:55:54.237242 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:55:54.237205    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q5kw8\" (UniqueName: \"kubernetes.io/projected/cf94ce76-24ab-4767-b43d-3abfff4688f8-kube-api-access-q5kw8\") pod \"maas-api-f5575fb46-29pdk\" (UID: \"cf94ce76-24ab-4767-b43d-3abfff4688f8\") " pod="odh-ai-gateway-infra/maas-api-f5575fb46-29pdk"
Oct 31 17:55:54.237242 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:55:54.237227    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cf94ce76-24ab-4767-b43d-3abfff4688f8-openshift-service-ca\") pod \"maas-api-f5575fb46-29pdk\" (UID: \"cf94ce76-24ab-4767-b43d-3abfff4688f8\") " pod="odh-ai-gateway-infra/maas-api-f5575fb46-29pdk"
Oct 31 17:55:54.237531 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:55:54.237312    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cf94ce76-24ab-4767-b43d-3abfff4688f8-maas-api-tls\") pod \"maas-api-f5575fb46-29pdk\" (UID: \"cf94ce76-24ab-4767-b43d-3abfff4688f8\") " pod="odh-ai-gateway-infra/maas-api-f5575fb46-29pdk"
Oct 31 17:55:54.237936 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:55:54.237910    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cf94ce76-24ab-4767-b43d-3abfff4688f8-openshift-service-ca\") pod \"maas-api-f5575fb46-29pdk\" (UID: \"cf94ce76-24ab-4767-b43d-3abfff4688f8\") " pod="odh-ai-gateway-infra/maas-api-f5575fb46-29pdk"
Oct 31 17:55:54.239543 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:55:54.239524    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cf94ce76-24ab-4767-b43d-3abfff4688f8-maas-api-tls\") pod \"maas-api-f5575fb46-29pdk\" (UID: \"cf94ce76-24ab-4767-b43d-3abfff4688f8\") " pod="odh-ai-gateway-infra/maas-api-f5575fb46-29pdk"
Oct 31 17:55:54.239632 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:55:54.239575    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cf94ce76-24ab-4767-b43d-3abfff4688f8-metrics-certs\") pod \"maas-api-f5575fb46-29pdk\" (UID: \"cf94ce76-24ab-4767-b43d-3abfff4688f8\") " pod="odh-ai-gateway-infra/maas-api-f5575fb46-29pdk"
Oct 31 17:55:54.258417 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:55:54.258388    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q5kw8\" (UniqueName: \"kubernetes.io/projected/cf94ce76-24ab-4767-b43d-3abfff4688f8-kube-api-access-q5kw8\") pod \"maas-api-f5575fb46-29pdk\" (UID: \"cf94ce76-24ab-4767-b43d-3abfff4688f8\") " pod="odh-ai-gateway-infra/maas-api-f5575fb46-29pdk"
Oct 31 17:55:54.283951 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:55:54.283923    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-f5575fb46-29pdk"
Oct 31 17:55:54.577754 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:55:54.577727    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-f5575fb46-29pdk"]
Oct 31 17:55:54.579742 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:55:54.579720    4276 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf94ce76_24ab_4767_b43d_3abfff4688f8.slice/crio-0a9ad38dd97fe21594b41504ed0fcb743cb77facc45e4772f8fb30fff6daf86d WatchSource:0}: Error finding container 0a9ad38dd97fe21594b41504ed0fcb743cb77facc45e4772f8fb30fff6daf86d: Status 404 returned error can't find the container with id 0a9ad38dd97fe21594b41504ed0fcb743cb77facc45e4772f8fb30fff6daf86d
Oct 31 17:55:54.581042 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:55:54.581028    4276 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 31 17:55:55.397058 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:55:55.397022    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-f5575fb46-29pdk" event={"ID":"cf94ce76-24ab-4767-b43d-3abfff4688f8","Type":"ContainerStarted","Data":"7515edee8113521463790d2d4797f75f29d8c9a13a54333efaa15acc52db71eb"}
Oct 31 17:55:55.397569 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:55:55.397073    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-f5575fb46-29pdk" event={"ID":"cf94ce76-24ab-4767-b43d-3abfff4688f8","Type":"ContainerStarted","Data":"0a9ad38dd97fe21594b41504ed0fcb743cb77facc45e4772f8fb30fff6daf86d"}
Oct 31 17:55:55.397569 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:55:55.397150    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-f5575fb46-29pdk"
Oct 31 17:55:55.438811 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:55:55.438759    4276 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-f5575fb46-29pdk" podStartSLOduration=2.052322098 podStartE2EDuration="2.438746774s" totalImagesPullingTime="386.424676ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 17:55:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 17:55:55.438254585 +0000 UTC m=+1718.235999512" watchObservedRunningTime="2026-10-31 17:55:55.438746774 +0000 UTC m=+1718.236491702"
Oct 31 17:56:00.727645 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:00.727614    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:56:01.085270 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:01.085235    4276 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-5d82e915-openshift-default-5f9ffd98b7-95tbg"]
Oct 31 17:56:01.089770 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:01.089743    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-5d82e915-openshift-default-5f9ffd98b7-95tbg"
Oct 31 17:56:01.101511 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:01.101481    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cde72f26-a14d-4b50-8ab9-25d401a34d45-istio-data\") pod \"e2e-ait-5d82e915-openshift-default-5f9ffd98b7-95tbg\" (UID: \"cde72f26-a14d-4b50-8ab9-25d401a34d45\") " pod="openshift-ingress/e2e-ait-5d82e915-openshift-default-5f9ffd98b7-95tbg"
Oct 31 17:56:01.101629 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:01.101533    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cde72f26-a14d-4b50-8ab9-25d401a34d45-workload-socket\") pod \"e2e-ait-5d82e915-openshift-default-5f9ffd98b7-95tbg\" (UID: \"cde72f26-a14d-4b50-8ab9-25d401a34d45\") " pod="openshift-ingress/e2e-ait-5d82e915-openshift-default-5f9ffd98b7-95tbg"
Oct 31 17:56:01.101629 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:01.101615    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tm25t\" (UniqueName: \"kubernetes.io/projected/cde72f26-a14d-4b50-8ab9-25d401a34d45-kube-api-access-tm25t\") pod \"e2e-ait-5d82e915-openshift-default-5f9ffd98b7-95tbg\" (UID: \"cde72f26-a14d-4b50-8ab9-25d401a34d45\") " pod="openshift-ingress/e2e-ait-5d82e915-openshift-default-5f9ffd98b7-95tbg"
Oct 31 17:56:01.101748 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:01.101680    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cde72f26-a14d-4b50-8ab9-25d401a34d45-istio-token\") pod \"e2e-ait-5d82e915-openshift-default-5f9ffd98b7-95tbg\" (UID: \"cde72f26-a14d-4b50-8ab9-25d401a34d45\") " pod="openshift-ingress/e2e-ait-5d82e915-openshift-default-5f9ffd98b7-95tbg"
Oct 31 17:56:01.101818 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:01.101798    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cde72f26-a14d-4b50-8ab9-25d401a34d45-istio-envoy\") pod \"e2e-ait-5d82e915-openshift-default-5f9ffd98b7-95tbg\" (UID: \"cde72f26-a14d-4b50-8ab9-25d401a34d45\") " pod="openshift-ingress/e2e-ait-5d82e915-openshift-default-5f9ffd98b7-95tbg"
Oct 31 17:56:01.101866 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:01.101835    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cde72f26-a14d-4b50-8ab9-25d401a34d45-istio-podinfo\") pod \"e2e-ait-5d82e915-openshift-default-5f9ffd98b7-95tbg\" (UID: \"cde72f26-a14d-4b50-8ab9-25d401a34d45\") " pod="openshift-ingress/e2e-ait-5d82e915-openshift-default-5f9ffd98b7-95tbg"
Oct 31 17:56:01.101923 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:01.101908    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cde72f26-a14d-4b50-8ab9-25d401a34d45-credential-socket\") pod \"e2e-ait-5d82e915-openshift-default-5f9ffd98b7-95tbg\" (UID: \"cde72f26-a14d-4b50-8ab9-25d401a34d45\") " pod="openshift-ingress/e2e-ait-5d82e915-openshift-default-5f9ffd98b7-95tbg"
Oct 31 17:56:01.102029 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:01.102005    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cde72f26-a14d-4b50-8ab9-25d401a34d45-istiod-ca-cert\") pod \"e2e-ait-5d82e915-openshift-default-5f9ffd98b7-95tbg\" (UID: \"cde72f26-a14d-4b50-8ab9-25d401a34d45\") " pod="openshift-ingress/e2e-ait-5d82e915-openshift-default-5f9ffd98b7-95tbg"
Oct 31 17:56:01.102110 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:01.102095    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cde72f26-a14d-4b50-8ab9-25d401a34d45-workload-certs\") pod \"e2e-ait-5d82e915-openshift-default-5f9ffd98b7-95tbg\" (UID: \"cde72f26-a14d-4b50-8ab9-25d401a34d45\") " pod="openshift-ingress/e2e-ait-5d82e915-openshift-default-5f9ffd98b7-95tbg"
Oct 31 17:56:01.105988 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:01.105968    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-5d82e915-openshift-default-dockercfg-hdh4l\""
Oct 31 17:56:01.121596 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:01.121568    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-5d82e915-openshift-default-5f9ffd98b7-95tbg"]
Oct 31 17:56:01.160894 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:01.160866    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:56:01.203134 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:01.203100    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cde72f26-a14d-4b50-8ab9-25d401a34d45-istio-token\") pod \"e2e-ait-5d82e915-openshift-default-5f9ffd98b7-95tbg\" (UID: \"cde72f26-a14d-4b50-8ab9-25d401a34d45\") " pod="openshift-ingress/e2e-ait-5d82e915-openshift-default-5f9ffd98b7-95tbg"
Oct 31 17:56:01.203315 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:01.203176    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cde72f26-a14d-4b50-8ab9-25d401a34d45-istio-envoy\") pod \"e2e-ait-5d82e915-openshift-default-5f9ffd98b7-95tbg\" (UID: \"cde72f26-a14d-4b50-8ab9-25d401a34d45\") " pod="openshift-ingress/e2e-ait-5d82e915-openshift-default-5f9ffd98b7-95tbg"
Oct 31 17:56:01.203315 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:01.203207    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cde72f26-a14d-4b50-8ab9-25d401a34d45-istio-podinfo\") pod \"e2e-ait-5d82e915-openshift-default-5f9ffd98b7-95tbg\" (UID: \"cde72f26-a14d-4b50-8ab9-25d401a34d45\") " pod="openshift-ingress/e2e-ait-5d82e915-openshift-default-5f9ffd98b7-95tbg"
Oct 31 17:56:01.203315 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:01.203244    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cde72f26-a14d-4b50-8ab9-25d401a34d45-credential-socket\") pod \"e2e-ait-5d82e915-openshift-default-5f9ffd98b7-95tbg\" (UID: \"cde72f26-a14d-4b50-8ab9-25d401a34d45\") " pod="openshift-ingress/e2e-ait-5d82e915-openshift-default-5f9ffd98b7-95tbg"
Oct 31 17:56:01.203315 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:01.203296    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cde72f26-a14d-4b50-8ab9-25d401a34d45-istiod-ca-cert\") pod \"e2e-ait-5d82e915-openshift-default-5f9ffd98b7-95tbg\" (UID: \"cde72f26-a14d-4b50-8ab9-25d401a34d45\") " pod="openshift-ingress/e2e-ait-5d82e915-openshift-default-5f9ffd98b7-95tbg"
Oct 31 17:56:01.203553 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:01.203359    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cde72f26-a14d-4b50-8ab9-25d401a34d45-workload-certs\") pod \"e2e-ait-5d82e915-openshift-default-5f9ffd98b7-95tbg\" (UID: \"cde72f26-a14d-4b50-8ab9-25d401a34d45\") " pod="openshift-ingress/e2e-ait-5d82e915-openshift-default-5f9ffd98b7-95tbg"
Oct 31 17:56:01.203553 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:01.203387    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cde72f26-a14d-4b50-8ab9-25d401a34d45-istio-data\") pod \"e2e-ait-5d82e915-openshift-default-5f9ffd98b7-95tbg\" (UID: \"cde72f26-a14d-4b50-8ab9-25d401a34d45\") " pod="openshift-ingress/e2e-ait-5d82e915-openshift-default-5f9ffd98b7-95tbg"
Oct 31 17:56:01.203553 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:01.203419    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cde72f26-a14d-4b50-8ab9-25d401a34d45-workload-socket\") pod \"e2e-ait-5d82e915-openshift-default-5f9ffd98b7-95tbg\" (UID: \"cde72f26-a14d-4b50-8ab9-25d401a34d45\") " pod="openshift-ingress/e2e-ait-5d82e915-openshift-default-5f9ffd98b7-95tbg"
Oct 31 17:56:01.203553 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:01.203472    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tm25t\" (UniqueName: \"kubernetes.io/projected/cde72f26-a14d-4b50-8ab9-25d401a34d45-kube-api-access-tm25t\") pod \"e2e-ait-5d82e915-openshift-default-5f9ffd98b7-95tbg\" (UID: \"cde72f26-a14d-4b50-8ab9-25d401a34d45\") " pod="openshift-ingress/e2e-ait-5d82e915-openshift-default-5f9ffd98b7-95tbg"
Oct 31 17:56:01.204105 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:01.204077    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cde72f26-a14d-4b50-8ab9-25d401a34d45-workload-certs\") pod \"e2e-ait-5d82e915-openshift-default-5f9ffd98b7-95tbg\" (UID: \"cde72f26-a14d-4b50-8ab9-25d401a34d45\") " pod="openshift-ingress/e2e-ait-5d82e915-openshift-default-5f9ffd98b7-95tbg"
Oct 31 17:56:01.204266 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:01.204236    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cde72f26-a14d-4b50-8ab9-25d401a34d45-istiod-ca-cert\") pod \"e2e-ait-5d82e915-openshift-default-5f9ffd98b7-95tbg\" (UID: \"cde72f26-a14d-4b50-8ab9-25d401a34d45\") " pod="openshift-ingress/e2e-ait-5d82e915-openshift-default-5f9ffd98b7-95tbg"
Oct 31 17:56:01.204395 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:01.204329    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cde72f26-a14d-4b50-8ab9-25d401a34d45-istio-data\") pod \"e2e-ait-5d82e915-openshift-default-5f9ffd98b7-95tbg\" (UID: \"cde72f26-a14d-4b50-8ab9-25d401a34d45\") " pod="openshift-ingress/e2e-ait-5d82e915-openshift-default-5f9ffd98b7-95tbg"
Oct 31 17:56:01.204601 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:01.204579    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cde72f26-a14d-4b50-8ab9-25d401a34d45-workload-socket\") pod \"e2e-ait-5d82e915-openshift-default-5f9ffd98b7-95tbg\" (UID: \"cde72f26-a14d-4b50-8ab9-25d401a34d45\") " pod="openshift-ingress/e2e-ait-5d82e915-openshift-default-5f9ffd98b7-95tbg"
Oct 31 17:56:01.204832 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:01.204808    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cde72f26-a14d-4b50-8ab9-25d401a34d45-credential-socket\") pod \"e2e-ait-5d82e915-openshift-default-5f9ffd98b7-95tbg\" (UID: \"cde72f26-a14d-4b50-8ab9-25d401a34d45\") " pod="openshift-ingress/e2e-ait-5d82e915-openshift-default-5f9ffd98b7-95tbg"
Oct 31 17:56:01.207083 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:01.207058    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cde72f26-a14d-4b50-8ab9-25d401a34d45-istio-envoy\") pod \"e2e-ait-5d82e915-openshift-default-5f9ffd98b7-95tbg\" (UID: \"cde72f26-a14d-4b50-8ab9-25d401a34d45\") " pod="openshift-ingress/e2e-ait-5d82e915-openshift-default-5f9ffd98b7-95tbg"
Oct 31 17:56:01.207784 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:01.207757    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cde72f26-a14d-4b50-8ab9-25d401a34d45-istio-podinfo\") pod \"e2e-ait-5d82e915-openshift-default-5f9ffd98b7-95tbg\" (UID: \"cde72f26-a14d-4b50-8ab9-25d401a34d45\") " pod="openshift-ingress/e2e-ait-5d82e915-openshift-default-5f9ffd98b7-95tbg"
Oct 31 17:56:01.228668 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:01.228641    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cde72f26-a14d-4b50-8ab9-25d401a34d45-istio-token\") pod \"e2e-ait-5d82e915-openshift-default-5f9ffd98b7-95tbg\" (UID: \"cde72f26-a14d-4b50-8ab9-25d401a34d45\") " pod="openshift-ingress/e2e-ait-5d82e915-openshift-default-5f9ffd98b7-95tbg"
Oct 31 17:56:01.229318 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:01.229298    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tm25t\" (UniqueName: \"kubernetes.io/projected/cde72f26-a14d-4b50-8ab9-25d401a34d45-kube-api-access-tm25t\") pod \"e2e-ait-5d82e915-openshift-default-5f9ffd98b7-95tbg\" (UID: \"cde72f26-a14d-4b50-8ab9-25d401a34d45\") " pod="openshift-ingress/e2e-ait-5d82e915-openshift-default-5f9ffd98b7-95tbg"
Oct 31 17:56:01.381043 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:01.380961    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:56:01.406695 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:01.406669    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-5d82e915-openshift-default-5f9ffd98b7-95tbg"
Oct 31 17:56:01.407346 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:01.407325    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-f5575fb46-29pdk"
Oct 31 17:56:01.737205 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:01.737174    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-5d82e915-openshift-default-5f9ffd98b7-95tbg"]
Oct 31 17:56:01.738319 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:56:01.738298    4276 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcde72f26_a14d_4b50_8ab9_25d401a34d45.slice/crio-76cdeb4dd2b2c1ec33787d0cc0742c0428fac8b4ba75ad5cf2a49e4279a9daf6 WatchSource:0}: Error finding container 76cdeb4dd2b2c1ec33787d0cc0742c0428fac8b4ba75ad5cf2a49e4279a9daf6: Status 404 returned error can't find the container with id 76cdeb4dd2b2c1ec33787d0cc0742c0428fac8b4ba75ad5cf2a49e4279a9daf6
Oct 31 17:56:01.740588 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:01.740554    4276 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 17:56:01.740692 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:01.740619    4276 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 17:56:01.740692 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:01.740647    4276 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 17:56:02.429630 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:02.429599    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-5d82e915-openshift-default-5f9ffd98b7-95tbg" event={"ID":"cde72f26-a14d-4b50-8ab9-25d401a34d45","Type":"ContainerStarted","Data":"08fd4c513a92c32c47f2afcbf92be494c17ce3259eda1c0c4040c601e27d23b4"}
Oct 31 17:56:02.429804 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:02.429654    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-5d82e915-openshift-default-5f9ffd98b7-95tbg" event={"ID":"cde72f26-a14d-4b50-8ab9-25d401a34d45","Type":"ContainerStarted","Data":"76cdeb4dd2b2c1ec33787d0cc0742c0428fac8b4ba75ad5cf2a49e4279a9daf6"}
Oct 31 17:56:02.500633 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:02.500576    4276 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-5d82e915-openshift-default-5f9ffd98b7-95tbg" podStartSLOduration=1.500558404 podStartE2EDuration="1.500558404s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 17:56:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 17:56:02.486191113 +0000 UTC m=+1725.283936037" watchObservedRunningTime="2026-10-31 17:56:02.500558404 +0000 UTC m=+1725.298303332"
Oct 31 17:56:03.407253 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:03.407221    4276 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-5d82e915-openshift-default-5f9ffd98b7-95tbg"
Oct 31 17:56:03.412351 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:03.412327    4276 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-5d82e915-openshift-default-5f9ffd98b7-95tbg"
Oct 31 17:56:03.434122 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:03.434094    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-5d82e915-openshift-default-5f9ffd98b7-95tbg"
Oct 31 17:56:03.434836 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:03.434820    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-5d82e915-openshift-default-5f9ffd98b7-95tbg"
Oct 31 17:56:04.128673 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:04.128645    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:56:04.258060 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:04.258030    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:56:05.144698 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:05.144670    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:56:05.708193 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:05.708160    4276 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-5d82e915-b6799bf66-j4v79"]
Oct 31 17:56:05.711747 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:05.711728    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5d82e915-b6799bf66-j4v79"
Oct 31 17:56:05.740546 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:05.740520    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-5d82e915-serving-cert\""
Oct 31 17:56:05.748544 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:05.748516    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-5d82e915-b6799bf66-j4v79"]
Oct 31 17:56:05.749031 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:05.749005    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b7a7a721-9f5b-4f43-a918-429c0ec13405-maas-api-tls\") pod \"maas-api-e2e-ait-5d82e915-b6799bf66-j4v79\" (UID: \"b7a7a721-9f5b-4f43-a918-429c0ec13405\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5d82e915-b6799bf66-j4v79"
Oct 31 17:56:05.749267 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:05.749237    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b7a7a721-9f5b-4f43-a918-429c0ec13405-metrics-certs\") pod \"maas-api-e2e-ait-5d82e915-b6799bf66-j4v79\" (UID: \"b7a7a721-9f5b-4f43-a918-429c0ec13405\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5d82e915-b6799bf66-j4v79"
Oct 31 17:56:05.749372 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:05.749336    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6gxlt\" (UniqueName: \"kubernetes.io/projected/b7a7a721-9f5b-4f43-a918-429c0ec13405-kube-api-access-6gxlt\") pod \"maas-api-e2e-ait-5d82e915-b6799bf66-j4v79\" (UID: \"b7a7a721-9f5b-4f43-a918-429c0ec13405\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5d82e915-b6799bf66-j4v79"
Oct 31 17:56:05.749471 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:05.749407    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b7a7a721-9f5b-4f43-a918-429c0ec13405-openshift-service-ca\") pod \"maas-api-e2e-ait-5d82e915-b6799bf66-j4v79\" (UID: \"b7a7a721-9f5b-4f43-a918-429c0ec13405\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5d82e915-b6799bf66-j4v79"
Oct 31 17:56:05.823433 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:05.823402    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:56:05.850768 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:05.850743    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b7a7a721-9f5b-4f43-a918-429c0ec13405-openshift-service-ca\") pod \"maas-api-e2e-ait-5d82e915-b6799bf66-j4v79\" (UID: \"b7a7a721-9f5b-4f43-a918-429c0ec13405\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5d82e915-b6799bf66-j4v79"
Oct 31 17:56:05.850908 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:05.850783    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b7a7a721-9f5b-4f43-a918-429c0ec13405-maas-api-tls\") pod \"maas-api-e2e-ait-5d82e915-b6799bf66-j4v79\" (UID: \"b7a7a721-9f5b-4f43-a918-429c0ec13405\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5d82e915-b6799bf66-j4v79"
Oct 31 17:56:05.850908 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:05.850823    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b7a7a721-9f5b-4f43-a918-429c0ec13405-metrics-certs\") pod \"maas-api-e2e-ait-5d82e915-b6799bf66-j4v79\" (UID: \"b7a7a721-9f5b-4f43-a918-429c0ec13405\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5d82e915-b6799bf66-j4v79"
Oct 31 17:56:05.850908 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:05.850865    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6gxlt\" (UniqueName: \"kubernetes.io/projected/b7a7a721-9f5b-4f43-a918-429c0ec13405-kube-api-access-6gxlt\") pod \"maas-api-e2e-ait-5d82e915-b6799bf66-j4v79\" (UID: \"b7a7a721-9f5b-4f43-a918-429c0ec13405\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5d82e915-b6799bf66-j4v79"
Oct 31 17:56:05.851520 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:05.851494    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b7a7a721-9f5b-4f43-a918-429c0ec13405-openshift-service-ca\") pod \"maas-api-e2e-ait-5d82e915-b6799bf66-j4v79\" (UID: \"b7a7a721-9f5b-4f43-a918-429c0ec13405\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5d82e915-b6799bf66-j4v79"
Oct 31 17:56:05.853156 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:05.853134    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b7a7a721-9f5b-4f43-a918-429c0ec13405-metrics-certs\") pod \"maas-api-e2e-ait-5d82e915-b6799bf66-j4v79\" (UID: \"b7a7a721-9f5b-4f43-a918-429c0ec13405\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5d82e915-b6799bf66-j4v79"
Oct 31 17:56:05.853316 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:05.853297    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b7a7a721-9f5b-4f43-a918-429c0ec13405-maas-api-tls\") pod \"maas-api-e2e-ait-5d82e915-b6799bf66-j4v79\" (UID: \"b7a7a721-9f5b-4f43-a918-429c0ec13405\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5d82e915-b6799bf66-j4v79"
Oct 31 17:56:05.871076 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:05.871049    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6gxlt\" (UniqueName: \"kubernetes.io/projected/b7a7a721-9f5b-4f43-a918-429c0ec13405-kube-api-access-6gxlt\") pod \"maas-api-e2e-ait-5d82e915-b6799bf66-j4v79\" (UID: \"b7a7a721-9f5b-4f43-a918-429c0ec13405\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5d82e915-b6799bf66-j4v79"
Oct 31 17:56:05.936565 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:05.936534    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:56:06.025461 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:06.025366    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5d82e915-b6799bf66-j4v79"
Oct 31 17:56:06.327689 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:06.327663    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-5d82e915-b6799bf66-j4v79"]
Oct 31 17:56:06.329433 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:56:06.329406    4276 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7a7a721_9f5b_4f43_a918_429c0ec13405.slice/crio-a2ce675f0a7c214aec521047ab4a36df7ad631faa10efe505bd444fbd271f8db WatchSource:0}: Error finding container a2ce675f0a7c214aec521047ab4a36df7ad631faa10efe505bd444fbd271f8db: Status 404 returned error can't find the container with id a2ce675f0a7c214aec521047ab4a36df7ad631faa10efe505bd444fbd271f8db
Oct 31 17:56:06.448938 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:06.448908    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5d82e915-b6799bf66-j4v79" event={"ID":"b7a7a721-9f5b-4f43-a918-429c0ec13405","Type":"ContainerStarted","Data":"a2ce675f0a7c214aec521047ab4a36df7ad631faa10efe505bd444fbd271f8db"}
Oct 31 17:56:06.596302 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:06.596236    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:56:06.682772 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:06.682737    4276 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-5d82e915-5765b4b7d5-v7zfp"]
Oct 31 17:56:06.687420 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:06.687396    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-5d82e915-5765b4b7d5-v7zfp"
Oct 31 17:56:06.703407 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:06.703387    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-5d82e915-5765b4b7d5-v7zfp"]
Oct 31 17:56:06.704568 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:06.704530    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-5d82e915-dockercfg-wsq58\""
Oct 31 17:56:06.707588 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:06.707565    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:56:06.742117 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:06.742092    4276 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-5d82e915-5fbb795f79-mvc4n"]
Oct 31 17:56:06.746473 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:06.746438    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-5d82e915-5fbb795f79-mvc4n"
Oct 31 17:56:06.757901 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:06.757884    4276 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-5d82e915\""
Oct 31 17:56:06.758159 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:06.758139    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9c603156-470b-4e0f-9491-268aecdeb027-plugins-config-volume\") pod \"payload-processing-e2e-ait-5d82e915-5765b4b7d5-v7zfp\" (UID: \"9c603156-470b-4e0f-9491-268aecdeb027\") " pod="openshift-ingress/payload-processing-e2e-ait-5d82e915-5765b4b7d5-v7zfp"
Oct 31 17:56:06.758207 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:06.758181    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nm26c\" (UniqueName: \"kubernetes.io/projected/9c603156-470b-4e0f-9491-268aecdeb027-kube-api-access-nm26c\") pod \"payload-processing-e2e-ait-5d82e915-5765b4b7d5-v7zfp\" (UID: \"9c603156-470b-4e0f-9491-268aecdeb027\") " pod="openshift-ingress/payload-processing-e2e-ait-5d82e915-5765b4b7d5-v7zfp"
Oct 31 17:56:06.766119 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:06.764588    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-5d82e915-5fbb795f79-mvc4n"]
Oct 31 17:56:06.859726 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:06.859645    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9c603156-470b-4e0f-9491-268aecdeb027-plugins-config-volume\") pod \"payload-processing-e2e-ait-5d82e915-5765b4b7d5-v7zfp\" (UID: \"9c603156-470b-4e0f-9491-268aecdeb027\") " pod="openshift-ingress/payload-processing-e2e-ait-5d82e915-5765b4b7d5-v7zfp"
Oct 31 17:56:06.859726 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:06.859711    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-knsqr\" (UniqueName: \"kubernetes.io/projected/e22b6e51-16e7-41af-9ed5-092f55ceb005-kube-api-access-knsqr\") pod \"payload-pre-processing-e2e-ait-5d82e915-5fbb795f79-mvc4n\" (UID: \"e22b6e51-16e7-41af-9ed5-092f55ceb005\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-5d82e915-5fbb795f79-mvc4n"
Oct 31 17:56:06.859951 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:06.859749    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nm26c\" (UniqueName: \"kubernetes.io/projected/9c603156-470b-4e0f-9491-268aecdeb027-kube-api-access-nm26c\") pod \"payload-processing-e2e-ait-5d82e915-5765b4b7d5-v7zfp\" (UID: \"9c603156-470b-4e0f-9491-268aecdeb027\") " pod="openshift-ingress/payload-processing-e2e-ait-5d82e915-5765b4b7d5-v7zfp"
Oct 31 17:56:06.859951 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:06.859896    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e22b6e51-16e7-41af-9ed5-092f55ceb005-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-5d82e915-5fbb795f79-mvc4n\" (UID: \"e22b6e51-16e7-41af-9ed5-092f55ceb005\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-5d82e915-5fbb795f79-mvc4n"
Oct 31 17:56:06.860339 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:06.860317    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9c603156-470b-4e0f-9491-268aecdeb027-plugins-config-volume\") pod \"payload-processing-e2e-ait-5d82e915-5765b4b7d5-v7zfp\" (UID: \"9c603156-470b-4e0f-9491-268aecdeb027\") " pod="openshift-ingress/payload-processing-e2e-ait-5d82e915-5765b4b7d5-v7zfp"
Oct 31 17:56:06.880205 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:06.880183    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nm26c\" (UniqueName: \"kubernetes.io/projected/9c603156-470b-4e0f-9491-268aecdeb027-kube-api-access-nm26c\") pod \"payload-processing-e2e-ait-5d82e915-5765b4b7d5-v7zfp\" (UID: \"9c603156-470b-4e0f-9491-268aecdeb027\") " pod="openshift-ingress/payload-processing-e2e-ait-5d82e915-5765b4b7d5-v7zfp"
Oct 31 17:56:06.962321 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:06.962275    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-knsqr\" (UniqueName: \"kubernetes.io/projected/e22b6e51-16e7-41af-9ed5-092f55ceb005-kube-api-access-knsqr\") pod \"payload-pre-processing-e2e-ait-5d82e915-5fbb795f79-mvc4n\" (UID: \"e22b6e51-16e7-41af-9ed5-092f55ceb005\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-5d82e915-5fbb795f79-mvc4n"
Oct 31 17:56:06.962584 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:06.962396    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e22b6e51-16e7-41af-9ed5-092f55ceb005-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-5d82e915-5fbb795f79-mvc4n\" (UID: \"e22b6e51-16e7-41af-9ed5-092f55ceb005\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-5d82e915-5fbb795f79-mvc4n"
Oct 31 17:56:06.963420 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:06.963387    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e22b6e51-16e7-41af-9ed5-092f55ceb005-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-5d82e915-5fbb795f79-mvc4n\" (UID: \"e22b6e51-16e7-41af-9ed5-092f55ceb005\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-5d82e915-5fbb795f79-mvc4n"
Oct 31 17:56:06.983268 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:06.983221    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-knsqr\" (UniqueName: \"kubernetes.io/projected/e22b6e51-16e7-41af-9ed5-092f55ceb005-kube-api-access-knsqr\") pod \"payload-pre-processing-e2e-ait-5d82e915-5fbb795f79-mvc4n\" (UID: \"e22b6e51-16e7-41af-9ed5-092f55ceb005\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-5d82e915-5fbb795f79-mvc4n"
Oct 31 17:56:07.000953 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:07.000923    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-5d82e915-5765b4b7d5-v7zfp"
Oct 31 17:56:07.076031 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:07.076003    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-5d82e915-5fbb795f79-mvc4n"
Oct 31 17:56:07.305819 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:07.305797    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-5d82e915-5765b4b7d5-v7zfp"]
Oct 31 17:56:07.307487 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:56:07.307458    4276 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c603156_470b_4e0f_9491_268aecdeb027.slice/crio-f7aa60c75fe2e02671e1da021dd72c1f97a3c18376ec17195bc544383460bff5 WatchSource:0}: Error finding container f7aa60c75fe2e02671e1da021dd72c1f97a3c18376ec17195bc544383460bff5: Status 404 returned error can't find the container with id f7aa60c75fe2e02671e1da021dd72c1f97a3c18376ec17195bc544383460bff5
Oct 31 17:56:07.385357 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:07.385299    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-5d82e915-5fbb795f79-mvc4n"]
Oct 31 17:56:07.388134 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:56:07.388112    4276 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode22b6e51_16e7_41af_9ed5_092f55ceb005.slice/crio-9d75efbbe5c59ce08b547d66edb950dd898406db094633d3ebb67bc8295cfb65 WatchSource:0}: Error finding container 9d75efbbe5c59ce08b547d66edb950dd898406db094633d3ebb67bc8295cfb65: Status 404 returned error can't find the container with id 9d75efbbe5c59ce08b547d66edb950dd898406db094633d3ebb67bc8295cfb65
Oct 31 17:56:07.456016 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:07.455976    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-5d82e915-5765b4b7d5-v7zfp" event={"ID":"9c603156-470b-4e0f-9491-268aecdeb027","Type":"ContainerStarted","Data":"7fba31e141e1a181a9f10109f44dd2cea92f2a886127c1b6d418f70f1751c7b3"}
Oct 31 17:56:07.456180 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:07.456039    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-5d82e915-5765b4b7d5-v7zfp" event={"ID":"9c603156-470b-4e0f-9491-268aecdeb027","Type":"ContainerStarted","Data":"f7aa60c75fe2e02671e1da021dd72c1f97a3c18376ec17195bc544383460bff5"}
Oct 31 17:56:07.456277 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:07.456249    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-5d82e915-5765b4b7d5-v7zfp"
Oct 31 17:56:07.457791 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:07.457768    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5d82e915-b6799bf66-j4v79" event={"ID":"b7a7a721-9f5b-4f43-a918-429c0ec13405","Type":"ContainerStarted","Data":"ab7d629461c9e014680ef9e2164a9dfbc1d1af1713ad7bfdee7294032c9c8a91"}
Oct 31 17:56:07.458095 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:07.457920    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5d82e915-b6799bf66-j4v79"
Oct 31 17:56:07.459569 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:07.459547    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-5d82e915-5fbb795f79-mvc4n" event={"ID":"e22b6e51-16e7-41af-9ed5-092f55ceb005","Type":"ContainerStarted","Data":"0a697f77378054e099f02805f6fca6040d79369bbe3b82b84bf6c860680cc7f9"}
Oct 31 17:56:07.459670 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:07.459591    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-5d82e915-5fbb795f79-mvc4n" event={"ID":"e22b6e51-16e7-41af-9ed5-092f55ceb005","Type":"ContainerStarted","Data":"9d75efbbe5c59ce08b547d66edb950dd898406db094633d3ebb67bc8295cfb65"}
Oct 31 17:56:07.459670 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:07.459639    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-5d82e915-5fbb795f79-mvc4n"
Oct 31 17:56:07.497344 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:07.497304    4276 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-5d82e915-5765b4b7d5-v7zfp" podStartSLOduration=1.497291899 podStartE2EDuration="1.497291899s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 17:56:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 17:56:07.496280897 +0000 UTC m=+1730.294025824" watchObservedRunningTime="2026-10-31 17:56:07.497291899 +0000 UTC m=+1730.295036827"
Oct 31 17:56:07.537225 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:07.537181    4276 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-5d82e915-5fbb795f79-mvc4n" podStartSLOduration=1.5371695920000001 podStartE2EDuration="1.537169592s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 17:56:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 17:56:07.535778499 +0000 UTC m=+1730.333523440" watchObservedRunningTime="2026-10-31 17:56:07.537169592 +0000 UTC m=+1730.334914520"
Oct 31 17:56:07.577642 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:07.577604    4276 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5d82e915-b6799bf66-j4v79" podStartSLOduration=2.174437958 podStartE2EDuration="2.577592131s" totalImagesPullingTime="403.154173ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 17:56:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 17:56:07.57631935 +0000 UTC m=+1730.374064280" watchObservedRunningTime="2026-10-31 17:56:07.577592131 +0000 UTC m=+1730.375337052"
Oct 31 17:56:07.763760 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:07.763680    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:56:08.710166 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:08.710134    4276 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-7bb55cd9d-x784h"]
Oct 31 17:56:08.710593 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:08.710370    4276 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-7bb55cd9d-x784h" podUID="87635e23-0e63-46d0-b92f-5f37b5720cc9" containerName="maas-api" containerID="cri-o://9a3a2a77b4db4c8d11a71a8c5e99571116ee76858fe8ac41610325495a281d8d" gracePeriod=30
Oct 31 17:56:09.419925 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:09.419890    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:56:09.989816 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:09.989789    4276 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7bb55cd9d-x784h"
Oct 31 17:56:10.027865 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:10.027836    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:56:10.096399 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:10.096317    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/87635e23-0e63-46d0-b92f-5f37b5720cc9-kube-api-access-cn2cp\" (UniqueName: \"kubernetes.io/projected/87635e23-0e63-46d0-b92f-5f37b5720cc9-kube-api-access-cn2cp\") pod \"87635e23-0e63-46d0-b92f-5f37b5720cc9\" (UID: \"87635e23-0e63-46d0-b92f-5f37b5720cc9\") "
Oct 31 17:56:10.096399 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:10.096390    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/87635e23-0e63-46d0-b92f-5f37b5720cc9-metrics-certs\" (UniqueName: \"kubernetes.io/secret/87635e23-0e63-46d0-b92f-5f37b5720cc9-metrics-certs\") pod \"87635e23-0e63-46d0-b92f-5f37b5720cc9\" (UID: \"87635e23-0e63-46d0-b92f-5f37b5720cc9\") "
Oct 31 17:56:10.096637 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:10.096471    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/87635e23-0e63-46d0-b92f-5f37b5720cc9-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/87635e23-0e63-46d0-b92f-5f37b5720cc9-openshift-service-ca\") pod \"87635e23-0e63-46d0-b92f-5f37b5720cc9\" (UID: \"87635e23-0e63-46d0-b92f-5f37b5720cc9\") "
Oct 31 17:56:10.096637 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:10.096550    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/87635e23-0e63-46d0-b92f-5f37b5720cc9-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/87635e23-0e63-46d0-b92f-5f37b5720cc9-maas-api-tls\") pod \"87635e23-0e63-46d0-b92f-5f37b5720cc9\" (UID: \"87635e23-0e63-46d0-b92f-5f37b5720cc9\") "
Oct 31 17:56:10.096893 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:10.096867    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/87635e23-0e63-46d0-b92f-5f37b5720cc9-openshift-service-ca" pod "87635e23-0e63-46d0-b92f-5f37b5720cc9" (UID: "87635e23-0e63-46d0-b92f-5f37b5720cc9"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 17:56:10.098486 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:10.098429    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/87635e23-0e63-46d0-b92f-5f37b5720cc9-metrics-certs" pod "87635e23-0e63-46d0-b92f-5f37b5720cc9" (UID: "87635e23-0e63-46d0-b92f-5f37b5720cc9"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 17:56:10.099041 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:10.099011    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/87635e23-0e63-46d0-b92f-5f37b5720cc9-maas-api-tls" pod "87635e23-0e63-46d0-b92f-5f37b5720cc9" (UID: "87635e23-0e63-46d0-b92f-5f37b5720cc9"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 17:56:10.099041 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:10.099019    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/87635e23-0e63-46d0-b92f-5f37b5720cc9-kube-api-access-cn2cp" pod "87635e23-0e63-46d0-b92f-5f37b5720cc9" (UID: "87635e23-0e63-46d0-b92f-5f37b5720cc9"). InnerVolumeSpecName "kube-api-access-cn2cp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 17:56:10.198364 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:10.198332    4276 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cn2cp\" (UniqueName: \"kubernetes.io/projected/87635e23-0e63-46d0-b92f-5f37b5720cc9-kube-api-access-cn2cp\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:56:10.198364 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:10.198359    4276 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/87635e23-0e63-46d0-b92f-5f37b5720cc9-metrics-certs\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:56:10.198364 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:10.198370    4276 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/87635e23-0e63-46d0-b92f-5f37b5720cc9-openshift-service-ca\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:56:10.198633 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:10.198380    4276 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/87635e23-0e63-46d0-b92f-5f37b5720cc9-maas-api-tls\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:56:10.485877 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:10.485788    4276 generic.go:356] "Generic (PLEG): container finished" podID="87635e23-0e63-46d0-b92f-5f37b5720cc9" containerID="9a3a2a77b4db4c8d11a71a8c5e99571116ee76858fe8ac41610325495a281d8d" exitCode=0
Oct 31 17:56:10.485877 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:10.485833    4276 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7bb55cd9d-x784h"
Oct 31 17:56:10.485877 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:10.485865    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7bb55cd9d-x784h" event={"ID":"87635e23-0e63-46d0-b92f-5f37b5720cc9","Type":"ContainerDied","Data":"9a3a2a77b4db4c8d11a71a8c5e99571116ee76858fe8ac41610325495a281d8d"}
Oct 31 17:56:10.486164 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:10.485922    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7bb55cd9d-x784h" event={"ID":"87635e23-0e63-46d0-b92f-5f37b5720cc9","Type":"ContainerDied","Data":"285998e90b39c10a09d9a1a56bfc34a78219b293f44ad0f8637d74ffc9164c6a"}
Oct 31 17:56:10.486164 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:10.485945    4276 scope.go:122] "RemoveContainer" containerID="9a3a2a77b4db4c8d11a71a8c5e99571116ee76858fe8ac41610325495a281d8d"
Oct 31 17:56:10.495232 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:10.495213    4276 scope.go:122] "RemoveContainer" containerID="9a3a2a77b4db4c8d11a71a8c5e99571116ee76858fe8ac41610325495a281d8d"
Oct 31 17:56:10.495528 ip-10-0-129-127 kubenswrapper[4276]: E1031 17:56:10.495506    4276 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9a3a2a77b4db4c8d11a71a8c5e99571116ee76858fe8ac41610325495a281d8d\": container with ID starting with 9a3a2a77b4db4c8d11a71a8c5e99571116ee76858fe8ac41610325495a281d8d not found: ID does not exist" containerID="9a3a2a77b4db4c8d11a71a8c5e99571116ee76858fe8ac41610325495a281d8d"
Oct 31 17:56:10.495620 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:10.495534    4276 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9a3a2a77b4db4c8d11a71a8c5e99571116ee76858fe8ac41610325495a281d8d"} err="failed to get container status \"9a3a2a77b4db4c8d11a71a8c5e99571116ee76858fe8ac41610325495a281d8d\": rpc error: code = NotFound desc = could not find container \"9a3a2a77b4db4c8d11a71a8c5e99571116ee76858fe8ac41610325495a281d8d\": container with ID starting with 9a3a2a77b4db4c8d11a71a8c5e99571116ee76858fe8ac41610325495a281d8d not found: ID does not exist"
Oct 31 17:56:10.540258 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:10.540234    4276 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-7bb55cd9d-x784h"]
Oct 31 17:56:10.580468 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:10.579509    4276 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-7bb55cd9d-x784h"]
Oct 31 17:56:11.253484 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:11.251042    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:56:11.392554 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:11.392521    4276 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="87635e23-0e63-46d0-b92f-5f37b5720cc9" path="/var/lib/kubelet/pods/87635e23-0e63-46d0-b92f-5f37b5720cc9/volumes"
Oct 31 17:56:12.234419 ip-10-0-129-127 kubenswrapper[4276]: E1031 17:56:12.234381    4276 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tg5qk" containerName="extractor"
Oct 31 17:56:13.048264 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:13.048231    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:56:13.479624 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:13.479543    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5d82e915-b6799bf66-j4v79"
Oct 31 17:56:14.871475 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:14.868900    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:56:15.105148 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:15.105115    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:56:16.728272 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:16.728201    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:56:18.416801 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:18.416773    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:56:18.436233 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:18.436204    4276 patch_prober.go:29] interesting pod/e2e-ait-5d82e915-openshift-default-5f9ffd98b7-95tbg container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.94:15021/healthz/ready\": dial tcp 10.133.0.94:15021: connect: connection refused" start-of-body=
Oct 31 17:56:18.436358 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:18.436258    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-5d82e915-openshift-default-5f9ffd98b7-95tbg" podUID="cde72f26-a14d-4b50-8ab9-25d401a34d45" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.94:15021/healthz/ready\": dial tcp 10.133.0.94:15021: connect: connection refused"
Oct 31 17:56:18.468281 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:18.468209    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-5d82e915-5765b4b7d5-v7zfp"
Oct 31 17:56:18.468281 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:18.468281    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-5d82e915-5fbb795f79-mvc4n"
Oct 31 17:56:18.528320 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:18.528224    4276 generic.go:356] "Generic (PLEG): container finished" podID="cde72f26-a14d-4b50-8ab9-25d401a34d45" containerID="08fd4c513a92c32c47f2afcbf92be494c17ce3259eda1c0c4040c601e27d23b4" exitCode=0
Oct 31 17:56:18.528320 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:18.528297    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-5d82e915-openshift-default-5f9ffd98b7-95tbg" event={"ID":"cde72f26-a14d-4b50-8ab9-25d401a34d45","Type":"ContainerDied","Data":"08fd4c513a92c32c47f2afcbf92be494c17ce3259eda1c0c4040c601e27d23b4"}
Oct 31 17:56:18.528805 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:18.528780    4276 scope.go:122] "RemoveContainer" containerID="08fd4c513a92c32c47f2afcbf92be494c17ce3259eda1c0c4040c601e27d23b4"
Oct 31 17:56:18.530206 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:18.530170    4276 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 17:56:18.530313 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:18.530266    4276 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 17:56:18.530385 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:18.530315    4276 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 17:56:19.534946 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:19.534903    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-5d82e915-openshift-default-5f9ffd98b7-95tbg" event={"ID":"cde72f26-a14d-4b50-8ab9-25d401a34d45","Type":"ContainerStarted","Data":"8aa4c6b117028ab09b864d08dd54929ca139e534392af0252377fc776da0f802"}
Oct 31 17:56:20.317081 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:20.317050    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:56:20.407393 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:20.407363    4276 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-5d82e915-openshift-default-5f9ffd98b7-95tbg"
Oct 31 17:56:20.407593 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:20.407574    4276 patch_prober.go:29] interesting pod/e2e-ait-5d82e915-openshift-default-5f9ffd98b7-95tbg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.94:15021/healthz/ready\": dial tcp 10.133.0.94:15021: connect: connection refused" start-of-body=
Oct 31 17:56:20.407651 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:20.407631    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-5d82e915-openshift-default-5f9ffd98b7-95tbg" podUID="cde72f26-a14d-4b50-8ab9-25d401a34d45" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.94:15021/healthz/ready\": dial tcp 10.133.0.94:15021: connect: connection refused"
Oct 31 17:56:21.406944 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:21.406906    4276 patch_prober.go:29] interesting pod/e2e-ait-5d82e915-openshift-default-5f9ffd98b7-95tbg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.94:15021/healthz/ready\": dial tcp 10.133.0.94:15021: connect: connection refused" start-of-body=
Oct 31 17:56:21.407316 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:21.406962    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-5d82e915-openshift-default-5f9ffd98b7-95tbg" podUID="cde72f26-a14d-4b50-8ab9-25d401a34d45" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.94:15021/healthz/ready\": dial tcp 10.133.0.94:15021: connect: connection refused"
Oct 31 17:56:22.278172 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:22.278104    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:56:22.407796 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:22.407765    4276 patch_prober.go:29] interesting pod/e2e-ait-5d82e915-openshift-default-5f9ffd98b7-95tbg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.94:15021/healthz/ready\": dial tcp 10.133.0.94:15021: connect: connection refused" start-of-body=
Oct 31 17:56:22.408186 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:22.407815    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-5d82e915-openshift-default-5f9ffd98b7-95tbg" podUID="cde72f26-a14d-4b50-8ab9-25d401a34d45" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.94:15021/healthz/ready\": dial tcp 10.133.0.94:15021: connect: connection refused"
Oct 31 17:56:23.407461 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:23.407425    4276 patch_prober.go:29] interesting pod/e2e-ait-5d82e915-openshift-default-5f9ffd98b7-95tbg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.94:15021/healthz/ready\": dial tcp 10.133.0.94:15021: connect: connection refused" start-of-body=
Oct 31 17:56:23.407643 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:23.407491    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-5d82e915-openshift-default-5f9ffd98b7-95tbg" podUID="cde72f26-a14d-4b50-8ab9-25d401a34d45" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.94:15021/healthz/ready\": dial tcp 10.133.0.94:15021: connect: connection refused"
Oct 31 17:56:24.203891 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:24.203851    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:56:24.406994 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:24.406964    4276 patch_prober.go:29] interesting pod/e2e-ait-5d82e915-openshift-default-5f9ffd98b7-95tbg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.94:15021/healthz/ready\": dial tcp 10.133.0.94:15021: connect: connection refused" start-of-body=
Oct 31 17:56:24.407162 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:24.407041    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-5d82e915-openshift-default-5f9ffd98b7-95tbg" podUID="cde72f26-a14d-4b50-8ab9-25d401a34d45" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.94:15021/healthz/ready\": dial tcp 10.133.0.94:15021: connect: connection refused"
Oct 31 17:56:25.411987 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:25.411956    4276 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-5d82e915-openshift-default-5f9ffd98b7-95tbg"
Oct 31 17:56:25.412391 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:25.412353    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-5d82e915-openshift-default-5f9ffd98b7-95tbg"
Oct 31 17:56:25.413108 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:25.413091    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-5d82e915-openshift-default-5f9ffd98b7-95tbg"
Oct 31 17:56:25.923932 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:25.923897    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:56:27.722890 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:27.722857    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:56:29.502267 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:29.502233    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:56:30.284560 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:30.284531    4276 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-5d82e915-5765b4b7d5-v7zfp"]
Oct 31 17:56:30.319652 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:30.319618    4276 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-5d82e915-5fbb795f79-mvc4n"]
Oct 31 17:56:30.443028 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:30.442996    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:56:30.551311 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:30.551285    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:56:30.951820 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:30.951739    4276 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-5d82e915-b6799bf66-j4v79"]
Oct 31 17:56:30.952033 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:30.951995    4276 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5d82e915-b6799bf66-j4v79" podUID="b7a7a721-9f5b-4f43-a918-429c0ec13405" containerName="maas-api" containerID="cri-o://ab7d629461c9e014680ef9e2164a9dfbc1d1af1713ad7bfdee7294032c9c8a91" gracePeriod=30
Oct 31 17:56:31.045836 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:31.045796    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:56:31.254327 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:31.253614    4276 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5d82e915-b6799bf66-j4v79"
Oct 31 17:56:31.342200 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:31.342171    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b7a7a721-9f5b-4f43-a918-429c0ec13405-kube-api-access-6gxlt\" (UniqueName: \"kubernetes.io/projected/b7a7a721-9f5b-4f43-a918-429c0ec13405-kube-api-access-6gxlt\") pod \"b7a7a721-9f5b-4f43-a918-429c0ec13405\" (UID: \"b7a7a721-9f5b-4f43-a918-429c0ec13405\") "
Oct 31 17:56:31.342357 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:31.342253    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b7a7a721-9f5b-4f43-a918-429c0ec13405-metrics-certs\" (UniqueName: \"kubernetes.io/secret/b7a7a721-9f5b-4f43-a918-429c0ec13405-metrics-certs\") pod \"b7a7a721-9f5b-4f43-a918-429c0ec13405\" (UID: \"b7a7a721-9f5b-4f43-a918-429c0ec13405\") "
Oct 31 17:56:31.342357 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:31.342307    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b7a7a721-9f5b-4f43-a918-429c0ec13405-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b7a7a721-9f5b-4f43-a918-429c0ec13405-openshift-service-ca\") pod \"b7a7a721-9f5b-4f43-a918-429c0ec13405\" (UID: \"b7a7a721-9f5b-4f43-a918-429c0ec13405\") "
Oct 31 17:56:31.342357 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:31.342334    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b7a7a721-9f5b-4f43-a918-429c0ec13405-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b7a7a721-9f5b-4f43-a918-429c0ec13405-maas-api-tls\") pod \"b7a7a721-9f5b-4f43-a918-429c0ec13405\" (UID: \"b7a7a721-9f5b-4f43-a918-429c0ec13405\") "
Oct 31 17:56:31.342699 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:31.342669    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b7a7a721-9f5b-4f43-a918-429c0ec13405-openshift-service-ca" pod "b7a7a721-9f5b-4f43-a918-429c0ec13405" (UID: "b7a7a721-9f5b-4f43-a918-429c0ec13405"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 17:56:31.342831 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:31.342746    4276 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b7a7a721-9f5b-4f43-a918-429c0ec13405-openshift-service-ca\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:56:31.344378 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:31.344349    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b7a7a721-9f5b-4f43-a918-429c0ec13405-metrics-certs" pod "b7a7a721-9f5b-4f43-a918-429c0ec13405" (UID: "b7a7a721-9f5b-4f43-a918-429c0ec13405"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 17:56:31.344521 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:31.344375    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b7a7a721-9f5b-4f43-a918-429c0ec13405-kube-api-access-6gxlt" pod "b7a7a721-9f5b-4f43-a918-429c0ec13405" (UID: "b7a7a721-9f5b-4f43-a918-429c0ec13405"). InnerVolumeSpecName "kube-api-access-6gxlt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 17:56:31.344521 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:31.344408    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b7a7a721-9f5b-4f43-a918-429c0ec13405-maas-api-tls" pod "b7a7a721-9f5b-4f43-a918-429c0ec13405" (UID: "b7a7a721-9f5b-4f43-a918-429c0ec13405"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 17:56:31.443850 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:31.443815    4276 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6gxlt\" (UniqueName: \"kubernetes.io/projected/b7a7a721-9f5b-4f43-a918-429c0ec13405-kube-api-access-6gxlt\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:56:31.443850 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:31.443846    4276 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b7a7a721-9f5b-4f43-a918-429c0ec13405-metrics-certs\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:56:31.443850 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:31.443854    4276 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b7a7a721-9f5b-4f43-a918-429c0ec13405-maas-api-tls\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:56:31.624475 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:31.624430    4276 generic.go:356] "Generic (PLEG): container finished" podID="b7a7a721-9f5b-4f43-a918-429c0ec13405" containerID="ab7d629461c9e014680ef9e2164a9dfbc1d1af1713ad7bfdee7294032c9c8a91" exitCode=0
Oct 31 17:56:31.625000 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:31.624506    4276 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5d82e915-b6799bf66-j4v79"
Oct 31 17:56:31.625000 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:31.624518    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5d82e915-b6799bf66-j4v79" event={"ID":"b7a7a721-9f5b-4f43-a918-429c0ec13405","Type":"ContainerDied","Data":"ab7d629461c9e014680ef9e2164a9dfbc1d1af1713ad7bfdee7294032c9c8a91"}
Oct 31 17:56:31.625000 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:31.624568    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5d82e915-b6799bf66-j4v79" event={"ID":"b7a7a721-9f5b-4f43-a918-429c0ec13405","Type":"ContainerDied","Data":"a2ce675f0a7c214aec521047ab4a36df7ad631faa10efe505bd444fbd271f8db"}
Oct 31 17:56:31.625000 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:31.624592    4276 scope.go:122] "RemoveContainer" containerID="ab7d629461c9e014680ef9e2164a9dfbc1d1af1713ad7bfdee7294032c9c8a91"
Oct 31 17:56:31.633336 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:31.633319    4276 scope.go:122] "RemoveContainer" containerID="ab7d629461c9e014680ef9e2164a9dfbc1d1af1713ad7bfdee7294032c9c8a91"
Oct 31 17:56:31.633635 ip-10-0-129-127 kubenswrapper[4276]: E1031 17:56:31.633616    4276 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ab7d629461c9e014680ef9e2164a9dfbc1d1af1713ad7bfdee7294032c9c8a91\": container with ID starting with ab7d629461c9e014680ef9e2164a9dfbc1d1af1713ad7bfdee7294032c9c8a91 not found: ID does not exist" containerID="ab7d629461c9e014680ef9e2164a9dfbc1d1af1713ad7bfdee7294032c9c8a91"
Oct 31 17:56:31.633702 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:31.633643    4276 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ab7d629461c9e014680ef9e2164a9dfbc1d1af1713ad7bfdee7294032c9c8a91"} err="failed to get container status \"ab7d629461c9e014680ef9e2164a9dfbc1d1af1713ad7bfdee7294032c9c8a91\": rpc error: code = NotFound desc = could not find container \"ab7d629461c9e014680ef9e2164a9dfbc1d1af1713ad7bfdee7294032c9c8a91\": container with ID starting with ab7d629461c9e014680ef9e2164a9dfbc1d1af1713ad7bfdee7294032c9c8a91 not found: ID does not exist"
Oct 31 17:56:31.678733 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:31.678712    4276 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-5d82e915-b6799bf66-j4v79"]
Oct 31 17:56:31.691677 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:31.691657    4276 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-5d82e915-b6799bf66-j4v79"]
Oct 31 17:56:33.365671 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:33.365642    4276 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b7a7a721-9f5b-4f43-a918-429c0ec13405" path="/var/lib/kubelet/pods/b7a7a721-9f5b-4f43-a918-429c0ec13405/volumes"
Oct 31 17:56:33.510089 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:33.510057    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:56:33.625390 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:33.625303    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:56:35.285581 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:35.285540    4276 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-5d82e915-5765b4b7d5-v7zfp" podUID="9c603156-470b-4e0f-9491-268aecdeb027" containerName="payload-processing" containerID="cri-o://7fba31e141e1a181a9f10109f44dd2cea92f2a886127c1b6d418f70f1751c7b3" gracePeriod=25
Oct 31 17:56:35.320344 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:35.320293    4276 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-5d82e915-5fbb795f79-mvc4n" podUID="e22b6e51-16e7-41af-9ed5-092f55ceb005" containerName="payload-pre-processing" containerID="cri-o://0a697f77378054e099f02805f6fca6040d79369bbe3b82b84bf6c860680cc7f9" gracePeriod=25
Oct 31 17:56:35.571686 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:35.571665    4276 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-5d82e915-5765b4b7d5-v7zfp"
Oct 31 17:56:35.597255 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:35.597229    4276 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-5d82e915-5fbb795f79-mvc4n"
Oct 31 17:56:35.646258 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:35.646230    4276 generic.go:356] "Generic (PLEG): container finished" podID="e22b6e51-16e7-41af-9ed5-092f55ceb005" containerID="0a697f77378054e099f02805f6fca6040d79369bbe3b82b84bf6c860680cc7f9" exitCode=0
Oct 31 17:56:35.646422 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:35.646281    4276 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-5d82e915-5fbb795f79-mvc4n"
Oct 31 17:56:35.646422 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:35.646294    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-5d82e915-5fbb795f79-mvc4n" event={"ID":"e22b6e51-16e7-41af-9ed5-092f55ceb005","Type":"ContainerDied","Data":"0a697f77378054e099f02805f6fca6040d79369bbe3b82b84bf6c860680cc7f9"}
Oct 31 17:56:35.646422 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:35.646344    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-5d82e915-5fbb795f79-mvc4n" event={"ID":"e22b6e51-16e7-41af-9ed5-092f55ceb005","Type":"ContainerDied","Data":"9d75efbbe5c59ce08b547d66edb950dd898406db094633d3ebb67bc8295cfb65"}
Oct 31 17:56:35.646422 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:35.646366    4276 scope.go:122] "RemoveContainer" containerID="0a697f77378054e099f02805f6fca6040d79369bbe3b82b84bf6c860680cc7f9"
Oct 31 17:56:35.647958 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:35.647928    4276 generic.go:356] "Generic (PLEG): container finished" podID="9c603156-470b-4e0f-9491-268aecdeb027" containerID="7fba31e141e1a181a9f10109f44dd2cea92f2a886127c1b6d418f70f1751c7b3" exitCode=0
Oct 31 17:56:35.648075 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:35.647962    4276 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-5d82e915-5765b4b7d5-v7zfp"
Oct 31 17:56:35.648075 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:35.647962    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-5d82e915-5765b4b7d5-v7zfp" event={"ID":"9c603156-470b-4e0f-9491-268aecdeb027","Type":"ContainerDied","Data":"7fba31e141e1a181a9f10109f44dd2cea92f2a886127c1b6d418f70f1751c7b3"}
Oct 31 17:56:35.648075 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:35.648015    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-5d82e915-5765b4b7d5-v7zfp" event={"ID":"9c603156-470b-4e0f-9491-268aecdeb027","Type":"ContainerDied","Data":"f7aa60c75fe2e02671e1da021dd72c1f97a3c18376ec17195bc544383460bff5"}
Oct 31 17:56:35.654909 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:35.654676    4276 scope.go:122] "RemoveContainer" containerID="0a697f77378054e099f02805f6fca6040d79369bbe3b82b84bf6c860680cc7f9"
Oct 31 17:56:35.655217 ip-10-0-129-127 kubenswrapper[4276]: E1031 17:56:35.655193    4276 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0a697f77378054e099f02805f6fca6040d79369bbe3b82b84bf6c860680cc7f9\": container with ID starting with 0a697f77378054e099f02805f6fca6040d79369bbe3b82b84bf6c860680cc7f9 not found: ID does not exist" containerID="0a697f77378054e099f02805f6fca6040d79369bbe3b82b84bf6c860680cc7f9"
Oct 31 17:56:35.655427 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:35.655389    4276 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0a697f77378054e099f02805f6fca6040d79369bbe3b82b84bf6c860680cc7f9"} err="failed to get container status \"0a697f77378054e099f02805f6fca6040d79369bbe3b82b84bf6c860680cc7f9\": rpc error: code = NotFound desc = could not find container \"0a697f77378054e099f02805f6fca6040d79369bbe3b82b84bf6c860680cc7f9\": container with ID starting with 0a697f77378054e099f02805f6fca6040d79369bbe3b82b84bf6c860680cc7f9 not found: ID does not exist"
Oct 31 17:56:35.655875 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:35.655859    4276 scope.go:122] "RemoveContainer" containerID="7fba31e141e1a181a9f10109f44dd2cea92f2a886127c1b6d418f70f1751c7b3"
Oct 31 17:56:35.665800 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:35.665781    4276 scope.go:122] "RemoveContainer" containerID="7fba31e141e1a181a9f10109f44dd2cea92f2a886127c1b6d418f70f1751c7b3"
Oct 31 17:56:35.666071 ip-10-0-129-127 kubenswrapper[4276]: E1031 17:56:35.666052    4276 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7fba31e141e1a181a9f10109f44dd2cea92f2a886127c1b6d418f70f1751c7b3\": container with ID starting with 7fba31e141e1a181a9f10109f44dd2cea92f2a886127c1b6d418f70f1751c7b3 not found: ID does not exist" containerID="7fba31e141e1a181a9f10109f44dd2cea92f2a886127c1b6d418f70f1751c7b3"
Oct 31 17:56:35.666124 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:35.666079    4276 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7fba31e141e1a181a9f10109f44dd2cea92f2a886127c1b6d418f70f1751c7b3"} err="failed to get container status \"7fba31e141e1a181a9f10109f44dd2cea92f2a886127c1b6d418f70f1751c7b3\": rpc error: code = NotFound desc = could not find container \"7fba31e141e1a181a9f10109f44dd2cea92f2a886127c1b6d418f70f1751c7b3\": container with ID starting with 7fba31e141e1a181a9f10109f44dd2cea92f2a886127c1b6d418f70f1751c7b3 not found: ID does not exist"
Oct 31 17:56:35.685632 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:35.685610    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e22b6e51-16e7-41af-9ed5-092f55ceb005-kube-api-access-knsqr\" (UniqueName: \"kubernetes.io/projected/e22b6e51-16e7-41af-9ed5-092f55ceb005-kube-api-access-knsqr\") pod \"e22b6e51-16e7-41af-9ed5-092f55ceb005\" (UID: \"e22b6e51-16e7-41af-9ed5-092f55ceb005\") "
Oct 31 17:56:35.685726 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:35.685712    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9c603156-470b-4e0f-9491-268aecdeb027-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9c603156-470b-4e0f-9491-268aecdeb027-plugins-config-volume\") pod \"9c603156-470b-4e0f-9491-268aecdeb027\" (UID: \"9c603156-470b-4e0f-9491-268aecdeb027\") "
Oct 31 17:56:35.685774 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:35.685735    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e22b6e51-16e7-41af-9ed5-092f55ceb005-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e22b6e51-16e7-41af-9ed5-092f55ceb005-plugins-config-volume\") pod \"e22b6e51-16e7-41af-9ed5-092f55ceb005\" (UID: \"e22b6e51-16e7-41af-9ed5-092f55ceb005\") "
Oct 31 17:56:35.685810 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:35.685801    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9c603156-470b-4e0f-9491-268aecdeb027-kube-api-access-nm26c\" (UniqueName: \"kubernetes.io/projected/9c603156-470b-4e0f-9491-268aecdeb027-kube-api-access-nm26c\") pod \"9c603156-470b-4e0f-9491-268aecdeb027\" (UID: \"9c603156-470b-4e0f-9491-268aecdeb027\") "
Oct 31 17:56:35.686159 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:35.686127    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9c603156-470b-4e0f-9491-268aecdeb027-plugins-config-volume" pod "9c603156-470b-4e0f-9491-268aecdeb027" (UID: "9c603156-470b-4e0f-9491-268aecdeb027"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 17:56:35.686235 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:35.686138    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e22b6e51-16e7-41af-9ed5-092f55ceb005-plugins-config-volume" pod "e22b6e51-16e7-41af-9ed5-092f55ceb005" (UID: "e22b6e51-16e7-41af-9ed5-092f55ceb005"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 17:56:35.687620 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:35.687594    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e22b6e51-16e7-41af-9ed5-092f55ceb005-kube-api-access-knsqr" pod "e22b6e51-16e7-41af-9ed5-092f55ceb005" (UID: "e22b6e51-16e7-41af-9ed5-092f55ceb005"). InnerVolumeSpecName "kube-api-access-knsqr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 17:56:35.687934 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:35.687912    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9c603156-470b-4e0f-9491-268aecdeb027-kube-api-access-nm26c" pod "9c603156-470b-4e0f-9491-268aecdeb027" (UID: "9c603156-470b-4e0f-9491-268aecdeb027"). InnerVolumeSpecName "kube-api-access-nm26c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 17:56:35.787285 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:35.787261    4276 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nm26c\" (UniqueName: \"kubernetes.io/projected/9c603156-470b-4e0f-9491-268aecdeb027-kube-api-access-nm26c\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:56:35.787285 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:35.787284    4276 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-knsqr\" (UniqueName: \"kubernetes.io/projected/e22b6e51-16e7-41af-9ed5-092f55ceb005-kube-api-access-knsqr\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:56:35.787415 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:35.787293    4276 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9c603156-470b-4e0f-9491-268aecdeb027-plugins-config-volume\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:56:35.787415 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:35.787302    4276 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e22b6e51-16e7-41af-9ed5-092f55ceb005-plugins-config-volume\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:56:35.977211 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:35.977179    4276 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-5d82e915-openshift-default-5f9ffd98b7-95tbg"]
Oct 31 17:56:35.977465 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:35.977422    4276 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-5d82e915-openshift-default-5f9ffd98b7-95tbg" podUID="cde72f26-a14d-4b50-8ab9-25d401a34d45" containerName="istio-proxy" containerID="cri-o://8aa4c6b117028ab09b864d08dd54929ca139e534392af0252377fc776da0f802" gracePeriod=30
Oct 31 17:56:36.006365 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:36.006330    4276 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-5d82e915-5fbb795f79-mvc4n"]
Oct 31 17:56:36.011689 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:36.011658    4276 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-5d82e915-5fbb795f79-mvc4n"]
Oct 31 17:56:36.039830 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:36.039807    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:56:36.050073 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:36.050052    4276 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-5d82e915-5765b4b7d5-v7zfp"]
Oct 31 17:56:36.053590 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:36.053565    4276 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-5d82e915-5765b4b7d5-v7zfp"]
Oct 31 17:56:37.211017 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:37.210960    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:56:37.365766 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:37.365735    4276 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9c603156-470b-4e0f-9491-268aecdeb027" path="/var/lib/kubelet/pods/9c603156-470b-4e0f-9491-268aecdeb027/volumes"
Oct 31 17:56:37.366113 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:37.366100    4276 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e22b6e51-16e7-41af-9ed5-092f55ceb005" path="/var/lib/kubelet/pods/e22b6e51-16e7-41af-9ed5-092f55ceb005/volumes"
Oct 31 17:56:37.591462 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:37.591418    4276 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-authcfg-http-139d26-openshift-default-6f6df4f89-9zpdg"]
Oct 31 17:56:37.592020 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:37.591996    4276 cpu_manager.go:414] "containerMap: removing container" podUID="e22b6e51-16e7-41af-9ed5-092f55ceb005" containerName="payload-pre-processing"
Oct 31 17:56:37.592148 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:37.592022    4276 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e22b6e51-16e7-41af-9ed5-092f55ceb005" containerName="payload-pre-processing"
Oct 31 17:56:37.592148 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:37.592045    4276 cpu_manager.go:414] "containerMap: removing container" podUID="b7a7a721-9f5b-4f43-a918-429c0ec13405" containerName="maas-api"
Oct 31 17:56:37.592148 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:37.592053    4276 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b7a7a721-9f5b-4f43-a918-429c0ec13405" containerName="maas-api"
Oct 31 17:56:37.592148 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:37.592085    4276 cpu_manager.go:414] "containerMap: removing container" podUID="9c603156-470b-4e0f-9491-268aecdeb027" containerName="payload-processing"
Oct 31 17:56:37.592148 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:37.592093    4276 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9c603156-470b-4e0f-9491-268aecdeb027" containerName="payload-processing"
Oct 31 17:56:37.592148 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:37.592105    4276 cpu_manager.go:414] "containerMap: removing container" podUID="87635e23-0e63-46d0-b92f-5f37b5720cc9" containerName="maas-api"
Oct 31 17:56:37.592148 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:37.592112    4276 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="87635e23-0e63-46d0-b92f-5f37b5720cc9" containerName="maas-api"
Oct 31 17:56:37.592373 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:37.592218    4276 memory_manager.go:365] "RemoveStaleState removing state" podUID="9c603156-470b-4e0f-9491-268aecdeb027" containerName="payload-processing"
Oct 31 17:56:37.592373 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:37.592229    4276 memory_manager.go:365] "RemoveStaleState removing state" podUID="87635e23-0e63-46d0-b92f-5f37b5720cc9" containerName="maas-api"
Oct 31 17:56:37.592373 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:37.592242    4276 memory_manager.go:365] "RemoveStaleState removing state" podUID="e22b6e51-16e7-41af-9ed5-092f55ceb005" containerName="payload-pre-processing"
Oct 31 17:56:37.592373 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:37.592252    4276 memory_manager.go:365] "RemoveStaleState removing state" podUID="b7a7a721-9f5b-4f43-a918-429c0ec13405" containerName="maas-api"
Oct 31 17:56:37.596024 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:37.596002    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-authcfg-http-139d26-openshift-default-6f6df4f89-9zpdg"
Oct 31 17:56:37.613409 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:37.613378    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-authcfg-http-139d26-openshift-default-6f6df4f89-9zpdg"]
Oct 31 17:56:37.625296 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:37.625273    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-authcfg-http-139d26-openshift-default-dockercfg-njdvj\""
Oct 31 17:56:37.654985 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:37.654959    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:56:37.705551 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:37.705523    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c179233-456f-488d-ba8f-bb9ec7d28fd5-workload-socket\") pod \"e2e-authcfg-http-139d26-openshift-default-6f6df4f89-9zpdg\" (UID: \"4c179233-456f-488d-ba8f-bb9ec7d28fd5\") " pod="openshift-ingress/e2e-authcfg-http-139d26-openshift-default-6f6df4f89-9zpdg"
Oct 31 17:56:37.705700 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:37.705571    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4c179233-456f-488d-ba8f-bb9ec7d28fd5-istio-token\") pod \"e2e-authcfg-http-139d26-openshift-default-6f6df4f89-9zpdg\" (UID: \"4c179233-456f-488d-ba8f-bb9ec7d28fd5\") " pod="openshift-ingress/e2e-authcfg-http-139d26-openshift-default-6f6df4f89-9zpdg"
Oct 31 17:56:37.705700 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:37.705689    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c179233-456f-488d-ba8f-bb9ec7d28fd5-credential-socket\") pod \"e2e-authcfg-http-139d26-openshift-default-6f6df4f89-9zpdg\" (UID: \"4c179233-456f-488d-ba8f-bb9ec7d28fd5\") " pod="openshift-ingress/e2e-authcfg-http-139d26-openshift-default-6f6df4f89-9zpdg"
Oct 31 17:56:37.705778 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:37.705747    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4c179233-456f-488d-ba8f-bb9ec7d28fd5-istio-data\") pod \"e2e-authcfg-http-139d26-openshift-default-6f6df4f89-9zpdg\" (UID: \"4c179233-456f-488d-ba8f-bb9ec7d28fd5\") " pod="openshift-ingress/e2e-authcfg-http-139d26-openshift-default-6f6df4f89-9zpdg"
Oct 31 17:56:37.705778 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:37.705770    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4c179233-456f-488d-ba8f-bb9ec7d28fd5-workload-certs\") pod \"e2e-authcfg-http-139d26-openshift-default-6f6df4f89-9zpdg\" (UID: \"4c179233-456f-488d-ba8f-bb9ec7d28fd5\") " pod="openshift-ingress/e2e-authcfg-http-139d26-openshift-default-6f6df4f89-9zpdg"
Oct 31 17:56:37.705850 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:37.705789    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p87jf\" (UniqueName: \"kubernetes.io/projected/4c179233-456f-488d-ba8f-bb9ec7d28fd5-kube-api-access-p87jf\") pod \"e2e-authcfg-http-139d26-openshift-default-6f6df4f89-9zpdg\" (UID: \"4c179233-456f-488d-ba8f-bb9ec7d28fd5\") " pod="openshift-ingress/e2e-authcfg-http-139d26-openshift-default-6f6df4f89-9zpdg"
Oct 31 17:56:37.705850 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:37.705843    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4c179233-456f-488d-ba8f-bb9ec7d28fd5-istio-envoy\") pod \"e2e-authcfg-http-139d26-openshift-default-6f6df4f89-9zpdg\" (UID: \"4c179233-456f-488d-ba8f-bb9ec7d28fd5\") " pod="openshift-ingress/e2e-authcfg-http-139d26-openshift-default-6f6df4f89-9zpdg"
Oct 31 17:56:37.705917 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:37.705888    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4c179233-456f-488d-ba8f-bb9ec7d28fd5-istiod-ca-cert\") pod \"e2e-authcfg-http-139d26-openshift-default-6f6df4f89-9zpdg\" (UID: \"4c179233-456f-488d-ba8f-bb9ec7d28fd5\") " pod="openshift-ingress/e2e-authcfg-http-139d26-openshift-default-6f6df4f89-9zpdg"
Oct 31 17:56:37.705966 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:37.705953    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4c179233-456f-488d-ba8f-bb9ec7d28fd5-istio-podinfo\") pod \"e2e-authcfg-http-139d26-openshift-default-6f6df4f89-9zpdg\" (UID: \"4c179233-456f-488d-ba8f-bb9ec7d28fd5\") " pod="openshift-ingress/e2e-authcfg-http-139d26-openshift-default-6f6df4f89-9zpdg"
Oct 31 17:56:37.766797 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:37.766759    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:56:37.806989 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:37.806944    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c179233-456f-488d-ba8f-bb9ec7d28fd5-workload-socket\") pod \"e2e-authcfg-http-139d26-openshift-default-6f6df4f89-9zpdg\" (UID: \"4c179233-456f-488d-ba8f-bb9ec7d28fd5\") " pod="openshift-ingress/e2e-authcfg-http-139d26-openshift-default-6f6df4f89-9zpdg"
Oct 31 17:56:37.807243 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:37.807034    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4c179233-456f-488d-ba8f-bb9ec7d28fd5-istio-token\") pod \"e2e-authcfg-http-139d26-openshift-default-6f6df4f89-9zpdg\" (UID: \"4c179233-456f-488d-ba8f-bb9ec7d28fd5\") " pod="openshift-ingress/e2e-authcfg-http-139d26-openshift-default-6f6df4f89-9zpdg"
Oct 31 17:56:37.807243 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:37.807078    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c179233-456f-488d-ba8f-bb9ec7d28fd5-credential-socket\") pod \"e2e-authcfg-http-139d26-openshift-default-6f6df4f89-9zpdg\" (UID: \"4c179233-456f-488d-ba8f-bb9ec7d28fd5\") " pod="openshift-ingress/e2e-authcfg-http-139d26-openshift-default-6f6df4f89-9zpdg"
Oct 31 17:56:37.807243 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:37.807128    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4c179233-456f-488d-ba8f-bb9ec7d28fd5-istio-data\") pod \"e2e-authcfg-http-139d26-openshift-default-6f6df4f89-9zpdg\" (UID: \"4c179233-456f-488d-ba8f-bb9ec7d28fd5\") " pod="openshift-ingress/e2e-authcfg-http-139d26-openshift-default-6f6df4f89-9zpdg"
Oct 31 17:56:37.807243 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:37.807161    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4c179233-456f-488d-ba8f-bb9ec7d28fd5-workload-certs\") pod \"e2e-authcfg-http-139d26-openshift-default-6f6df4f89-9zpdg\" (UID: \"4c179233-456f-488d-ba8f-bb9ec7d28fd5\") " pod="openshift-ingress/e2e-authcfg-http-139d26-openshift-default-6f6df4f89-9zpdg"
Oct 31 17:56:37.807243 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:37.807187    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p87jf\" (UniqueName: \"kubernetes.io/projected/4c179233-456f-488d-ba8f-bb9ec7d28fd5-kube-api-access-p87jf\") pod \"e2e-authcfg-http-139d26-openshift-default-6f6df4f89-9zpdg\" (UID: \"4c179233-456f-488d-ba8f-bb9ec7d28fd5\") " pod="openshift-ingress/e2e-authcfg-http-139d26-openshift-default-6f6df4f89-9zpdg"
Oct 31 17:56:37.807243 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:37.807241    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4c179233-456f-488d-ba8f-bb9ec7d28fd5-istio-envoy\") pod \"e2e-authcfg-http-139d26-openshift-default-6f6df4f89-9zpdg\" (UID: \"4c179233-456f-488d-ba8f-bb9ec7d28fd5\") " pod="openshift-ingress/e2e-authcfg-http-139d26-openshift-default-6f6df4f89-9zpdg"
Oct 31 17:56:37.807641 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:37.807270    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4c179233-456f-488d-ba8f-bb9ec7d28fd5-istiod-ca-cert\") pod \"e2e-authcfg-http-139d26-openshift-default-6f6df4f89-9zpdg\" (UID: \"4c179233-456f-488d-ba8f-bb9ec7d28fd5\") " pod="openshift-ingress/e2e-authcfg-http-139d26-openshift-default-6f6df4f89-9zpdg"
Oct 31 17:56:37.807641 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:37.807317    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4c179233-456f-488d-ba8f-bb9ec7d28fd5-istio-podinfo\") pod \"e2e-authcfg-http-139d26-openshift-default-6f6df4f89-9zpdg\" (UID: \"4c179233-456f-488d-ba8f-bb9ec7d28fd5\") " pod="openshift-ingress/e2e-authcfg-http-139d26-openshift-default-6f6df4f89-9zpdg"
Oct 31 17:56:37.807641 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:37.807403    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c179233-456f-488d-ba8f-bb9ec7d28fd5-workload-socket\") pod \"e2e-authcfg-http-139d26-openshift-default-6f6df4f89-9zpdg\" (UID: \"4c179233-456f-488d-ba8f-bb9ec7d28fd5\") " pod="openshift-ingress/e2e-authcfg-http-139d26-openshift-default-6f6df4f89-9zpdg"
Oct 31 17:56:37.807641 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:37.807485    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c179233-456f-488d-ba8f-bb9ec7d28fd5-credential-socket\") pod \"e2e-authcfg-http-139d26-openshift-default-6f6df4f89-9zpdg\" (UID: \"4c179233-456f-488d-ba8f-bb9ec7d28fd5\") " pod="openshift-ingress/e2e-authcfg-http-139d26-openshift-default-6f6df4f89-9zpdg"
Oct 31 17:56:37.807641 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:37.807553    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4c179233-456f-488d-ba8f-bb9ec7d28fd5-istio-data\") pod \"e2e-authcfg-http-139d26-openshift-default-6f6df4f89-9zpdg\" (UID: \"4c179233-456f-488d-ba8f-bb9ec7d28fd5\") " pod="openshift-ingress/e2e-authcfg-http-139d26-openshift-default-6f6df4f89-9zpdg"
Oct 31 17:56:37.807641 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:37.807613    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4c179233-456f-488d-ba8f-bb9ec7d28fd5-workload-certs\") pod \"e2e-authcfg-http-139d26-openshift-default-6f6df4f89-9zpdg\" (UID: \"4c179233-456f-488d-ba8f-bb9ec7d28fd5\") " pod="openshift-ingress/e2e-authcfg-http-139d26-openshift-default-6f6df4f89-9zpdg"
Oct 31 17:56:37.808005 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:37.807985    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4c179233-456f-488d-ba8f-bb9ec7d28fd5-istiod-ca-cert\") pod \"e2e-authcfg-http-139d26-openshift-default-6f6df4f89-9zpdg\" (UID: \"4c179233-456f-488d-ba8f-bb9ec7d28fd5\") " pod="openshift-ingress/e2e-authcfg-http-139d26-openshift-default-6f6df4f89-9zpdg"
Oct 31 17:56:37.809613 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:37.809594    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4c179233-456f-488d-ba8f-bb9ec7d28fd5-istio-envoy\") pod \"e2e-authcfg-http-139d26-openshift-default-6f6df4f89-9zpdg\" (UID: \"4c179233-456f-488d-ba8f-bb9ec7d28fd5\") " pod="openshift-ingress/e2e-authcfg-http-139d26-openshift-default-6f6df4f89-9zpdg"
Oct 31 17:56:37.809768 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:37.809748    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4c179233-456f-488d-ba8f-bb9ec7d28fd5-istio-podinfo\") pod \"e2e-authcfg-http-139d26-openshift-default-6f6df4f89-9zpdg\" (UID: \"4c179233-456f-488d-ba8f-bb9ec7d28fd5\") " pod="openshift-ingress/e2e-authcfg-http-139d26-openshift-default-6f6df4f89-9zpdg"
Oct 31 17:56:37.826949 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:37.826923    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4c179233-456f-488d-ba8f-bb9ec7d28fd5-istio-token\") pod \"e2e-authcfg-http-139d26-openshift-default-6f6df4f89-9zpdg\" (UID: \"4c179233-456f-488d-ba8f-bb9ec7d28fd5\") " pod="openshift-ingress/e2e-authcfg-http-139d26-openshift-default-6f6df4f89-9zpdg"
Oct 31 17:56:37.827162 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:37.827141    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p87jf\" (UniqueName: \"kubernetes.io/projected/4c179233-456f-488d-ba8f-bb9ec7d28fd5-kube-api-access-p87jf\") pod \"e2e-authcfg-http-139d26-openshift-default-6f6df4f89-9zpdg\" (UID: \"4c179233-456f-488d-ba8f-bb9ec7d28fd5\") " pod="openshift-ingress/e2e-authcfg-http-139d26-openshift-default-6f6df4f89-9zpdg"
Oct 31 17:56:37.911663 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:37.911597    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-authcfg-http-139d26-openshift-default-6f6df4f89-9zpdg"
Oct 31 17:56:38.221821 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:38.221792    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-authcfg-http-139d26-openshift-default-6f6df4f89-9zpdg"]
Oct 31 17:56:38.223250 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:56:38.223228    4276 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c179233_456f_488d_ba8f_bb9ec7d28fd5.slice/crio-d6588385758a0a7e09f4706009018d19da8955ab3ab816acf508b686cbe4ab0f WatchSource:0}: Error finding container d6588385758a0a7e09f4706009018d19da8955ab3ab816acf508b686cbe4ab0f: Status 404 returned error can't find the container with id d6588385758a0a7e09f4706009018d19da8955ab3ab816acf508b686cbe4ab0f
Oct 31 17:56:38.225383 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:38.225355    4276 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 17:56:38.225465 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:38.225417    4276 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 17:56:38.225513 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:38.225472    4276 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 17:56:38.664891 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:38.664852    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-authcfg-http-139d26-openshift-default-6f6df4f89-9zpdg" event={"ID":"4c179233-456f-488d-ba8f-bb9ec7d28fd5","Type":"ContainerStarted","Data":"d08047bdf3cb7eb8999926137f6f061f46f43183b466df2014957af420da2cee"}
Oct 31 17:56:38.665044 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:38.664911    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-authcfg-http-139d26-openshift-default-6f6df4f89-9zpdg" event={"ID":"4c179233-456f-488d-ba8f-bb9ec7d28fd5","Type":"ContainerStarted","Data":"d6588385758a0a7e09f4706009018d19da8955ab3ab816acf508b686cbe4ab0f"}
Oct 31 17:56:38.708988 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:38.708929    4276 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-authcfg-http-139d26-openshift-default-6f6df4f89-9zpdg" podStartSLOduration=1.708916521 podStartE2EDuration="1.708916521s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 17:56:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 17:56:38.707264376 +0000 UTC m=+1761.505009304" watchObservedRunningTime="2026-10-31 17:56:38.708916521 +0000 UTC m=+1761.506661446"
Oct 31 17:56:38.912384 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:38.912354    4276 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-authcfg-http-139d26-openshift-default-6f6df4f89-9zpdg"
Oct 31 17:56:39.918243 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:39.918215    4276 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-authcfg-http-139d26-openshift-default-6f6df4f89-9zpdg"
Oct 31 17:56:40.414665 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:40.414630    4276 patch_prober.go:29] interesting pod/e2e-ait-5d82e915-openshift-default-5f9ffd98b7-95tbg container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 31 17:56:40.414837 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:40.414685    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-5d82e915-openshift-default-5f9ffd98b7-95tbg" podUID="cde72f26-a14d-4b50-8ab9-25d401a34d45" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 17:56:40.673709 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:40.673637    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-authcfg-http-139d26-openshift-default-6f6df4f89-9zpdg"
Oct 31 17:56:40.674678 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:40.674660    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-authcfg-http-139d26-openshift-default-6f6df4f89-9zpdg"
Oct 31 17:56:40.976251 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:40.976180    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:56:41.091533 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:41.091505    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:56:41.227812 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:41.227762    4276 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-5d82e915-openshift-default-5f9ffd98b7-95tbg"
Oct 31 17:56:41.341742 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:41.341708    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cde72f26-a14d-4b50-8ab9-25d401a34d45-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cde72f26-a14d-4b50-8ab9-25d401a34d45-workload-socket\") pod \"cde72f26-a14d-4b50-8ab9-25d401a34d45\" (UID: \"cde72f26-a14d-4b50-8ab9-25d401a34d45\") "
Oct 31 17:56:41.341742 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:41.341746    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cde72f26-a14d-4b50-8ab9-25d401a34d45-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cde72f26-a14d-4b50-8ab9-25d401a34d45-istio-envoy\") pod \"cde72f26-a14d-4b50-8ab9-25d401a34d45\" (UID: \"cde72f26-a14d-4b50-8ab9-25d401a34d45\") "
Oct 31 17:56:41.341971 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:41.341778    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cde72f26-a14d-4b50-8ab9-25d401a34d45-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cde72f26-a14d-4b50-8ab9-25d401a34d45-istio-data\") pod \"cde72f26-a14d-4b50-8ab9-25d401a34d45\" (UID: \"cde72f26-a14d-4b50-8ab9-25d401a34d45\") "
Oct 31 17:56:41.341971 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:41.341839    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/cde72f26-a14d-4b50-8ab9-25d401a34d45-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cde72f26-a14d-4b50-8ab9-25d401a34d45-istio-podinfo\") pod \"cde72f26-a14d-4b50-8ab9-25d401a34d45\" (UID: \"cde72f26-a14d-4b50-8ab9-25d401a34d45\") "
Oct 31 17:56:41.341971 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:41.341898    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cde72f26-a14d-4b50-8ab9-25d401a34d45-kube-api-access-tm25t\" (UniqueName: \"kubernetes.io/projected/cde72f26-a14d-4b50-8ab9-25d401a34d45-kube-api-access-tm25t\") pod \"cde72f26-a14d-4b50-8ab9-25d401a34d45\" (UID: \"cde72f26-a14d-4b50-8ab9-25d401a34d45\") "
Oct 31 17:56:41.341971 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:41.341961    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/cde72f26-a14d-4b50-8ab9-25d401a34d45-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cde72f26-a14d-4b50-8ab9-25d401a34d45-istiod-ca-cert\") pod \"cde72f26-a14d-4b50-8ab9-25d401a34d45\" (UID: \"cde72f26-a14d-4b50-8ab9-25d401a34d45\") "
Oct 31 17:56:41.342193 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:41.342022    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cde72f26-a14d-4b50-8ab9-25d401a34d45-workload-socket" pod "cde72f26-a14d-4b50-8ab9-25d401a34d45" (UID: "cde72f26-a14d-4b50-8ab9-25d401a34d45"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:56:41.342193 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:41.342039    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cde72f26-a14d-4b50-8ab9-25d401a34d45-istio-token\" (UniqueName: \"kubernetes.io/projected/cde72f26-a14d-4b50-8ab9-25d401a34d45-istio-token\") pod \"cde72f26-a14d-4b50-8ab9-25d401a34d45\" (UID: \"cde72f26-a14d-4b50-8ab9-25d401a34d45\") "
Oct 31 17:56:41.342193 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:41.342068    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cde72f26-a14d-4b50-8ab9-25d401a34d45-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cde72f26-a14d-4b50-8ab9-25d401a34d45-credential-socket\") pod \"cde72f26-a14d-4b50-8ab9-25d401a34d45\" (UID: \"cde72f26-a14d-4b50-8ab9-25d401a34d45\") "
Oct 31 17:56:41.342193 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:41.342095    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cde72f26-a14d-4b50-8ab9-25d401a34d45-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cde72f26-a14d-4b50-8ab9-25d401a34d45-workload-certs\") pod \"cde72f26-a14d-4b50-8ab9-25d401a34d45\" (UID: \"cde72f26-a14d-4b50-8ab9-25d401a34d45\") "
Oct 31 17:56:41.342193 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:41.342132    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cde72f26-a14d-4b50-8ab9-25d401a34d45-istio-data" pod "cde72f26-a14d-4b50-8ab9-25d401a34d45" (UID: "cde72f26-a14d-4b50-8ab9-25d401a34d45"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:56:41.342503 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:41.342361    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cde72f26-a14d-4b50-8ab9-25d401a34d45-istiod-ca-cert" pod "cde72f26-a14d-4b50-8ab9-25d401a34d45" (UID: "cde72f26-a14d-4b50-8ab9-25d401a34d45"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 17:56:41.342503 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:41.342385    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cde72f26-a14d-4b50-8ab9-25d401a34d45-credential-socket" pod "cde72f26-a14d-4b50-8ab9-25d401a34d45" (UID: "cde72f26-a14d-4b50-8ab9-25d401a34d45"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:56:41.342622 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:41.342571    4276 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cde72f26-a14d-4b50-8ab9-25d401a34d45-istiod-ca-cert\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:56:41.342622 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:41.342571    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cde72f26-a14d-4b50-8ab9-25d401a34d45-workload-certs" pod "cde72f26-a14d-4b50-8ab9-25d401a34d45" (UID: "cde72f26-a14d-4b50-8ab9-25d401a34d45"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:56:41.342622 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:41.342594    4276 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cde72f26-a14d-4b50-8ab9-25d401a34d45-credential-socket\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:56:41.342622 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:41.342609    4276 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cde72f26-a14d-4b50-8ab9-25d401a34d45-workload-socket\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:56:41.342778 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:41.342623    4276 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cde72f26-a14d-4b50-8ab9-25d401a34d45-istio-data\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:56:41.344421 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:41.344399    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cde72f26-a14d-4b50-8ab9-25d401a34d45-istio-envoy" pod "cde72f26-a14d-4b50-8ab9-25d401a34d45" (UID: "cde72f26-a14d-4b50-8ab9-25d401a34d45"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:56:41.344563 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:41.344426    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cde72f26-a14d-4b50-8ab9-25d401a34d45-kube-api-access-tm25t" pod "cde72f26-a14d-4b50-8ab9-25d401a34d45" (UID: "cde72f26-a14d-4b50-8ab9-25d401a34d45"). InnerVolumeSpecName "kube-api-access-tm25t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 17:56:41.344563 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:41.344517    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cde72f26-a14d-4b50-8ab9-25d401a34d45-istio-token" pod "cde72f26-a14d-4b50-8ab9-25d401a34d45" (UID: "cde72f26-a14d-4b50-8ab9-25d401a34d45"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 17:56:41.344563 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:41.344531    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/cde72f26-a14d-4b50-8ab9-25d401a34d45-istio-podinfo" pod "cde72f26-a14d-4b50-8ab9-25d401a34d45" (UID: "cde72f26-a14d-4b50-8ab9-25d401a34d45"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 17:56:41.444055 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:41.444024    4276 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cde72f26-a14d-4b50-8ab9-25d401a34d45-istio-envoy\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:56:41.444055 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:41.444049    4276 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cde72f26-a14d-4b50-8ab9-25d401a34d45-istio-podinfo\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:56:41.444055 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:41.444058    4276 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tm25t\" (UniqueName: \"kubernetes.io/projected/cde72f26-a14d-4b50-8ab9-25d401a34d45-kube-api-access-tm25t\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:56:41.444263 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:41.444067    4276 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cde72f26-a14d-4b50-8ab9-25d401a34d45-istio-token\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:56:41.444263 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:41.444076    4276 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cde72f26-a14d-4b50-8ab9-25d401a34d45-workload-certs\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:56:41.680110 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:41.680072    4276 generic.go:356] "Generic (PLEG): container finished" podID="cde72f26-a14d-4b50-8ab9-25d401a34d45" containerID="8aa4c6b117028ab09b864d08dd54929ca139e534392af0252377fc776da0f802" exitCode=0
Oct 31 17:56:41.680294 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:41.680122    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-5d82e915-openshift-default-5f9ffd98b7-95tbg" event={"ID":"cde72f26-a14d-4b50-8ab9-25d401a34d45","Type":"ContainerDied","Data":"8aa4c6b117028ab09b864d08dd54929ca139e534392af0252377fc776da0f802"}
Oct 31 17:56:41.680294 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:41.680181    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-5d82e915-openshift-default-5f9ffd98b7-95tbg" event={"ID":"cde72f26-a14d-4b50-8ab9-25d401a34d45","Type":"ContainerDied","Data":"76cdeb4dd2b2c1ec33787d0cc0742c0428fac8b4ba75ad5cf2a49e4279a9daf6"}
Oct 31 17:56:41.680294 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:41.680209    4276 scope.go:122] "RemoveContainer" containerID="8aa4c6b117028ab09b864d08dd54929ca139e534392af0252377fc776da0f802"
Oct 31 17:56:41.680294 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:41.680133    4276 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-5d82e915-openshift-default-5f9ffd98b7-95tbg"
Oct 31 17:56:41.689067 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:41.689046    4276 scope.go:122] "RemoveContainer" containerID="08fd4c513a92c32c47f2afcbf92be494c17ce3259eda1c0c4040c601e27d23b4"
Oct 31 17:56:41.696866 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:41.696850    4276 scope.go:122] "RemoveContainer" containerID="8aa4c6b117028ab09b864d08dd54929ca139e534392af0252377fc776da0f802"
Oct 31 17:56:41.697133 ip-10-0-129-127 kubenswrapper[4276]: E1031 17:56:41.697115    4276 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8aa4c6b117028ab09b864d08dd54929ca139e534392af0252377fc776da0f802\": container with ID starting with 8aa4c6b117028ab09b864d08dd54929ca139e534392af0252377fc776da0f802 not found: ID does not exist" containerID="8aa4c6b117028ab09b864d08dd54929ca139e534392af0252377fc776da0f802"
Oct 31 17:56:41.697199 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:41.697141    4276 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8aa4c6b117028ab09b864d08dd54929ca139e534392af0252377fc776da0f802"} err="failed to get container status \"8aa4c6b117028ab09b864d08dd54929ca139e534392af0252377fc776da0f802\": rpc error: code = NotFound desc = could not find container \"8aa4c6b117028ab09b864d08dd54929ca139e534392af0252377fc776da0f802\": container with ID starting with 8aa4c6b117028ab09b864d08dd54929ca139e534392af0252377fc776da0f802 not found: ID does not exist"
Oct 31 17:56:41.697199 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:41.697158    4276 scope.go:122] "RemoveContainer" containerID="08fd4c513a92c32c47f2afcbf92be494c17ce3259eda1c0c4040c601e27d23b4"
Oct 31 17:56:41.697398 ip-10-0-129-127 kubenswrapper[4276]: E1031 17:56:41.697382    4276 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"08fd4c513a92c32c47f2afcbf92be494c17ce3259eda1c0c4040c601e27d23b4\": container with ID starting with 08fd4c513a92c32c47f2afcbf92be494c17ce3259eda1c0c4040c601e27d23b4 not found: ID does not exist" containerID="08fd4c513a92c32c47f2afcbf92be494c17ce3259eda1c0c4040c601e27d23b4"
Oct 31 17:56:41.697463 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:41.697404    4276 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"08fd4c513a92c32c47f2afcbf92be494c17ce3259eda1c0c4040c601e27d23b4"} err="failed to get container status \"08fd4c513a92c32c47f2afcbf92be494c17ce3259eda1c0c4040c601e27d23b4\": rpc error: code = NotFound desc = could not find container \"08fd4c513a92c32c47f2afcbf92be494c17ce3259eda1c0c4040c601e27d23b4\": container with ID starting with 08fd4c513a92c32c47f2afcbf92be494c17ce3259eda1c0c4040c601e27d23b4 not found: ID does not exist"
Oct 31 17:56:41.734020 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:41.733991    4276 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-5d82e915-openshift-default-5f9ffd98b7-95tbg"]
Oct 31 17:56:41.747974 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:41.747951    4276 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-5d82e915-openshift-default-5f9ffd98b7-95tbg"]
Oct 31 17:56:42.233981 ip-10-0-129-127 kubenswrapper[4276]: E1031 17:56:42.233938    4276 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tg5qk" containerName="extractor"
Oct 31 17:56:42.469094 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:42.469063    4276 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-authcfg-http-139d26-57b4db8749-kkqrt"]
Oct 31 17:56:42.469673 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:42.469603    4276 cpu_manager.go:414] "containerMap: removing container" podUID="cde72f26-a14d-4b50-8ab9-25d401a34d45" containerName="istio-proxy"
Oct 31 17:56:42.469673 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:42.469669    4276 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cde72f26-a14d-4b50-8ab9-25d401a34d45" containerName="istio-proxy"
Oct 31 17:56:42.469898 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:42.469697    4276 cpu_manager.go:414] "containerMap: removing container" podUID="cde72f26-a14d-4b50-8ab9-25d401a34d45" containerName="istio-proxy"
Oct 31 17:56:42.469898 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:42.469705    4276 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cde72f26-a14d-4b50-8ab9-25d401a34d45" containerName="istio-proxy"
Oct 31 17:56:42.469898 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:42.469808    4276 memory_manager.go:365] "RemoveStaleState removing state" podUID="cde72f26-a14d-4b50-8ab9-25d401a34d45" containerName="istio-proxy"
Oct 31 17:56:42.469898 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:42.469825    4276 memory_manager.go:365] "RemoveStaleState removing state" podUID="cde72f26-a14d-4b50-8ab9-25d401a34d45" containerName="istio-proxy"
Oct 31 17:56:42.473971 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:42.473954    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-139d26-57b4db8749-kkqrt"
Oct 31 17:56:42.495216 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:42.495156    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-authcfg-http-139d26-57b4db8749-kkqrt"]
Oct 31 17:56:42.551498 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:42.551438    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8c75f040-baeb-4bba-9497-6b70b3288b83-metrics-certs\") pod \"maas-api-e2e-authcfg-http-139d26-57b4db8749-kkqrt\" (UID: \"8c75f040-baeb-4bba-9497-6b70b3288b83\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-139d26-57b4db8749-kkqrt"
Oct 31 17:56:42.551693 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:42.551578    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8c75f040-baeb-4bba-9497-6b70b3288b83-openshift-service-ca\") pod \"maas-api-e2e-authcfg-http-139d26-57b4db8749-kkqrt\" (UID: \"8c75f040-baeb-4bba-9497-6b70b3288b83\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-139d26-57b4db8749-kkqrt"
Oct 31 17:56:42.551693 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:42.551631    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bfxmh\" (UniqueName: \"kubernetes.io/projected/8c75f040-baeb-4bba-9497-6b70b3288b83-kube-api-access-bfxmh\") pod \"maas-api-e2e-authcfg-http-139d26-57b4db8749-kkqrt\" (UID: \"8c75f040-baeb-4bba-9497-6b70b3288b83\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-139d26-57b4db8749-kkqrt"
Oct 31 17:56:42.551813 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:42.551749    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8c75f040-baeb-4bba-9497-6b70b3288b83-maas-api-tls\") pod \"maas-api-e2e-authcfg-http-139d26-57b4db8749-kkqrt\" (UID: \"8c75f040-baeb-4bba-9497-6b70b3288b83\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-139d26-57b4db8749-kkqrt"
Oct 31 17:56:42.577641 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:42.577610    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-authcfg-http-139d26-serving-cert\""
Oct 31 17:56:42.578779 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:42.578751    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:56:42.652991 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:42.652965    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8c75f040-baeb-4bba-9497-6b70b3288b83-metrics-certs\") pod \"maas-api-e2e-authcfg-http-139d26-57b4db8749-kkqrt\" (UID: \"8c75f040-baeb-4bba-9497-6b70b3288b83\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-139d26-57b4db8749-kkqrt"
Oct 31 17:56:42.653154 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:42.653015    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8c75f040-baeb-4bba-9497-6b70b3288b83-openshift-service-ca\") pod \"maas-api-e2e-authcfg-http-139d26-57b4db8749-kkqrt\" (UID: \"8c75f040-baeb-4bba-9497-6b70b3288b83\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-139d26-57b4db8749-kkqrt"
Oct 31 17:56:42.653154 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:42.653061    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bfxmh\" (UniqueName: \"kubernetes.io/projected/8c75f040-baeb-4bba-9497-6b70b3288b83-kube-api-access-bfxmh\") pod \"maas-api-e2e-authcfg-http-139d26-57b4db8749-kkqrt\" (UID: \"8c75f040-baeb-4bba-9497-6b70b3288b83\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-139d26-57b4db8749-kkqrt"
Oct 31 17:56:42.653154 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:42.653106    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8c75f040-baeb-4bba-9497-6b70b3288b83-maas-api-tls\") pod \"maas-api-e2e-authcfg-http-139d26-57b4db8749-kkqrt\" (UID: \"8c75f040-baeb-4bba-9497-6b70b3288b83\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-139d26-57b4db8749-kkqrt"
Oct 31 17:56:42.653761 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:42.653739    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8c75f040-baeb-4bba-9497-6b70b3288b83-openshift-service-ca\") pod \"maas-api-e2e-authcfg-http-139d26-57b4db8749-kkqrt\" (UID: \"8c75f040-baeb-4bba-9497-6b70b3288b83\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-139d26-57b4db8749-kkqrt"
Oct 31 17:56:42.655335 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:42.655316    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8c75f040-baeb-4bba-9497-6b70b3288b83-metrics-certs\") pod \"maas-api-e2e-authcfg-http-139d26-57b4db8749-kkqrt\" (UID: \"8c75f040-baeb-4bba-9497-6b70b3288b83\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-139d26-57b4db8749-kkqrt"
Oct 31 17:56:42.655439 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:42.655423    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8c75f040-baeb-4bba-9497-6b70b3288b83-maas-api-tls\") pod \"maas-api-e2e-authcfg-http-139d26-57b4db8749-kkqrt\" (UID: \"8c75f040-baeb-4bba-9497-6b70b3288b83\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-139d26-57b4db8749-kkqrt"
Oct 31 17:56:42.675077 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:42.675057    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bfxmh\" (UniqueName: \"kubernetes.io/projected/8c75f040-baeb-4bba-9497-6b70b3288b83-kube-api-access-bfxmh\") pod \"maas-api-e2e-authcfg-http-139d26-57b4db8749-kkqrt\" (UID: \"8c75f040-baeb-4bba-9497-6b70b3288b83\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-139d26-57b4db8749-kkqrt"
Oct 31 17:56:42.780392 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:42.780265    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:56:42.787286 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:42.787243    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-139d26-57b4db8749-kkqrt"
Oct 31 17:56:42.847322 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:42.845322    4276 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-authcfg-http-139d26-574fb59b4c-slv48"]
Oct 31 17:56:42.849607 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:42.849583    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-139d26-574fb59b4c-slv48"
Oct 31 17:56:42.855273 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:42.854968    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/453dd8b4-8c09-4c76-82e8-6891e9609684-plugins-config-volume\") pod \"payload-pre-processing-e2e-authcfg-http-139d26-574fb59b4c-slv48\" (UID: \"453dd8b4-8c09-4c76-82e8-6891e9609684\") " pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-139d26-574fb59b4c-slv48"
Oct 31 17:56:42.855273 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:42.855031    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xclcn\" (UniqueName: \"kubernetes.io/projected/453dd8b4-8c09-4c76-82e8-6891e9609684-kube-api-access-xclcn\") pod \"payload-pre-processing-e2e-authcfg-http-139d26-574fb59b4c-slv48\" (UID: \"453dd8b4-8c09-4c76-82e8-6891e9609684\") " pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-139d26-574fb59b4c-slv48"
Oct 31 17:56:42.870572 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:42.866891    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-authcfg-http-139d26-574fb59b4c-slv48"]
Oct 31 17:56:42.870572 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:42.867118    4276 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-authcfg-http-139d26\""
Oct 31 17:56:42.877882 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:42.877863    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-authcfg-http-139d26-dockercfg-nm4t2\""
Oct 31 17:56:42.900933 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:42.900894    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:56:42.936563 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:42.936429    4276 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-authcfg-http-139d26-865dbd6585-gcmr9"]
Oct 31 17:56:42.940435 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:42.940411    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-authcfg-http-139d26-865dbd6585-gcmr9"
Oct 31 17:56:42.956068 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:42.955995    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f9d2226c-82f2-48cb-b1cc-1ee87b3f323d-plugins-config-volume\") pod \"payload-processing-e2e-authcfg-http-139d26-865dbd6585-gcmr9\" (UID: \"f9d2226c-82f2-48cb-b1cc-1ee87b3f323d\") " pod="openshift-ingress/payload-processing-e2e-authcfg-http-139d26-865dbd6585-gcmr9"
Oct 31 17:56:42.956068 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:42.956057    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/453dd8b4-8c09-4c76-82e8-6891e9609684-plugins-config-volume\") pod \"payload-pre-processing-e2e-authcfg-http-139d26-574fb59b4c-slv48\" (UID: \"453dd8b4-8c09-4c76-82e8-6891e9609684\") " pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-139d26-574fb59b4c-slv48"
Oct 31 17:56:42.956275 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:42.956094    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wg2tr\" (UniqueName: \"kubernetes.io/projected/f9d2226c-82f2-48cb-b1cc-1ee87b3f323d-kube-api-access-wg2tr\") pod \"payload-processing-e2e-authcfg-http-139d26-865dbd6585-gcmr9\" (UID: \"f9d2226c-82f2-48cb-b1cc-1ee87b3f323d\") " pod="openshift-ingress/payload-processing-e2e-authcfg-http-139d26-865dbd6585-gcmr9"
Oct 31 17:56:42.956275 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:42.956156    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xclcn\" (UniqueName: \"kubernetes.io/projected/453dd8b4-8c09-4c76-82e8-6891e9609684-kube-api-access-xclcn\") pod \"payload-pre-processing-e2e-authcfg-http-139d26-574fb59b4c-slv48\" (UID: \"453dd8b4-8c09-4c76-82e8-6891e9609684\") " pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-139d26-574fb59b4c-slv48"
Oct 31 17:56:42.957368 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:42.957342    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/453dd8b4-8c09-4c76-82e8-6891e9609684-plugins-config-volume\") pod \"payload-pre-processing-e2e-authcfg-http-139d26-574fb59b4c-slv48\" (UID: \"453dd8b4-8c09-4c76-82e8-6891e9609684\") " pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-139d26-574fb59b4c-slv48"
Oct 31 17:56:42.958227 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:42.958202    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-authcfg-http-139d26-865dbd6585-gcmr9"]
Oct 31 17:56:42.976972 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:42.976841    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xclcn\" (UniqueName: \"kubernetes.io/projected/453dd8b4-8c09-4c76-82e8-6891e9609684-kube-api-access-xclcn\") pod \"payload-pre-processing-e2e-authcfg-http-139d26-574fb59b4c-slv48\" (UID: \"453dd8b4-8c09-4c76-82e8-6891e9609684\") " pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-139d26-574fb59b4c-slv48"
Oct 31 17:56:43.057701 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:43.057673    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f9d2226c-82f2-48cb-b1cc-1ee87b3f323d-plugins-config-volume\") pod \"payload-processing-e2e-authcfg-http-139d26-865dbd6585-gcmr9\" (UID: \"f9d2226c-82f2-48cb-b1cc-1ee87b3f323d\") " pod="openshift-ingress/payload-processing-e2e-authcfg-http-139d26-865dbd6585-gcmr9"
Oct 31 17:56:43.057862 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:43.057721    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wg2tr\" (UniqueName: \"kubernetes.io/projected/f9d2226c-82f2-48cb-b1cc-1ee87b3f323d-kube-api-access-wg2tr\") pod \"payload-processing-e2e-authcfg-http-139d26-865dbd6585-gcmr9\" (UID: \"f9d2226c-82f2-48cb-b1cc-1ee87b3f323d\") " pod="openshift-ingress/payload-processing-e2e-authcfg-http-139d26-865dbd6585-gcmr9"
Oct 31 17:56:43.058488 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:43.058467    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f9d2226c-82f2-48cb-b1cc-1ee87b3f323d-plugins-config-volume\") pod \"payload-processing-e2e-authcfg-http-139d26-865dbd6585-gcmr9\" (UID: \"f9d2226c-82f2-48cb-b1cc-1ee87b3f323d\") " pod="openshift-ingress/payload-processing-e2e-authcfg-http-139d26-865dbd6585-gcmr9"
Oct 31 17:56:43.077777 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:43.077751    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wg2tr\" (UniqueName: \"kubernetes.io/projected/f9d2226c-82f2-48cb-b1cc-1ee87b3f323d-kube-api-access-wg2tr\") pod \"payload-processing-e2e-authcfg-http-139d26-865dbd6585-gcmr9\" (UID: \"f9d2226c-82f2-48cb-b1cc-1ee87b3f323d\") " pod="openshift-ingress/payload-processing-e2e-authcfg-http-139d26-865dbd6585-gcmr9"
Oct 31 17:56:43.083517 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:43.083493    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-authcfg-http-139d26-57b4db8749-kkqrt"]
Oct 31 17:56:43.085595 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:56:43.085570    4276 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c75f040_baeb_4bba_9497_6b70b3288b83.slice/crio-9f30a6b140ae9c42080cbd3cbc1036e0c415636c3bb9e8e9b794cd6c5c0b8d2e WatchSource:0}: Error finding container 9f30a6b140ae9c42080cbd3cbc1036e0c415636c3bb9e8e9b794cd6c5c0b8d2e: Status 404 returned error can't find the container with id 9f30a6b140ae9c42080cbd3cbc1036e0c415636c3bb9e8e9b794cd6c5c0b8d2e
Oct 31 17:56:43.167605 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:43.167586    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-139d26-574fb59b4c-slv48"
Oct 31 17:56:43.257540 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:43.257512    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-authcfg-http-139d26-865dbd6585-gcmr9"
Oct 31 17:56:43.366655 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:43.366626    4276 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="cde72f26-a14d-4b50-8ab9-25d401a34d45" path="/var/lib/kubelet/pods/cde72f26-a14d-4b50-8ab9-25d401a34d45/volumes"
Oct 31 17:56:43.470234 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:43.465360    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-authcfg-http-139d26-574fb59b4c-slv48"]
Oct 31 17:56:43.557995 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:43.557965    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-authcfg-http-139d26-865dbd6585-gcmr9"]
Oct 31 17:56:43.559483 ip-10-0-129-127 kubenswrapper[4276]: W1031 17:56:43.559414    4276 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf9d2226c_82f2_48cb_b1cc_1ee87b3f323d.slice/crio-edd586a7c1bc7c50579fb2e96c1682e90736f92bfe0f6809c23493b86c2e9f8c WatchSource:0}: Error finding container edd586a7c1bc7c50579fb2e96c1682e90736f92bfe0f6809c23493b86c2e9f8c: Status 404 returned error can't find the container with id edd586a7c1bc7c50579fb2e96c1682e90736f92bfe0f6809c23493b86c2e9f8c
Oct 31 17:56:43.694286 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:43.694244    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-authcfg-http-139d26-865dbd6585-gcmr9" event={"ID":"f9d2226c-82f2-48cb-b1cc-1ee87b3f323d","Type":"ContainerStarted","Data":"6bfa03b6766f162f89baa61ddfd02135824b6aeeb620dd755ab2cb8be914fbe6"}
Oct 31 17:56:43.694519 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:43.694309    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-authcfg-http-139d26-865dbd6585-gcmr9" event={"ID":"f9d2226c-82f2-48cb-b1cc-1ee87b3f323d","Type":"ContainerStarted","Data":"edd586a7c1bc7c50579fb2e96c1682e90736f92bfe0f6809c23493b86c2e9f8c"}
Oct 31 17:56:43.694519 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:43.694429    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-authcfg-http-139d26-865dbd6585-gcmr9"
Oct 31 17:56:43.696112 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:43.696069    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-139d26-57b4db8749-kkqrt" event={"ID":"8c75f040-baeb-4bba-9497-6b70b3288b83","Type":"ContainerStarted","Data":"ba7ac01f17b7624d80f52f75b46f57da62c33582f4b7a1f518586d29f9912c03"}
Oct 31 17:56:43.696246 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:43.696117    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-139d26-57b4db8749-kkqrt" event={"ID":"8c75f040-baeb-4bba-9497-6b70b3288b83","Type":"ContainerStarted","Data":"9f30a6b140ae9c42080cbd3cbc1036e0c415636c3bb9e8e9b794cd6c5c0b8d2e"}
Oct 31 17:56:43.696246 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:43.696162    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-139d26-57b4db8749-kkqrt"
Oct 31 17:56:43.697743 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:43.697721    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-139d26-574fb59b4c-slv48" event={"ID":"453dd8b4-8c09-4c76-82e8-6891e9609684","Type":"ContainerStarted","Data":"3f5c7fb1c6ae548b5f2103c096a6b9b8edfcda6b082d6f21fbf4b088198d4c89"}
Oct 31 17:56:43.697841 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:43.697755    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-139d26-574fb59b4c-slv48" event={"ID":"453dd8b4-8c09-4c76-82e8-6891e9609684","Type":"ContainerStarted","Data":"7f75232ebdff3759aaa7d3c33c80b3795f7a881f2728f2e8cf973fc991c1c69b"}
Oct 31 17:56:43.697913 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:43.697854    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-139d26-574fb59b4c-slv48"
Oct 31 17:56:43.735008 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:43.734967    4276 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-authcfg-http-139d26-865dbd6585-gcmr9" podStartSLOduration=1.7349545100000001 podStartE2EDuration="1.73495451s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 17:56:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 17:56:43.733786039 +0000 UTC m=+1766.531530966" watchObservedRunningTime="2026-10-31 17:56:43.73495451 +0000 UTC m=+1766.532699467"
Oct 31 17:56:43.775757 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:43.775709    4276 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-139d26-57b4db8749-kkqrt" podStartSLOduration=1.44436231 podStartE2EDuration="1.775697347s" totalImagesPullingTime="331.335037ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 17:56:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 17:56:43.773967554 +0000 UTC m=+1766.571712492" watchObservedRunningTime="2026-10-31 17:56:43.775697347 +0000 UTC m=+1766.573442275"
Oct 31 17:56:43.813982 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:43.813898    4276 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-139d26-574fb59b4c-slv48" podStartSLOduration=1.813885282 podStartE2EDuration="1.813885282s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 17:56:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 17:56:43.813016402 +0000 UTC m=+1766.610761330" watchObservedRunningTime="2026-10-31 17:56:43.813885282 +0000 UTC m=+1766.611630211"
Oct 31 17:56:44.427351 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:44.427317    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:56:46.123196 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:46.123163    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:56:47.926462 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:47.926401    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:56:49.708092 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:49.708063    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-139d26-57b4db8749-kkqrt"
Oct 31 17:56:49.823859 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:49.823823    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:56:51.671467 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:51.667945    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:56:53.516264 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:53.516231    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:56:54.705973 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:54.705934    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-authcfg-http-139d26-865dbd6585-gcmr9"
Oct 31 17:56:54.713802 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:54.713767    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-139d26-574fb59b4c-slv48"
Oct 31 17:56:55.225340 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:55.225305    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:56:55.800492 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:55.800457    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:56:55.966774 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:55.965969    4276 patch_prober.go:29] interesting pod/e2e-authcfg-http-139d26-openshift-default-6f6df4f89-9zpdg container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.98:15021/healthz/ready\": read tcp 10.133.0.2:41664->10.133.0.98:15021: read: connection reset by peer" start-of-body=
Oct 31 17:56:55.966774 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:55.966046    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-authcfg-http-139d26-openshift-default-6f6df4f89-9zpdg" podUID="4c179233-456f-488d-ba8f-bb9ec7d28fd5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.98:15021/healthz/ready\": read tcp 10.133.0.2:41664->10.133.0.98:15021: read: connection reset by peer"
Oct 31 17:56:56.761558 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:56.761526    4276 generic.go:356] "Generic (PLEG): container finished" podID="4c179233-456f-488d-ba8f-bb9ec7d28fd5" containerID="d08047bdf3cb7eb8999926137f6f061f46f43183b466df2014957af420da2cee" exitCode=0
Oct 31 17:56:56.761730 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:56.761600    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-authcfg-http-139d26-openshift-default-6f6df4f89-9zpdg" event={"ID":"4c179233-456f-488d-ba8f-bb9ec7d28fd5","Type":"ContainerDied","Data":"d08047bdf3cb7eb8999926137f6f061f46f43183b466df2014957af420da2cee"}
Oct 31 17:56:56.761985 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:56.761961    4276 scope.go:122] "RemoveContainer" containerID="d08047bdf3cb7eb8999926137f6f061f46f43183b466df2014957af420da2cee"
Oct 31 17:56:56.763204 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:56.763171    4276 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 17:56:56.763279 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:56.763262    4276 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 17:56:56.763335 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:56.763319    4276 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 17:56:57.029621 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:57.029549    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:56:57.780423 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:57.780386    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-authcfg-http-139d26-openshift-default-6f6df4f89-9zpdg" event={"ID":"4c179233-456f-488d-ba8f-bb9ec7d28fd5","Type":"ContainerStarted","Data":"1730376a748e62be4e93e976738daa31f4600c2fa6078cd1aaf278c5d05b5332"}
Oct 31 17:56:57.912233 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:57.912202    4276 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-authcfg-http-139d26-openshift-default-6f6df4f89-9zpdg"
Oct 31 17:56:57.912425 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:57.912405    4276 patch_prober.go:29] interesting pod/e2e-authcfg-http-139d26-openshift-default-6f6df4f89-9zpdg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.98:15021/healthz/ready\": dial tcp 10.133.0.98:15021: connect: connection refused" start-of-body=
Oct 31 17:56:57.912511 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:57.912471    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-authcfg-http-139d26-openshift-default-6f6df4f89-9zpdg" podUID="4c179233-456f-488d-ba8f-bb9ec7d28fd5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.98:15021/healthz/ready\": dial tcp 10.133.0.98:15021: connect: connection refused"
Oct 31 17:56:58.770394 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:58.770358    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:56:58.912810 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:58.912779    4276 patch_prober.go:29] interesting pod/e2e-authcfg-http-139d26-openshift-default-6f6df4f89-9zpdg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.98:15021/healthz/ready\": dial tcp 10.133.0.98:15021: connect: connection refused" start-of-body=
Oct 31 17:56:58.913020 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:58.912832    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-authcfg-http-139d26-openshift-default-6f6df4f89-9zpdg" podUID="4c179233-456f-488d-ba8f-bb9ec7d28fd5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.98:15021/healthz/ready\": dial tcp 10.133.0.98:15021: connect: connection refused"
Oct 31 17:56:59.911965 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:59.911930    4276 patch_prober.go:29] interesting pod/e2e-authcfg-http-139d26-openshift-default-6f6df4f89-9zpdg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.98:15021/healthz/ready\": dial tcp 10.133.0.98:15021: connect: connection refused" start-of-body=
Oct 31 17:56:59.912363 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:56:59.911982    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-authcfg-http-139d26-openshift-default-6f6df4f89-9zpdg" podUID="4c179233-456f-488d-ba8f-bb9ec7d28fd5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.98:15021/healthz/ready\": dial tcp 10.133.0.98:15021: connect: connection refused"
Oct 31 17:57:00.574467 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:57:00.573590    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:57:00.912138 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:57:00.912058    4276 patch_prober.go:29] interesting pod/e2e-authcfg-http-139d26-openshift-default-6f6df4f89-9zpdg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.98:15021/healthz/ready\": dial tcp 10.133.0.98:15021: connect: connection refused" start-of-body=
Oct 31 17:57:00.912138 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:57:00.912120    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-authcfg-http-139d26-openshift-default-6f6df4f89-9zpdg" podUID="4c179233-456f-488d-ba8f-bb9ec7d28fd5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.98:15021/healthz/ready\": dial tcp 10.133.0.98:15021: connect: connection refused"
Oct 31 17:57:01.912135 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:57:01.912098    4276 patch_prober.go:29] interesting pod/e2e-authcfg-http-139d26-openshift-default-6f6df4f89-9zpdg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.98:15021/healthz/ready\": dial tcp 10.133.0.98:15021: connect: connection refused" start-of-body=
Oct 31 17:57:01.912557 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:57:01.912163    4276 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-authcfg-http-139d26-openshift-default-6f6df4f89-9zpdg" podUID="4c179233-456f-488d-ba8f-bb9ec7d28fd5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.98:15021/healthz/ready\": dial tcp 10.133.0.98:15021: connect: connection refused"
Oct 31 17:57:02.348811 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:57:02.348769    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:57:02.916099 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:57:02.916062    4276 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-authcfg-http-139d26-openshift-default-6f6df4f89-9zpdg"
Oct 31 17:57:02.916501 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:57:02.916335    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-authcfg-http-139d26-openshift-default-6f6df4f89-9zpdg"
Oct 31 17:57:02.917131 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:57:02.917113    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-authcfg-http-139d26-openshift-default-6f6df4f89-9zpdg"
Oct 31 17:57:04.115880 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:57:04.115847    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:57:05.251567 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:57:05.251531    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:57:05.736738 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:57:05.736702    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:57:07.529862 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:57:07.529809    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:57:09.213898 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:57:09.213864    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:57:10.131425 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:57:10.131393    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:57:10.930903 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:57:10.930868    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:57:12.233956 ip-10-0-129-127 kubenswrapper[4276]: E1031 17:57:12.233915    4276 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tg5qk" containerName="extractor"
Oct 31 17:57:12.627801 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:57:12.627766    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:57:14.924066 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:57:14.924024    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:57:15.135330 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:57:15.135293    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:57:16.112806 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:57:16.112755    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:57:17.529944 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:57:17.529919    4276 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zpf4d_5819fd64-9ac0-4436-b8fe-dcd3a27827ac/ovn-acl-logging/0.log"
Oct 31 17:57:17.535686 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:57:17.535667    4276 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zpf4d_5819fd64-9ac0-4436-b8fe-dcd3a27827ac/ovn-acl-logging/0.log"
Oct 31 17:57:17.818236 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:57:17.818195    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:57:19.507283 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:57:19.507243    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:57:21.219020 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:57:21.218984    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:57:23.000932 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:57:23.000897    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:57:24.733537 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:57:24.733508    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:57:26.517714 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:57:26.517683    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:57:28.214862 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:57:28.214819    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:57:29.912683 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:57:29.912646    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:57:31.623548 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:57:31.623519    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:57:33.317817 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:57:33.317783    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:57:35.015199 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:57:35.015162    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:57:36.719330 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:57:36.719298    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:57:38.423530 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:57:38.423503    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:57:40.115765 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:57:40.115730    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:57:41.900568 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:57:41.900533    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:57:42.234145 ip-10-0-129-127 kubenswrapper[4276]: E1031 17:57:42.234067    4276 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tg5qk" containerName="extractor"
Oct 31 17:57:43.554470 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:57:43.554416    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:57:45.323157 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:57:45.323123    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:57:47.018387 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:57:47.018351    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:57:48.730432 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:57:48.730399    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:57:50.501579 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:57:50.501549    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:57:52.226734 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:57:52.226696    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:57:53.933927 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:57:53.933876    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:57:55.617085 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:57:55.617053    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:57:57.335282 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:57:57.335247    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:57:59.109587 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:57:59.109559    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:58:00.820788 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:58:00.820714    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:58:02.633643 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:58:02.633608    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:58:04.385144 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:58:04.385111    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:58:05.354828 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:58:05.354796    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:58:06.027399 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:58:06.027368    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:58:07.711107 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:58:07.711068    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:58:09.440432 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:58:09.440399    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:58:10.247978 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:58:10.247945    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:58:11.146269 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:58:11.146235    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:58:12.233669 ip-10-0-129-127 kubenswrapper[4276]: E1031 17:58:12.233634    4276 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tg5qk" containerName="extractor"
Oct 31 17:58:12.917780 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:58:12.917745    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:58:14.624742 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:58:14.624709    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:58:15.249317 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:58:15.249286    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:58:16.316779 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:58:16.316748    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:58:18.020872 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:58:18.020824    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:58:19.820646 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:58:19.820598    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:58:21.517362 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:58:21.517328    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:58:23.302470 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:58:23.302410    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:58:25.013023 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:58:25.012977    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:58:26.739635 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:58:26.739604    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:58:28.440438 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:58:28.440376    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:58:30.118367 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:58:30.118339    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:58:31.816428 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:58:31.816394    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:58:33.511584 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:58:33.511550    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:58:35.212284 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:58:35.212249    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:58:36.975902 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:58:36.975869    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:58:39.224674 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:58:39.224636    4276 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-authcfg-http-139d26-865dbd6585-gcmr9"]
Oct 31 17:58:39.269845 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:58:39.269813    4276 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-authcfg-http-139d26-574fb59b4c-slv48"]
Oct 31 17:58:39.391490 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:58:39.391434    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:58:39.508036 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:58:39.507956    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:58:42.233887 ip-10-0-129-127 kubenswrapper[4276]: E1031 17:58:42.233850    4276 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tg5qk" containerName="extractor"
Oct 31 17:58:44.225607 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:58:44.225552    4276 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-authcfg-http-139d26-865dbd6585-gcmr9" podUID="f9d2226c-82f2-48cb-b1cc-1ee87b3f323d" containerName="payload-processing" containerID="cri-o://6bfa03b6766f162f89baa61ddfd02135824b6aeeb620dd755ab2cb8be914fbe6" gracePeriod=25
Oct 31 17:58:44.271053 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:58:44.271022    4276 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-139d26-574fb59b4c-slv48" podUID="453dd8b4-8c09-4c76-82e8-6891e9609684" containerName="payload-pre-processing" containerID="cri-o://3f5c7fb1c6ae548b5f2103c096a6b9b8edfcda6b082d6f21fbf4b088198d4c89" gracePeriod=25
Oct 31 17:58:44.527041 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:58:44.527011    4276 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-authcfg-http-139d26-865dbd6585-gcmr9"
Oct 31 17:58:44.557383 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:58:44.557360    4276 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-139d26-574fb59b4c-slv48"
Oct 31 17:58:44.685738 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:58:44.685700    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f9d2226c-82f2-48cb-b1cc-1ee87b3f323d-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f9d2226c-82f2-48cb-b1cc-1ee87b3f323d-plugins-config-volume\") pod \"f9d2226c-82f2-48cb-b1cc-1ee87b3f323d\" (UID: \"f9d2226c-82f2-48cb-b1cc-1ee87b3f323d\") "
Oct 31 17:58:44.685975 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:58:44.685770    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/453dd8b4-8c09-4c76-82e8-6891e9609684-kube-api-access-xclcn\" (UniqueName: \"kubernetes.io/projected/453dd8b4-8c09-4c76-82e8-6891e9609684-kube-api-access-xclcn\") pod \"453dd8b4-8c09-4c76-82e8-6891e9609684\" (UID: \"453dd8b4-8c09-4c76-82e8-6891e9609684\") "
Oct 31 17:58:44.685975 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:58:44.685827    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f9d2226c-82f2-48cb-b1cc-1ee87b3f323d-kube-api-access-wg2tr\" (UniqueName: \"kubernetes.io/projected/f9d2226c-82f2-48cb-b1cc-1ee87b3f323d-kube-api-access-wg2tr\") pod \"f9d2226c-82f2-48cb-b1cc-1ee87b3f323d\" (UID: \"f9d2226c-82f2-48cb-b1cc-1ee87b3f323d\") "
Oct 31 17:58:44.685975 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:58:44.685932    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/453dd8b4-8c09-4c76-82e8-6891e9609684-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/453dd8b4-8c09-4c76-82e8-6891e9609684-plugins-config-volume\") pod \"453dd8b4-8c09-4c76-82e8-6891e9609684\" (UID: \"453dd8b4-8c09-4c76-82e8-6891e9609684\") "
Oct 31 17:58:44.686283 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:58:44.686118    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f9d2226c-82f2-48cb-b1cc-1ee87b3f323d-plugins-config-volume" pod "f9d2226c-82f2-48cb-b1cc-1ee87b3f323d" (UID: "f9d2226c-82f2-48cb-b1cc-1ee87b3f323d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 17:58:44.686413 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:58:44.686298    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/453dd8b4-8c09-4c76-82e8-6891e9609684-plugins-config-volume" pod "453dd8b4-8c09-4c76-82e8-6891e9609684" (UID: "453dd8b4-8c09-4c76-82e8-6891e9609684"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 17:58:44.687960 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:58:44.687939    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/453dd8b4-8c09-4c76-82e8-6891e9609684-kube-api-access-xclcn" pod "453dd8b4-8c09-4c76-82e8-6891e9609684" (UID: "453dd8b4-8c09-4c76-82e8-6891e9609684"). InnerVolumeSpecName "kube-api-access-xclcn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 17:58:44.688047 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:58:44.687979    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f9d2226c-82f2-48cb-b1cc-1ee87b3f323d-kube-api-access-wg2tr" pod "f9d2226c-82f2-48cb-b1cc-1ee87b3f323d" (UID: "f9d2226c-82f2-48cb-b1cc-1ee87b3f323d"). InnerVolumeSpecName "kube-api-access-wg2tr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 17:58:44.786861 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:58:44.786780    4276 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f9d2226c-82f2-48cb-b1cc-1ee87b3f323d-plugins-config-volume\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:58:44.786861 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:58:44.786807    4276 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xclcn\" (UniqueName: \"kubernetes.io/projected/453dd8b4-8c09-4c76-82e8-6891e9609684-kube-api-access-xclcn\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:58:44.786861 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:58:44.786816    4276 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wg2tr\" (UniqueName: \"kubernetes.io/projected/f9d2226c-82f2-48cb-b1cc-1ee87b3f323d-kube-api-access-wg2tr\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:58:44.786861 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:58:44.786826    4276 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/453dd8b4-8c09-4c76-82e8-6891e9609684-plugins-config-volume\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:58:45.289717 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:58:45.289679    4276 generic.go:356] "Generic (PLEG): container finished" podID="453dd8b4-8c09-4c76-82e8-6891e9609684" containerID="3f5c7fb1c6ae548b5f2103c096a6b9b8edfcda6b082d6f21fbf4b088198d4c89" exitCode=0
Oct 31 17:58:45.290189 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:58:45.289724    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-139d26-574fb59b4c-slv48" event={"ID":"453dd8b4-8c09-4c76-82e8-6891e9609684","Type":"ContainerDied","Data":"3f5c7fb1c6ae548b5f2103c096a6b9b8edfcda6b082d6f21fbf4b088198d4c89"}
Oct 31 17:58:45.290189 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:58:45.289738    4276 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-139d26-574fb59b4c-slv48"
Oct 31 17:58:45.290189 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:58:45.289796    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-139d26-574fb59b4c-slv48" event={"ID":"453dd8b4-8c09-4c76-82e8-6891e9609684","Type":"ContainerDied","Data":"7f75232ebdff3759aaa7d3c33c80b3795f7a881f2728f2e8cf973fc991c1c69b"}
Oct 31 17:58:45.290189 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:58:45.289818    4276 scope.go:122] "RemoveContainer" containerID="3f5c7fb1c6ae548b5f2103c096a6b9b8edfcda6b082d6f21fbf4b088198d4c89"
Oct 31 17:58:45.291325 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:58:45.291281    4276 generic.go:356] "Generic (PLEG): container finished" podID="f9d2226c-82f2-48cb-b1cc-1ee87b3f323d" containerID="6bfa03b6766f162f89baa61ddfd02135824b6aeeb620dd755ab2cb8be914fbe6" exitCode=0
Oct 31 17:58:45.291325 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:58:45.291314    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-authcfg-http-139d26-865dbd6585-gcmr9" event={"ID":"f9d2226c-82f2-48cb-b1cc-1ee87b3f323d","Type":"ContainerDied","Data":"6bfa03b6766f162f89baa61ddfd02135824b6aeeb620dd755ab2cb8be914fbe6"}
Oct 31 17:58:45.291532 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:58:45.291345    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-authcfg-http-139d26-865dbd6585-gcmr9" event={"ID":"f9d2226c-82f2-48cb-b1cc-1ee87b3f323d","Type":"ContainerDied","Data":"edd586a7c1bc7c50579fb2e96c1682e90736f92bfe0f6809c23493b86c2e9f8c"}
Oct 31 17:58:45.291532 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:58:45.291347    4276 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-authcfg-http-139d26-865dbd6585-gcmr9"
Oct 31 17:58:45.299758 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:58:45.299742    4276 scope.go:122] "RemoveContainer" containerID="3f5c7fb1c6ae548b5f2103c096a6b9b8edfcda6b082d6f21fbf4b088198d4c89"
Oct 31 17:58:45.300025 ip-10-0-129-127 kubenswrapper[4276]: E1031 17:58:45.300009    4276 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3f5c7fb1c6ae548b5f2103c096a6b9b8edfcda6b082d6f21fbf4b088198d4c89\": container with ID starting with 3f5c7fb1c6ae548b5f2103c096a6b9b8edfcda6b082d6f21fbf4b088198d4c89 not found: ID does not exist" containerID="3f5c7fb1c6ae548b5f2103c096a6b9b8edfcda6b082d6f21fbf4b088198d4c89"
Oct 31 17:58:45.300073 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:58:45.300032    4276 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3f5c7fb1c6ae548b5f2103c096a6b9b8edfcda6b082d6f21fbf4b088198d4c89"} err="failed to get container status \"3f5c7fb1c6ae548b5f2103c096a6b9b8edfcda6b082d6f21fbf4b088198d4c89\": rpc error: code = NotFound desc = could not find container \"3f5c7fb1c6ae548b5f2103c096a6b9b8edfcda6b082d6f21fbf4b088198d4c89\": container with ID starting with 3f5c7fb1c6ae548b5f2103c096a6b9b8edfcda6b082d6f21fbf4b088198d4c89 not found: ID does not exist"
Oct 31 17:58:45.300073 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:58:45.300049    4276 scope.go:122] "RemoveContainer" containerID="6bfa03b6766f162f89baa61ddfd02135824b6aeeb620dd755ab2cb8be914fbe6"
Oct 31 17:58:45.307582 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:58:45.307563    4276 scope.go:122] "RemoveContainer" containerID="6bfa03b6766f162f89baa61ddfd02135824b6aeeb620dd755ab2cb8be914fbe6"
Oct 31 17:58:45.307826 ip-10-0-129-127 kubenswrapper[4276]: E1031 17:58:45.307808    4276 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6bfa03b6766f162f89baa61ddfd02135824b6aeeb620dd755ab2cb8be914fbe6\": container with ID starting with 6bfa03b6766f162f89baa61ddfd02135824b6aeeb620dd755ab2cb8be914fbe6 not found: ID does not exist" containerID="6bfa03b6766f162f89baa61ddfd02135824b6aeeb620dd755ab2cb8be914fbe6"
Oct 31 17:58:45.307897 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:58:45.307837    4276 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6bfa03b6766f162f89baa61ddfd02135824b6aeeb620dd755ab2cb8be914fbe6"} err="failed to get container status \"6bfa03b6766f162f89baa61ddfd02135824b6aeeb620dd755ab2cb8be914fbe6\": rpc error: code = NotFound desc = could not find container \"6bfa03b6766f162f89baa61ddfd02135824b6aeeb620dd755ab2cb8be914fbe6\": container with ID starting with 6bfa03b6766f162f89baa61ddfd02135824b6aeeb620dd755ab2cb8be914fbe6 not found: ID does not exist"
Oct 31 17:58:45.344015 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:58:45.343989    4276 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-authcfg-http-139d26-574fb59b4c-slv48"]
Oct 31 17:58:45.357804 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:58:45.357781    4276 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-authcfg-http-139d26-574fb59b4c-slv48"]
Oct 31 17:58:45.366280 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:58:45.366258    4276 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="453dd8b4-8c09-4c76-82e8-6891e9609684" path="/var/lib/kubelet/pods/453dd8b4-8c09-4c76-82e8-6891e9609684/volumes"
Oct 31 17:58:45.393436 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:58:45.393406    4276 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-authcfg-http-139d26-865dbd6585-gcmr9"]
Oct 31 17:58:45.396033 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:58:45.396008    4276 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-authcfg-http-139d26-865dbd6585-gcmr9"]
Oct 31 17:58:45.914958 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:58:45.914925    4276 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-authcfg-http-139d26-57b4db8749-kkqrt"]
Oct 31 17:58:45.915394 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:58:45.915357    4276 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-139d26-57b4db8749-kkqrt" podUID="8c75f040-baeb-4bba-9497-6b70b3288b83" containerName="maas-api" containerID="cri-o://ba7ac01f17b7624d80f52f75b46f57da62c33582f4b7a1f518586d29f9912c03" gracePeriod=30
Oct 31 17:58:45.994352 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:58:45.994324    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:58:46.181451 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:58:46.181373    4276 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-139d26-57b4db8749-kkqrt"
Oct 31 17:58:46.296947 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:58:46.296918    4276 generic.go:356] "Generic (PLEG): container finished" podID="8c75f040-baeb-4bba-9497-6b70b3288b83" containerID="ba7ac01f17b7624d80f52f75b46f57da62c33582f4b7a1f518586d29f9912c03" exitCode=0
Oct 31 17:58:46.297401 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:58:46.296960    4276 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-139d26-57b4db8749-kkqrt"
Oct 31 17:58:46.297401 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:58:46.297002    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-139d26-57b4db8749-kkqrt" event={"ID":"8c75f040-baeb-4bba-9497-6b70b3288b83","Type":"ContainerDied","Data":"ba7ac01f17b7624d80f52f75b46f57da62c33582f4b7a1f518586d29f9912c03"}
Oct 31 17:58:46.297401 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:58:46.297060    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-139d26-57b4db8749-kkqrt" event={"ID":"8c75f040-baeb-4bba-9497-6b70b3288b83","Type":"ContainerDied","Data":"9f30a6b140ae9c42080cbd3cbc1036e0c415636c3bb9e8e9b794cd6c5c0b8d2e"}
Oct 31 17:58:46.297401 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:58:46.297097    4276 scope.go:122] "RemoveContainer" containerID="ba7ac01f17b7624d80f52f75b46f57da62c33582f4b7a1f518586d29f9912c03"
Oct 31 17:58:46.298927 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:58:46.298899    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8c75f040-baeb-4bba-9497-6b70b3288b83-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8c75f040-baeb-4bba-9497-6b70b3288b83-maas-api-tls\") pod \"8c75f040-baeb-4bba-9497-6b70b3288b83\" (UID: \"8c75f040-baeb-4bba-9497-6b70b3288b83\") "
Oct 31 17:58:46.299028 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:58:46.298939    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8c75f040-baeb-4bba-9497-6b70b3288b83-kube-api-access-bfxmh\" (UniqueName: \"kubernetes.io/projected/8c75f040-baeb-4bba-9497-6b70b3288b83-kube-api-access-bfxmh\") pod \"8c75f040-baeb-4bba-9497-6b70b3288b83\" (UID: \"8c75f040-baeb-4bba-9497-6b70b3288b83\") "
Oct 31 17:58:46.299092 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:58:46.299024    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8c75f040-baeb-4bba-9497-6b70b3288b83-metrics-certs\" (UniqueName: \"kubernetes.io/secret/8c75f040-baeb-4bba-9497-6b70b3288b83-metrics-certs\") pod \"8c75f040-baeb-4bba-9497-6b70b3288b83\" (UID: \"8c75f040-baeb-4bba-9497-6b70b3288b83\") "
Oct 31 17:58:46.299154 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:58:46.299099    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8c75f040-baeb-4bba-9497-6b70b3288b83-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8c75f040-baeb-4bba-9497-6b70b3288b83-openshift-service-ca\") pod \"8c75f040-baeb-4bba-9497-6b70b3288b83\" (UID: \"8c75f040-baeb-4bba-9497-6b70b3288b83\") "
Oct 31 17:58:46.299504 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:58:46.299482    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8c75f040-baeb-4bba-9497-6b70b3288b83-openshift-service-ca" pod "8c75f040-baeb-4bba-9497-6b70b3288b83" (UID: "8c75f040-baeb-4bba-9497-6b70b3288b83"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 17:58:46.300951 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:58:46.300925    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8c75f040-baeb-4bba-9497-6b70b3288b83-maas-api-tls" pod "8c75f040-baeb-4bba-9497-6b70b3288b83" (UID: "8c75f040-baeb-4bba-9497-6b70b3288b83"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 17:58:46.301097 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:58:46.301016    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8c75f040-baeb-4bba-9497-6b70b3288b83-kube-api-access-bfxmh" pod "8c75f040-baeb-4bba-9497-6b70b3288b83" (UID: "8c75f040-baeb-4bba-9497-6b70b3288b83"). InnerVolumeSpecName "kube-api-access-bfxmh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 17:58:46.301357 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:58:46.301340    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8c75f040-baeb-4bba-9497-6b70b3288b83-metrics-certs" pod "8c75f040-baeb-4bba-9497-6b70b3288b83" (UID: "8c75f040-baeb-4bba-9497-6b70b3288b83"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 17:58:46.317896 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:58:46.317876    4276 scope.go:122] "RemoveContainer" containerID="ba7ac01f17b7624d80f52f75b46f57da62c33582f4b7a1f518586d29f9912c03"
Oct 31 17:58:46.318148 ip-10-0-129-127 kubenswrapper[4276]: E1031 17:58:46.318131    4276 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ba7ac01f17b7624d80f52f75b46f57da62c33582f4b7a1f518586d29f9912c03\": container with ID starting with ba7ac01f17b7624d80f52f75b46f57da62c33582f4b7a1f518586d29f9912c03 not found: ID does not exist" containerID="ba7ac01f17b7624d80f52f75b46f57da62c33582f4b7a1f518586d29f9912c03"
Oct 31 17:58:46.318197 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:58:46.318157    4276 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ba7ac01f17b7624d80f52f75b46f57da62c33582f4b7a1f518586d29f9912c03"} err="failed to get container status \"ba7ac01f17b7624d80f52f75b46f57da62c33582f4b7a1f518586d29f9912c03\": rpc error: code = NotFound desc = could not find container \"ba7ac01f17b7624d80f52f75b46f57da62c33582f4b7a1f518586d29f9912c03\": container with ID starting with ba7ac01f17b7624d80f52f75b46f57da62c33582f4b7a1f518586d29f9912c03 not found: ID does not exist"
Oct 31 17:58:46.400581 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:58:46.400546    4276 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8c75f040-baeb-4bba-9497-6b70b3288b83-maas-api-tls\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:58:46.400581 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:58:46.400582    4276 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bfxmh\" (UniqueName: \"kubernetes.io/projected/8c75f040-baeb-4bba-9497-6b70b3288b83-kube-api-access-bfxmh\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:58:46.400779 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:58:46.400598    4276 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8c75f040-baeb-4bba-9497-6b70b3288b83-metrics-certs\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:58:46.400779 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:58:46.400613    4276 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8c75f040-baeb-4bba-9497-6b70b3288b83-openshift-service-ca\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:58:46.493485 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:58:46.493380    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:58:46.608489 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:58:46.608461    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:58:46.646210 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:58:46.646182    4276 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-authcfg-http-139d26-57b4db8749-kkqrt"]
Oct 31 17:58:46.652194 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:58:46.652172    4276 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-authcfg-http-139d26-57b4db8749-kkqrt"]
Oct 31 17:58:47.153810 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:58:47.153783    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:58:47.365406 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:58:47.365366    4276 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8c75f040-baeb-4bba-9497-6b70b3288b83" path="/var/lib/kubelet/pods/8c75f040-baeb-4bba-9497-6b70b3288b83/volumes"
Oct 31 17:58:47.365917 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:58:47.365900    4276 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f9d2226c-82f2-48cb-b1cc-1ee87b3f323d" path="/var/lib/kubelet/pods/f9d2226c-82f2-48cb-b1cc-1ee87b3f323d/volumes"
Oct 31 17:58:52.680245 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:58:52.680209    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:58:52.802636 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:58:52.802599    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:58:52.950500 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:58:52.950406    4276 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-authcfg-http-139d26-openshift-default-6f6df4f89-9zpdg"]
Oct 31 17:58:52.950715 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:58:52.950690    4276 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-authcfg-http-139d26-openshift-default-6f6df4f89-9zpdg" podUID="4c179233-456f-488d-ba8f-bb9ec7d28fd5" containerName="istio-proxy" containerID="cri-o://1730376a748e62be4e93e976738daa31f4600c2fa6078cd1aaf278c5d05b5332" gracePeriod=30
Oct 31 17:58:52.995947 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:58:52.995918    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:58:58.198142 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:58:58.198117    4276 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-authcfg-http-139d26-openshift-default-6f6df4f89-9zpdg"
Oct 31 17:58:58.222146 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:58:58.222093    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4c179233-456f-488d-ba8f-bb9ec7d28fd5-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4c179233-456f-488d-ba8f-bb9ec7d28fd5-workload-certs\") pod \"4c179233-456f-488d-ba8f-bb9ec7d28fd5\" (UID: \"4c179233-456f-488d-ba8f-bb9ec7d28fd5\") "
Oct 31 17:58:58.222254 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:58:58.222197    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4c179233-456f-488d-ba8f-bb9ec7d28fd5-istio-token\" (UniqueName: \"kubernetes.io/projected/4c179233-456f-488d-ba8f-bb9ec7d28fd5-istio-token\") pod \"4c179233-456f-488d-ba8f-bb9ec7d28fd5\" (UID: \"4c179233-456f-488d-ba8f-bb9ec7d28fd5\") "
Oct 31 17:58:58.222254 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:58:58.222238    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4c179233-456f-488d-ba8f-bb9ec7d28fd5-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4c179233-456f-488d-ba8f-bb9ec7d28fd5-istiod-ca-cert\") pod \"4c179233-456f-488d-ba8f-bb9ec7d28fd5\" (UID: \"4c179233-456f-488d-ba8f-bb9ec7d28fd5\") "
Oct 31 17:58:58.222351 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:58:58.222270    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4c179233-456f-488d-ba8f-bb9ec7d28fd5-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c179233-456f-488d-ba8f-bb9ec7d28fd5-credential-socket\") pod \"4c179233-456f-488d-ba8f-bb9ec7d28fd5\" (UID: \"4c179233-456f-488d-ba8f-bb9ec7d28fd5\") "
Oct 31 17:58:58.222351 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:58:58.222312    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4c179233-456f-488d-ba8f-bb9ec7d28fd5-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4c179233-456f-488d-ba8f-bb9ec7d28fd5-istio-data\") pod \"4c179233-456f-488d-ba8f-bb9ec7d28fd5\" (UID: \"4c179233-456f-488d-ba8f-bb9ec7d28fd5\") "
Oct 31 17:58:58.222351 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:58:58.222339    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/4c179233-456f-488d-ba8f-bb9ec7d28fd5-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4c179233-456f-488d-ba8f-bb9ec7d28fd5-istio-podinfo\") pod \"4c179233-456f-488d-ba8f-bb9ec7d28fd5\" (UID: \"4c179233-456f-488d-ba8f-bb9ec7d28fd5\") "
Oct 31 17:58:58.222545 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:58:58.222373    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c179233-456f-488d-ba8f-bb9ec7d28fd5-workload-certs" pod "4c179233-456f-488d-ba8f-bb9ec7d28fd5" (UID: "4c179233-456f-488d-ba8f-bb9ec7d28fd5"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:58:58.222545 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:58:58.222394    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4c179233-456f-488d-ba8f-bb9ec7d28fd5-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c179233-456f-488d-ba8f-bb9ec7d28fd5-workload-socket\") pod \"4c179233-456f-488d-ba8f-bb9ec7d28fd5\" (UID: \"4c179233-456f-488d-ba8f-bb9ec7d28fd5\") "
Oct 31 17:58:58.222545 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:58:58.222427    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4c179233-456f-488d-ba8f-bb9ec7d28fd5-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4c179233-456f-488d-ba8f-bb9ec7d28fd5-istio-envoy\") pod \"4c179233-456f-488d-ba8f-bb9ec7d28fd5\" (UID: \"4c179233-456f-488d-ba8f-bb9ec7d28fd5\") "
Oct 31 17:58:58.222545 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:58:58.222487    4276 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4c179233-456f-488d-ba8f-bb9ec7d28fd5-kube-api-access-p87jf\" (UniqueName: \"kubernetes.io/projected/4c179233-456f-488d-ba8f-bb9ec7d28fd5-kube-api-access-p87jf\") pod \"4c179233-456f-488d-ba8f-bb9ec7d28fd5\" (UID: \"4c179233-456f-488d-ba8f-bb9ec7d28fd5\") "
Oct 31 17:58:58.222744 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:58:58.222637    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c179233-456f-488d-ba8f-bb9ec7d28fd5-credential-socket" pod "4c179233-456f-488d-ba8f-bb9ec7d28fd5" (UID: "4c179233-456f-488d-ba8f-bb9ec7d28fd5"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:58:58.222744 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:58:58.222644    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4c179233-456f-488d-ba8f-bb9ec7d28fd5-istiod-ca-cert" pod "4c179233-456f-488d-ba8f-bb9ec7d28fd5" (UID: "4c179233-456f-488d-ba8f-bb9ec7d28fd5"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 17:58:58.222839 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:58:58.222770    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c179233-456f-488d-ba8f-bb9ec7d28fd5-workload-socket" pod "4c179233-456f-488d-ba8f-bb9ec7d28fd5" (UID: "4c179233-456f-488d-ba8f-bb9ec7d28fd5"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:58:58.222937 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:58:58.222917    4276 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4c179233-456f-488d-ba8f-bb9ec7d28fd5-istiod-ca-cert\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:58:58.223016 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:58:58.222943    4276 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c179233-456f-488d-ba8f-bb9ec7d28fd5-credential-socket\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:58:58.223016 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:58:58.222958    4276 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c179233-456f-488d-ba8f-bb9ec7d28fd5-workload-socket\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:58:58.223016 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:58:58.222973    4276 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4c179233-456f-488d-ba8f-bb9ec7d28fd5-workload-certs\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:58:58.223191 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:58:58.223072    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c179233-456f-488d-ba8f-bb9ec7d28fd5-istio-data" pod "4c179233-456f-488d-ba8f-bb9ec7d28fd5" (UID: "4c179233-456f-488d-ba8f-bb9ec7d28fd5"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:58:58.224419 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:58:58.224399    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4c179233-456f-488d-ba8f-bb9ec7d28fd5-istio-podinfo" pod "4c179233-456f-488d-ba8f-bb9ec7d28fd5" (UID: "4c179233-456f-488d-ba8f-bb9ec7d28fd5"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 17:58:58.224733 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:58:58.224716    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c179233-456f-488d-ba8f-bb9ec7d28fd5-istio-envoy" pod "4c179233-456f-488d-ba8f-bb9ec7d28fd5" (UID: "4c179233-456f-488d-ba8f-bb9ec7d28fd5"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 17:58:58.224832 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:58:58.224809    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4c179233-456f-488d-ba8f-bb9ec7d28fd5-istio-token" pod "4c179233-456f-488d-ba8f-bb9ec7d28fd5" (UID: "4c179233-456f-488d-ba8f-bb9ec7d28fd5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 17:58:58.224894 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:58:58.224843    4276 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4c179233-456f-488d-ba8f-bb9ec7d28fd5-kube-api-access-p87jf" pod "4c179233-456f-488d-ba8f-bb9ec7d28fd5" (UID: "4c179233-456f-488d-ba8f-bb9ec7d28fd5"). InnerVolumeSpecName "kube-api-access-p87jf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 17:58:58.324291 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:58:58.324261    4276 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4c179233-456f-488d-ba8f-bb9ec7d28fd5-istio-token\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:58:58.324291 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:58:58.324287    4276 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4c179233-456f-488d-ba8f-bb9ec7d28fd5-istio-data\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:58:58.324291 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:58:58.324296    4276 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4c179233-456f-488d-ba8f-bb9ec7d28fd5-istio-podinfo\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:58:58.324534 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:58:58.324303    4276 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4c179233-456f-488d-ba8f-bb9ec7d28fd5-istio-envoy\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:58:58.324534 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:58:58.324312    4276 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p87jf\" (UniqueName: \"kubernetes.io/projected/4c179233-456f-488d-ba8f-bb9ec7d28fd5-kube-api-access-p87jf\") on node \"ip-10-0-129-127.ec2.internal\" DevicePath \"\""
Oct 31 17:58:58.364214 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:58:58.364185    4276 generic.go:356] "Generic (PLEG): container finished" podID="4c179233-456f-488d-ba8f-bb9ec7d28fd5" containerID="1730376a748e62be4e93e976738daa31f4600c2fa6078cd1aaf278c5d05b5332" exitCode=0
Oct 31 17:58:58.364368 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:58:58.364231    4276 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-authcfg-http-139d26-openshift-default-6f6df4f89-9zpdg"
Oct 31 17:58:58.364368 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:58:58.364249    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-authcfg-http-139d26-openshift-default-6f6df4f89-9zpdg" event={"ID":"4c179233-456f-488d-ba8f-bb9ec7d28fd5","Type":"ContainerDied","Data":"1730376a748e62be4e93e976738daa31f4600c2fa6078cd1aaf278c5d05b5332"}
Oct 31 17:58:58.364368 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:58:58.364295    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-authcfg-http-139d26-openshift-default-6f6df4f89-9zpdg" event={"ID":"4c179233-456f-488d-ba8f-bb9ec7d28fd5","Type":"ContainerDied","Data":"d6588385758a0a7e09f4706009018d19da8955ab3ab816acf508b686cbe4ab0f"}
Oct 31 17:58:58.364368 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:58:58.364324    4276 scope.go:122] "RemoveContainer" containerID="1730376a748e62be4e93e976738daa31f4600c2fa6078cd1aaf278c5d05b5332"
Oct 31 17:58:58.373907 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:58:58.373886    4276 scope.go:122] "RemoveContainer" containerID="d08047bdf3cb7eb8999926137f6f061f46f43183b466df2014957af420da2cee"
Oct 31 17:58:58.381576 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:58:58.381554    4276 scope.go:122] "RemoveContainer" containerID="1730376a748e62be4e93e976738daa31f4600c2fa6078cd1aaf278c5d05b5332"
Oct 31 17:58:58.381827 ip-10-0-129-127 kubenswrapper[4276]: E1031 17:58:58.381806    4276 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1730376a748e62be4e93e976738daa31f4600c2fa6078cd1aaf278c5d05b5332\": container with ID starting with 1730376a748e62be4e93e976738daa31f4600c2fa6078cd1aaf278c5d05b5332 not found: ID does not exist" containerID="1730376a748e62be4e93e976738daa31f4600c2fa6078cd1aaf278c5d05b5332"
Oct 31 17:58:58.381938 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:58:58.381835    4276 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1730376a748e62be4e93e976738daa31f4600c2fa6078cd1aaf278c5d05b5332"} err="failed to get container status \"1730376a748e62be4e93e976738daa31f4600c2fa6078cd1aaf278c5d05b5332\": rpc error: code = NotFound desc = could not find container \"1730376a748e62be4e93e976738daa31f4600c2fa6078cd1aaf278c5d05b5332\": container with ID starting with 1730376a748e62be4e93e976738daa31f4600c2fa6078cd1aaf278c5d05b5332 not found: ID does not exist"
Oct 31 17:58:58.381938 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:58:58.381856    4276 scope.go:122] "RemoveContainer" containerID="d08047bdf3cb7eb8999926137f6f061f46f43183b466df2014957af420da2cee"
Oct 31 17:58:58.382089 ip-10-0-129-127 kubenswrapper[4276]: E1031 17:58:58.382075    4276 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d08047bdf3cb7eb8999926137f6f061f46f43183b466df2014957af420da2cee\": container with ID starting with d08047bdf3cb7eb8999926137f6f061f46f43183b466df2014957af420da2cee not found: ID does not exist" containerID="d08047bdf3cb7eb8999926137f6f061f46f43183b466df2014957af420da2cee"
Oct 31 17:58:58.382157 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:58:58.382091    4276 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d08047bdf3cb7eb8999926137f6f061f46f43183b466df2014957af420da2cee"} err="failed to get container status \"d08047bdf3cb7eb8999926137f6f061f46f43183b466df2014957af420da2cee\": rpc error: code = NotFound desc = could not find container \"d08047bdf3cb7eb8999926137f6f061f46f43183b466df2014957af420da2cee\": container with ID starting with d08047bdf3cb7eb8999926137f6f061f46f43183b466df2014957af420da2cee not found: ID does not exist"
Oct 31 17:58:58.417839 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:58:58.417818    4276 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-authcfg-http-139d26-openshift-default-6f6df4f89-9zpdg"]
Oct 31 17:58:58.423789 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:58:58.423770    4276 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-authcfg-http-139d26-openshift-default-6f6df4f89-9zpdg"]
Oct 31 17:58:59.367280 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:58:59.367250    4276 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4c179233-456f-488d-ba8f-bb9ec7d28fd5" path="/var/lib/kubelet/pods/4c179233-456f-488d-ba8f-bb9ec7d28fd5/volumes"
Oct 31 17:59:05.470621 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:59:05.470586    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:59:10.351984 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:59:10.351935    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:59:12.347216 ip-10-0-129-127 kubenswrapper[4276]: E1031 17:59:12.347145    4276 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tg5qk" containerName="extractor"
Oct 31 17:59:12.632051 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:59:12.631959    4276 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7d9897b54c-98fcj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 17:59:12.632355 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:59:12.632324    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-98fcj" podUID="9421021e-be39-4906-8fae-da1ce9ba4047" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 17:59:12.751080 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:59:12.750599    4276 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7d9897b54c-p4v97 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 17:59:12.751080 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:59:12.750666    4276 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-p4v97" podUID="3ffeb078-cb4d-4293-b7fb-eb6fd305d96b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 17:59:15.354069 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:59:15.354028    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:59:22.498643 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:59:22.498605    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:59:24.663823 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:59:24.663790    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:59:24.775644 ip-10-0-129-127 kubenswrapper[4276]: I1031 17:59:24.775610    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 17:59:39.602850 ip-10-0-129-127 kubenswrapper[4276]: E1031 17:59:39.602811    4276 controller.go:251] "Failed to update lease" err="Put \"https://a756852771c0044279d6ba9a0e15a6dd-c7c3820165848559.elb.us-east-2.amazonaws.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ip-10-0-129-127.ec2.internal?timeout=10s\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 31 17:59:40.111031 ip-10-0-129-127 kubenswrapper[4276]: E1031 17:59:40.110983    4276 kubelet_node_status.go:524] "Error updating node status, will retry" err="failed to patch status \"{\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"type\\\":\\\"DiskPressure\\\"},{\\\"type\\\":\\\"PIDPressure\\\"},{\\\"type\\\":\\\"Ready\\\"}],\\\"conditions\\\":[{\\\"lastHeartbeatTime\\\":\\\"2026-10-31T17:59:30Z\\\",\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-10-31T17:59:30Z\\\",\\\"type\\\":\\\"DiskPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-10-31T17:59:30Z\\\",\\\"type\\\":\\\"PIDPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-10-31T17:59:30Z\\\",\\\"type\\\":\\\"Ready\\\"}],\\\"runtimeHandlers\\\":[{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"runc\\\"},{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"crun\\\"},{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"\\\"}]}}\" for node \"ip-10-0-129-127.ec2.internal\": Patch \"https://a756852771c0044279d6ba9a0e15a6dd-c7c3820165848559.elb.us-east-2.amazonaws.com:6443/api/v1/nodes/ip-10-0-129-127.ec2.internal/status?timeout=10s\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 31 17:59:42.232907 ip-10-0-129-127 kubenswrapper[4276]: E1031 17:59:42.232870    4276 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tg5qk" containerName="extractor"
Oct 31 17:59:49.604003 ip-10-0-129-127 kubenswrapper[4276]: E1031 17:59:49.603957    4276 controller.go:251] "Failed to update lease" err="Put \"https://a756852771c0044279d6ba9a0e15a6dd-c7c3820165848559.elb.us-east-2.amazonaws.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ip-10-0-129-127.ec2.internal?timeout=10s\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 31 17:59:50.111825 ip-10-0-129-127 kubenswrapper[4276]: E1031 17:59:50.111787    4276 kubelet_node_status.go:524] "Error updating node status, will retry" err="error getting node \"ip-10-0-129-127.ec2.internal\": Get \"https://a756852771c0044279d6ba9a0e15a6dd-c7c3820165848559.elb.us-east-2.amazonaws.com:6443/api/v1/nodes/ip-10-0-129-127.ec2.internal?timeout=10s\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 31 17:59:59.604841 ip-10-0-129-127 kubenswrapper[4276]: E1031 17:59:59.604801    4276 controller.go:251] "Failed to update lease" err="Put \"https://a756852771c0044279d6ba9a0e15a6dd-c7c3820165848559.elb.us-east-2.amazonaws.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ip-10-0-129-127.ec2.internal?timeout=10s\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 31 18:00:00.113143 ip-10-0-129-127 kubenswrapper[4276]: E1031 18:00:00.113113    4276 kubelet_node_status.go:524] "Error updating node status, will retry" err="error getting node \"ip-10-0-129-127.ec2.internal\": Get \"https://a756852771c0044279d6ba9a0e15a6dd-c7c3820165848559.elb.us-east-2.amazonaws.com:6443/api/v1/nodes/ip-10-0-129-127.ec2.internal?timeout=10s\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 31 18:00:04.243313 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:04.243262    4276 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-console\"/\"service-ca\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Oct 31 18:00:04.243313 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:04.243262    4276 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Oct 31 18:00:04.243313 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:04.243300    4276 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Oct 31 18:00:04.243313 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:04.243270    4276 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Oct 31 18:00:04.244007 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:04.243301    4276 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Oct 31 18:00:04.244007 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:04.243339    4276 reflector.go:578] "Warning: watch ended with error" reflector="object-\"kube-system\"/\"original-pull-secret\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Oct 31 18:00:04.244007 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:04.243350    4276 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Oct 31 18:00:04.244007 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:04.243370    4276 reflector.go:578] "Warning: watch ended with error" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-k6pvq\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Oct 31 18:00:04.244007 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:04.243317    4276 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Oct 31 18:00:04.244007 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:04.243388    4276 reflector.go:578] "Warning: watch ended with error" reflector="object-\"llm\"/\"default-dockercfg-vhxrl\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Oct 31 18:00:04.244007 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:04.243299    4276 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Oct 31 18:00:04.244007 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:04.243262    4276 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Oct 31 18:00:04.244007 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:04.243400    4276 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-console\"/\"console-serving-cert\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Oct 31 18:00:04.244007 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:04.243287    4276 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Oct 31 18:00:04.244007 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:04.243405    4276 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-dns\"/\"dns-default\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Oct 31 18:00:04.244007 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:04.243429    4276 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Oct 31 18:00:04.244007 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:04.243372    4276 reflector.go:578] "Warning: watch ended with error" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-8bpqv\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Oct 31 18:00:04.244007 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:04.243471    4276 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Oct 31 18:00:04.244007 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:04.243478    4276 reflector.go:578] "Warning: watch ended with error" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Oct 31 18:00:04.244007 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:04.243485    4276 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.CSIDriver" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Oct 31 18:00:04.244007 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:04.243522    4276 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-f4hkh\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Oct 31 18:00:04.244007 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:04.243565    4276 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Oct 31 18:00:04.244007 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:04.243580    4276 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Oct 31 18:00:04.244007 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:04.243602    4276 reflector.go:578] "Warning: watch ended with error" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Oct 31 18:00:04.244007 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:04.243603    4276 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-hpj5l\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Oct 31 18:00:04.244007 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:04.243615    4276 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Oct 31 18:00:04.244007 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:04.243584    4276 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Oct 31 18:00:04.244007 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:04.243644    4276 reflector.go:578] "Warning: watch ended with error" reflector="pkg/kubelet/config/apiserver.go:66" type="*v1.Pod" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Oct 31 18:00:04.244007 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:04.243653    4276 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Oct 31 18:00:04.244007 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:04.243666    4276 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Oct 31 18:00:04.244007 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:04.243668    4276 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Oct 31 18:00:04.244007 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:04.243686    4276 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Oct 31 18:00:04.244007 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:04.243688    4276 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Oct 31 18:00:04.244007 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:04.243704    4276 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-5bhk8\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Oct 31 18:00:04.244007 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:04.243719    4276 reflector.go:578] "Warning: watch ended with error" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Oct 31 18:00:04.244007 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:04.243724    4276 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-zv4z8\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Oct 31 18:00:04.244007 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:04.243741    4276 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Oct 31 18:00:04.244007 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:04.243749    4276 reflector.go:578] "Warning: watch ended with error" reflector="object-\"kube-system\"/\"konnectivity-agent\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Oct 31 18:00:04.244007 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:04.243758    4276 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-6krd7\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Oct 31 18:00:04.244007 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:04.243786    4276 reflector.go:578] "Warning: watch ended with error" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Oct 31 18:00:04.244007 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:04.243790    4276 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-console\"/\"downloads-dockercfg-dd7hr\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Oct 31 18:00:04.244007 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:04.243798    4276 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Oct 31 18:00:04.244007 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:04.243821    4276 reflector.go:578] "Warning: watch ended with error" reflector="object-\"llm\"/\"echo-simulator-kserve-self-signed-certs\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Oct 31 18:00:04.244007 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:04.243827    4276 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Oct 31 18:00:04.244007 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:04.243842    4276 reflector.go:578] "Warning: watch ended with error" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Oct 31 18:00:04.244007 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:04.243866    4276 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Oct 31 18:00:04.244007 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:04.243868    4276 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-service-ca\"/\"signing-key\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Oct 31 18:00:04.244007 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:04.243873    4276 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-8gjvp\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Oct 31 18:00:04.244007 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:04.243894    4276 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Oct 31 18:00:04.244007 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:04.243908    4276 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Oct 31 18:00:04.244007 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:04.243912    4276 reflector.go:578] "Warning: watch ended with error" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Oct 31 18:00:04.244007 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:04.243916    4276 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Oct 31 18:00:04.244007 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:04.243931    4276 reflector.go:578] "Warning: watch ended with error" reflector="object-\"opendatahub-operator-system\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Oct 31 18:00:04.244007 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:04.243935    4276 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-z2hdt\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Oct 31 18:00:04.244007 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:04.243949    4276 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Oct 31 18:00:04.244007 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:04.243976    4276 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-5lqk4\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Oct 31 18:00:04.244007 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:04.243984    4276 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-console\"/\"console-dockercfg-ht46x\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Oct 31 18:00:04.244007 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:04.244003    4276 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Oct 31 18:00:04.244007 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:04.244040    4276 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-x4c5q\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Oct 31 18:00:04.244007 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:04.244048    4276 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-console\"/\"console-config\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Oct 31 18:00:04.246272 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:04.244080    4276 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Oct 31 18:00:04.246272 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:04.244085    4276 reflector.go:578] "Warning: watch ended with error" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-f5lr6\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Oct 31 18:00:04.246272 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:04.244088    4276 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-xbzzg\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Oct 31 18:00:04.246272 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:04.244108    4276 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Oct 31 18:00:04.246272 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:04.244118    4276 reflector.go:578] "Warning: watch ended with error" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-tj66q\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Oct 31 18:00:04.246272 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:04.244130    4276 reflector.go:578] "Warning: watch ended with error" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Oct 31 18:00:04.246272 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:04.244137    4276 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Oct 31 18:00:04.246272 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:04.244146    4276 reflector.go:578] "Warning: watch ended with error" reflector="object-\"opendatahub\"/\"ai-gateway-controller-metrics-tls\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Oct 31 18:00:04.246272 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:04.244158    4276 reflector.go:578] "Warning: watch ended with error" reflector="object-\"opendatahub-operator-system\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Oct 31 18:00:04.246272 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:04.244170    4276 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Oct 31 18:00:04.246272 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:04.244163    4276 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-console\"/\"console-oauth-config\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Oct 31 18:00:04.246272 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:04.244182    4276 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Oct 31 18:00:04.246272 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:04.244185    4276 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Oct 31 18:00:04.246272 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:04.244205    4276 reflector.go:578] "Warning: watch ended with error" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Oct 31 18:00:04.246272 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:04.244216    4276 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Oct 31 18:00:04.246272 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:04.244236    4276 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Oct 31 18:00:04.246272 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:04.244241    4276 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Node" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Oct 31 18:00:04.246272 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:04.244247    4276 reflector.go:578] "Warning: watch ended with error" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Oct 31 18:00:04.246272 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:04.244255    4276 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-dns\"/\"dns-dockercfg-4g54t\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Oct 31 18:00:04.246272 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:04.244258    4276 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Oct 31 18:00:04.246272 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:04.244269    4276 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Oct 31 18:00:04.246272 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:04.244276    4276 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-mcfpr\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Oct 31 18:00:04.246272 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:04.244270    4276 reflector.go:578] "Warning: watch ended with error" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-manager-dockercfg-wsnj7\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Oct 31 18:00:04.246272 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:04.244292    4276 reflector.go:578] "Warning: watch ended with error" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-metrics-tls\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Oct 31 18:00:04.246272 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:04.244296    4276 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-9kmpm\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Oct 31 18:00:04.246272 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:04.244291    4276 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Oct 31 18:00:04.246272 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:04.244305    4276 reflector.go:578] "Warning: watch ended with error" reflector="object-\"opendatahub\"/\"ai-gateway-controller-dockercfg-lwz2q\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Oct 31 18:00:04.246272 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:04.244313    4276 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Oct 31 18:00:04.246272 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:04.244326    4276 reflector.go:578] "Warning: watch ended with error" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Oct 31 18:00:04.246272 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:04.244337    4276 reflector.go:578] "Warning: watch ended with error" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Oct 31 18:00:04.246272 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:04.244344    4276 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Oct 31 18:00:04.246272 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:04.244324    4276 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Oct 31 18:00:04.246272 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:04.244339    4276 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Service" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Oct 31 18:00:04.246272 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:04.244364    4276 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Oct 31 18:00:04.246272 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:04.244363    4276 reflector.go:578] "Warning: watch ended with error" reflector="object-\"opendatahub\"/\"odh-trusted-ca-bundle\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Oct 31 18:00:04.246272 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:04.244364    4276 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Oct 31 18:00:04.246272 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:04.244353    4276 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-dm9tc\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Oct 31 18:00:04.246272 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:04.244389    4276 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-g2t4b\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Oct 31 18:00:04.246272 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:04.244393    4276 reflector.go:578] "Warning: watch ended with error" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Oct 31 18:00:04.246272 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:04.244392    4276 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Oct 31 18:00:04.246272 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:04.244403    4276 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-8d7mp\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Oct 31 18:00:04.246272 ip-10-0-129-127 kubenswrapper[4276]: E1031 18:00:04.244388    4276 controller.go:251] "Failed to update lease" err="Put \"https://a756852771c0044279d6ba9a0e15a6dd-c7c3820165848559.elb.us-east-2.amazonaws.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ip-10-0-129-127.ec2.internal?timeout=10s\": http2: client connection lost"
Oct 31 18:00:04.246272 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:04.244417    4276 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-dhv4b\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Oct 31 18:00:04.246272 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:04.244424    4276 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Oct 31 18:00:04.246272 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:04.244426    4276 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Oct 31 18:00:04.246272 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:04.244457    4276 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Oct 31 18:00:04.246272 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:04.244462    4276 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.RuntimeClass" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Oct 31 18:00:04.246272 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:04.244479    4276 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-9gvm6\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Oct 31 18:00:04.246272 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:04.244483    4276 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Oct 31 18:00:04.246272 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:04.244490    4276 reflector.go:578] "Warning: watch ended with error" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Oct 31 18:00:04.246272 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:04.244504    4276 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Oct 31 18:00:04.246272 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:04.244507    4276 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-vvnlg\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Oct 31 18:00:04.246272 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:04.244513    4276 reflector.go:578] "Warning: watch ended with error" reflector="object-\"llm\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Oct 31 18:00:04.246272 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:04.244510    4276 reflector.go:578] "Warning: watch ended with error" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Oct 31 18:00:04.246272 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:04.244529    4276 reflector.go:578] "Warning: watch ended with error" reflector="object-\"opendatahub\"/\"kserve-module-controller-manager-dockercfg-zm2zr\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Oct 31 18:00:04.246272 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:04.244540    4276 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Oct 31 18:00:04.246272 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:04.244544    4276 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Oct 31 18:00:04.246272 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:04.244539    4276 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-4vqh2\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Oct 31 18:00:04.246272 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:04.244562    4276 reflector.go:578] "Warning: watch ended with error" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Oct 31 18:00:04.246272 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:04.244566    4276 reflector.go:578] "Warning: watch ended with error" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-manager-service-cert\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Oct 31 18:00:04.246272 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:04.244562    4276 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Oct 31 18:00:04.246272 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:04.244576    4276 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Oct 31 18:00:04.246272 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:04.244583    4276 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Oct 31 18:00:04.246272 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:04.244584    4276 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-nk5tf\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Oct 31 18:00:04.246272 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:04.244607    4276 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Oct 31 18:00:04.246272 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:04.244618    4276 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Oct 31 18:00:04.246272 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:04.244622    4276 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Oct 31 18:00:04.246272 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:04.244625    4276 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Oct 31 18:00:04.246272 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:04.244633    4276 reflector.go:578] "Warning: watch ended with error" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-4s8xz\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding"
Oct 31 18:00:05.068965 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:05.068935    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Oct 31 18:00:05.071983 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:05.071965    4276 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Oct 31 18:00:05.078839 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:05.078816    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Oct 31 18:00:05.081763 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:05.081739    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-8d7mp\""
Oct 31 18:00:05.089787 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:05.089766    4276 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Oct 31 18:00:05.089787 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:05.089772    4276 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Oct 31 18:00:05.092107 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:05.092091    4276 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Oct 31 18:00:05.096639 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:05.096624    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-8bpqv\""
Oct 31 18:00:05.101793 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:05.101774    4276 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Oct 31 18:00:05.107036 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:05.107011    4276 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Oct 31 18:00:05.116829 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:05.116811    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Oct 31 18:00:05.124374 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:05.124357    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Oct 31 18:00:05.141191 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:05.141169    4276 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Oct 31 18:00:05.141282 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:05.141250    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-g2t4b\""
Oct 31 18:00:05.151219 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:05.151202    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Oct 31 18:00:05.151321 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:05.151307    4276 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 31 18:00:05.164676 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:05.164654    4276 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Oct 31 18:00:05.164806 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:05.164711    4276 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Oct 31 18:00:05.165082 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:05.165065    4276 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"odh-trusted-ca-bundle\""
Oct 31 18:00:05.171666 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:05.171647    4276 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Oct 31 18:00:05.183194 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:05.183181    4276 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Oct 31 18:00:05.184274 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:05.184257    4276 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Oct 31 18:00:05.184329 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:05.184285    4276 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub-operator-system\"/\"kube-root-ca.crt\""
Oct 31 18:00:05.187662 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:05.187646    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-5lqk4\""
Oct 31 18:00:05.194634 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:05.194619    4276 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 31 18:00:05.224250 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:05.224230    4276 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Oct 31 18:00:05.233437 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:05.233422    4276 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Oct 31 18:00:05.238129 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:05.238113    4276 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Oct 31 18:00:05.245019 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:05.245006    4276 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Oct 31 18:00:05.252840 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:05.252825    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-4g54t\""
Oct 31 18:00:05.260715 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:05.260692    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Oct 31 18:00:05.271267 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:05.271240    4276 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Oct 31 18:00:05.271461 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:05.271430    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-mcfpr\""
Oct 31 18:00:05.274601 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:05.274586    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-z2hdt\""
Oct 31 18:00:05.276584 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:05.276563    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-metrics-tls\""
Oct 31 18:00:05.281149 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:05.281136    4276 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Oct 31 18:00:05.286103 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:05.286088    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Oct 31 18:00:05.288868 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:05.288851    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-module-controller-manager-dockercfg-zm2zr\""
Oct 31 18:00:05.289368 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:05.289355    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"downloads-dockercfg-dd7hr\""
Oct 31 18:00:05.290777 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:05.290761    4276 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Oct 31 18:00:05.290865 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:05.290823    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Oct 31 18:00:05.290865 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:05.290858    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Oct 31 18:00:05.302313 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:05.302293    4276 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Oct 31 18:00:05.315937 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:05.315919    4276 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Oct 31 18:00:05.333671 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:05.333616    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-5bhk8\""
Oct 31 18:00:05.336583 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:05.336568    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-4vqh2\""
Oct 31 18:00:05.349512 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:05.349489    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Oct 31 18:00:05.349512 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:05.349503    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-9kmpm\""
Oct 31 18:00:05.350221 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:05.350209    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-8gjvp\""
Oct 31 18:00:05.353402 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:05.353387    4276 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Oct 31 18:00:05.360323 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:05.360302    4276 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Oct 31 18:00:05.363712 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:05.363696    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Oct 31 18:00:05.367007 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:05.366970    4276 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Oct 31 18:00:05.367107 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:05.367079    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"echo-simulator-kserve-self-signed-certs\""
Oct 31 18:00:05.371859 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:05.371843    4276 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Oct 31 18:00:05.372126 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:05.372109    4276 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Oct 31 18:00:05.374109 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:05.374094    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Oct 31 18:00:05.378605 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:05.378588    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Oct 31 18:00:05.384370 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:05.384354    4276 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Oct 31 18:00:05.387035 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:05.387019    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-9gvm6\""
Oct 31 18:00:05.405700 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:05.405681    4276 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Oct 31 18:00:05.411105 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:05.411088    4276 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 31 18:00:05.412910 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:05.412896    4276 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Oct 31 18:00:05.420201 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:05.420183    4276 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Oct 31 18:00:05.430903 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:05.430886    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-x4c5q\""
Oct 31 18:00:05.432931 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:05.432917    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Oct 31 18:00:05.438103 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:05.438086    4276 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Oct 31 18:00:05.439121 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:05.439107    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"ai-gateway-controller-dockercfg-lwz2q\""
Oct 31 18:00:05.457762 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:05.457743    4276 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Oct 31 18:00:05.462823 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:05.462806    4276 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Oct 31 18:00:05.468078 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:05.468063    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Oct 31 18:00:05.488722 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:05.488702    4276 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Oct 31 18:00:05.499612 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:05.499593    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-4s8xz\""
Oct 31 18:00:05.511304 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:05.511283    4276 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Oct 31 18:00:05.511402 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:05.511388    4276 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Oct 31 18:00:05.522032 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:05.522017    4276 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Oct 31 18:00:05.532450 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:05.532430    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-ht46x\""
Oct 31 18:00:05.545191 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:05.545173    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-vhxrl\""
Oct 31 18:00:05.551401 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:05.551382    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Oct 31 18:00:05.566295 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:05.566276    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-xbzzg\""
Oct 31 18:00:05.569289 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:05.569275    4276 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Oct 31 18:00:05.572323 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:05.572310    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Oct 31 18:00:05.579388 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:05.579374    4276 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Oct 31 18:00:05.589912 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:05.589861    4276 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 31 18:00:05.591097 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:05.591083    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Oct 31 18:00:05.596099 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:05.596085    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Oct 31 18:00:05.601950 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:05.601931    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 18:00:05.602743 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:05.602726    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-vvnlg\""
Oct 31 18:00:05.602817 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:05.602766    4276 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Oct 31 18:00:05.611488 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:05.611474    4276 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Oct 31 18:00:05.613138 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:05.613125    4276 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub-operator-system\"/\"openshift-service-ca.crt\""
Oct 31 18:00:05.629611 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:05.629591    4276 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Oct 31 18:00:05.635967 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:05.635948    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"ai-gateway-controller-metrics-tls\""
Oct 31 18:00:05.644277 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:05.644261    4276 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Oct 31 18:00:05.661026 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:05.661004    4276 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Oct 31 18:00:05.661026 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:05.661021    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-dhv4b\""
Oct 31 18:00:05.662031 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:05.662019    4276 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Oct 31 18:00:05.663100 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:05.663085    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-k6pvq\""
Oct 31 18:00:05.667367 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:05.667352    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-manager-dockercfg-wsnj7\""
Oct 31 18:00:05.667459 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:05.667403    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-tj66q\""
Oct 31 18:00:05.674578 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:05.674562    4276 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Oct 31 18:00:05.690525 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:05.690505    4276 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Oct 31 18:00:05.705219 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:05.705200    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Oct 31 18:00:05.706798 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:05.706782    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-zv4z8\""
Oct 31 18:00:05.712075 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:05.712060    4276 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Oct 31 18:00:05.721427 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:05.721411    4276 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Oct 31 18:00:05.723697 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:05.723680    4276 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Oct 31 18:00:05.724875 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:05.724860    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Oct 31 18:00:05.724948 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:05.724894    4276 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Oct 31 18:00:05.725475 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:05.725464    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Oct 31 18:00:05.731517 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:05.731502    4276 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Oct 31 18:00:05.741260 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:05.741245    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-nk5tf\""
Oct 31 18:00:05.744054 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:05.744039    4276 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Oct 31 18:00:05.745175 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:05.745161    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Oct 31 18:00:05.746211 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:05.746193    4276 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Oct 31 18:00:05.750359 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:05.750346    4276 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Oct 31 18:00:05.756655 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:05.756642    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-manager-service-cert\""
Oct 31 18:00:05.762993 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:05.762977    4276 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 31 18:00:05.773572 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:05.773558    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-f5lr6\""
Oct 31 18:00:05.781156 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:05.781138    4276 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 31 18:00:05.784112 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:05.784099    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-dm9tc\""
Oct 31 18:00:05.787397 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:05.787382    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Oct 31 18:00:05.794858 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:05.794844    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-6krd7\""
Oct 31 18:00:05.801700 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:05.801687    4276 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Oct 31 18:00:05.809805 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:05.809790    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Oct 31 18:00:05.810694 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:05.810683    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-hpj5l\""
Oct 31 18:00:05.831861 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:05.831846    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Oct 31 18:00:05.843882 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:05.843835    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-f4hkh\""
Oct 31 18:00:05.852884 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:05.852870    4276 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Oct 31 18:00:10.463364 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:10.463332    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 18:00:12.233846 ip-10-0-129-127 kubenswrapper[4276]: E1031 18:00:12.233808    4276 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tg5qk" containerName="extractor"
Oct 31 18:00:15.462628 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:15.462598    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 18:00:15.507047 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:15.507002    4276 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_ai-gateway-controller-848fd4df45-tbvzh_9474cc98-e6c9-4c56-ab07-664b1c552440/manager/0.log"
Oct 31 18:00:16.025085 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:16.025057    4276 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-module-controller-manager-7566cf5fc8-mb26r_de946e91-8b1c-42d2-b859-e7d93a6fb7cd/manager/1.log"
Oct 31 18:00:16.480734 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:16.480684    4276 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-58b696d4f-nlh9h_39302afc-f80b-481b-925d-eaeffc127236/manager/0.log"
Oct 31 18:00:17.720383 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:17.720334    4276 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-module-controller-manager-7566cf5fc8-mb26r_de946e91-8b1c-42d2-b859-e7d93a6fb7cd/manager/0.log"
Oct 31 18:00:17.836014 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:17.835984    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 18:00:18.459146 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:18.459098    4276 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-f5575fb46-29pdk_cf94ce76-24ab-4767-b43d-3abfff4688f8/maas-api/0.log"
Oct 31 18:00:24.592735 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:24.592704    4276 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-77485b4dcc-7nnp2_fc3fdf2a-ea8b-4a6a-a63b-aa5bd7f77486/kube-auth-proxy/0.log"
Oct 31 18:00:25.121097 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:25.121070    4276 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-7d9897b54c-98fcj_9421021e-be39-4906-8fae-da1ce9ba4047/istio-proxy/0.log"
Oct 31 18:00:25.502233 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:25.502151    4276 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-7d9897b54c-b8gmm_5530ee8f-be36-43d3-a268-051794343d8b/istio-proxy/0.log"
Oct 31 18:00:26.048705 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:26.048671    4276 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-7d9897b54c-p4v97_3ffeb078-cb4d-4293-b7fb-eb6fd305d96b/istio-proxy/0.log"
Oct 31 18:00:26.238419 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:26.238384    4276 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-7d9897b54c-pfv5k_a0744b4f-5676-4385-b50c-5796ca2365a0/istio-proxy/0.log"
Oct 31 18:00:26.805837 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:26.805803    4276 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7b98bb678c-w9z9b_8d0cf02b-9581-4727-aac8-4047b97998c1/payload-pre-processing/0.log"
Oct 31 18:00:26.996091 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:26.996052    4276 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-55fbbbf9dc-lbpsv_d8705881-7dcb-40a0-b599-e14596d7697c/payload-processing/0.log"
Oct 31 18:00:28.086912 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:28.086885    4276 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-cbd4f5586-bzprg_8f8e3a21-f02f-4bcd-94f5-587416670156/main/0.log"
Oct 31 18:00:28.266763 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:28.266724    4276 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-7b998b79d9-mpnd7_1800b0cf-d90b-455f-a6d3-ece0eea69cec/main/0.log"
Oct 31 18:00:28.637375 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:28.637347    4276 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-69b588897d-ctkqw_cfd9048c-8ac0-4f5a-abec-71f640158e61/main/0.log"
Oct 31 18:00:28.985175 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:28.985096    4276 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_echo-simulator-kserve-64b6f8f96c-68q2x_bd5cf2da-6510-434d-81f9-579baecddaf3/main/0.log"
Oct 31 18:00:38.981679 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:38.981644    4276 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_ai-gateway-controller-848fd4df45-tbvzh_9474cc98-e6c9-4c56-ab07-664b1c552440/manager/0.log"
Oct 31 18:00:39.376679 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:39.376648    4276 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-module-controller-manager-7566cf5fc8-mb26r_de946e91-8b1c-42d2-b859-e7d93a6fb7cd/manager/1.log"
Oct 31 18:00:39.814895 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:39.814841    4276 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-58b696d4f-nlh9h_39302afc-f80b-481b-925d-eaeffc127236/manager/0.log"
Oct 31 18:00:40.973512 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:40.973488    4276 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-f5575fb46-29pdk_cf94ce76-24ab-4767-b43d-3abfff4688f8/maas-api/0.log"
Oct 31 18:00:42.233558 ip-10-0-129-127 kubenswrapper[4276]: E1031 18:00:42.233517    4276 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tg5qk" containerName="extractor"
Oct 31 18:00:45.492147 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:45.492114    4276 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-77485b4dcc-7nnp2_fc3fdf2a-ea8b-4a6a-a63b-aa5bd7f77486/kube-auth-proxy/0.log"
Oct 31 18:00:46.069117 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:46.069068    4276 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-7d9897b54c-98fcj_9421021e-be39-4906-8fae-da1ce9ba4047/istio-proxy/0.log"
Oct 31 18:00:46.445123 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:46.445041    4276 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-7d9897b54c-b8gmm_5530ee8f-be36-43d3-a268-051794343d8b/istio-proxy/0.log"
Oct 31 18:00:47.026757 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:47.026727    4276 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-7d9897b54c-p4v97_3ffeb078-cb4d-4293-b7fb-eb6fd305d96b/istio-proxy/0.log"
Oct 31 18:00:47.216690 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:47.216664    4276 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-7d9897b54c-pfv5k_a0744b4f-5676-4385-b50c-5796ca2365a0/istio-proxy/0.log"
Oct 31 18:00:47.765106 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:47.765075    4276 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7b98bb678c-w9z9b_8d0cf02b-9581-4727-aac8-4047b97998c1/payload-pre-processing/0.log"
Oct 31 18:00:47.937437 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:47.937409    4276 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-55fbbbf9dc-lbpsv_d8705881-7dcb-40a0-b599-e14596d7697c/payload-processing/0.log"
Oct 31 18:00:48.636540 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:48.636516    4276 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-cbd4f5586-bzprg_8f8e3a21-f02f-4bcd-94f5-587416670156/main/0.log"
Oct 31 18:00:48.821982 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:48.821957    4276 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-7b998b79d9-mpnd7_1800b0cf-d90b-455f-a6d3-ece0eea69cec/main/0.log"
Oct 31 18:00:49.188897 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:49.188863    4276 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-69b588897d-ctkqw_cfd9048c-8ac0-4f5a-abec-71f640158e61/main/0.log"
Oct 31 18:00:49.573846 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:49.573817    4276 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_echo-simulator-kserve-64b6f8f96c-68q2x_bd5cf2da-6510-434d-81f9-579baecddaf3/main/0.log"
Oct 31 18:00:50.790352 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:50.790316    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 18:00:58.327603 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:00:58.327555    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 18:01:05.681793 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:01:05.681693    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 18:01:10.623594 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:01:10.623555    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 18:01:12.233938 ip-10-0-129-127 kubenswrapper[4276]: E1031 18:01:12.233894    4276 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tg5qk" containerName="extractor"
Oct 31 18:01:15.595364 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:01:15.595321    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 18:01:17.049794 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:01:17.049754    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 18:01:17.167743 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:01:17.167713    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 18:01:24.841154 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:01:24.841118    4276 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-58b696d4f-nlh9h_39302afc-f80b-481b-925d-eaeffc127236/manager/0.log"
Oct 31 18:01:25.128062 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:01:25.127980    4276 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-58b696d4f-nlh9h_39302afc-f80b-481b-925d-eaeffc127236/manager/0.log"
Oct 31 18:01:25.816007 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:01:25.815977    4276 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-55fbbbf9dc-lbpsv_d8705881-7dcb-40a0-b599-e14596d7697c/payload-processing/0.log"
Oct 31 18:01:25.988107 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:01:25.988078    4276 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-55fbbbf9dc-lbpsv_d8705881-7dcb-40a0-b599-e14596d7697c/payload-processing/0.log"
Oct 31 18:01:42.234532 ip-10-0-129-127 kubenswrapper[4276]: E1031 18:01:42.234489    4276 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tg5qk" containerName="extractor"
Oct 31 18:02:05.782658 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:02:05.782623    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 18:02:08.407305 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:02:08.407275    4276 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-58b696d4f-nlh9h_39302afc-f80b-481b-925d-eaeffc127236/manager/0.log"
Oct 31 18:02:08.699905 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:02:08.699832    4276 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-58b696d4f-nlh9h_39302afc-f80b-481b-925d-eaeffc127236/manager/0.log"
Oct 31 18:02:09.392586 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:02:09.392553    4276 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-55fbbbf9dc-lbpsv_d8705881-7dcb-40a0-b599-e14596d7697c/payload-processing/0.log"
Oct 31 18:02:09.579264 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:02:09.579237    4276 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-55fbbbf9dc-lbpsv_d8705881-7dcb-40a0-b599-e14596d7697c/payload-processing/0.log"
Oct 31 18:02:09.911628 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:02:09.911595    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 18:02:09.997723 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:02:09.997696    4276 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-must-gather-csf86/must-gather-g856m"]
Oct 31 18:02:09.998049 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:02:09.998037    4276 cpu_manager.go:414] "containerMap: removing container" podUID="f9d2226c-82f2-48cb-b1cc-1ee87b3f323d" containerName="payload-processing"
Oct 31 18:02:09.998104 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:02:09.998050    4276 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f9d2226c-82f2-48cb-b1cc-1ee87b3f323d" containerName="payload-processing"
Oct 31 18:02:09.998104 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:02:09.998065    4276 cpu_manager.go:414] "containerMap: removing container" podUID="4c179233-456f-488d-ba8f-bb9ec7d28fd5" containerName="istio-proxy"
Oct 31 18:02:09.998104 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:02:09.998070    4276 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4c179233-456f-488d-ba8f-bb9ec7d28fd5" containerName="istio-proxy"
Oct 31 18:02:09.998104 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:02:09.998083    4276 cpu_manager.go:414] "containerMap: removing container" podUID="4c179233-456f-488d-ba8f-bb9ec7d28fd5" containerName="istio-proxy"
Oct 31 18:02:09.998104 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:02:09.998088    4276 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4c179233-456f-488d-ba8f-bb9ec7d28fd5" containerName="istio-proxy"
Oct 31 18:02:09.998104 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:02:09.998094    4276 cpu_manager.go:414] "containerMap: removing container" podUID="8c75f040-baeb-4bba-9497-6b70b3288b83" containerName="maas-api"
Oct 31 18:02:09.998104 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:02:09.998099    4276 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8c75f040-baeb-4bba-9497-6b70b3288b83" containerName="maas-api"
Oct 31 18:02:09.998104 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:02:09.998105    4276 cpu_manager.go:414] "containerMap: removing container" podUID="453dd8b4-8c09-4c76-82e8-6891e9609684" containerName="payload-pre-processing"
Oct 31 18:02:09.998398 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:02:09.998110    4276 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="453dd8b4-8c09-4c76-82e8-6891e9609684" containerName="payload-pre-processing"
Oct 31 18:02:09.998398 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:02:09.998179    4276 memory_manager.go:365] "RemoveStaleState removing state" podUID="f9d2226c-82f2-48cb-b1cc-1ee87b3f323d" containerName="payload-processing"
Oct 31 18:02:09.998398 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:02:09.998186    4276 memory_manager.go:365] "RemoveStaleState removing state" podUID="4c179233-456f-488d-ba8f-bb9ec7d28fd5" containerName="istio-proxy"
Oct 31 18:02:09.998398 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:02:09.998194    4276 memory_manager.go:365] "RemoveStaleState removing state" podUID="4c179233-456f-488d-ba8f-bb9ec7d28fd5" containerName="istio-proxy"
Oct 31 18:02:09.998398 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:02:09.998199    4276 memory_manager.go:365] "RemoveStaleState removing state" podUID="8c75f040-baeb-4bba-9497-6b70b3288b83" containerName="maas-api"
Oct 31 18:02:09.998398 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:02:09.998204    4276 memory_manager.go:365] "RemoveStaleState removing state" podUID="453dd8b4-8c09-4c76-82e8-6891e9609684" containerName="payload-pre-processing"
Oct 31 18:02:10.001111 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:02:10.001096    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-csf86/must-gather-g856m"
Oct 31 18:02:10.013195 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:02:10.013175    4276 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-csf86\"/\"kube-root-ca.crt\""
Oct 31 18:02:10.022793 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:02:10.022774    4276 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-csf86\"/\"openshift-service-ca.crt\""
Oct 31 18:02:10.022889 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:02:10.022821    4276 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-csf86\"/\"default-dockercfg-5qfvc\""
Oct 31 18:02:10.026065 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:02:10.026045    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-csf86/must-gather-g856m"]
Oct 31 18:02:10.037493 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:02:10.037467    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 18:02:10.115414 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:02:10.115387    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nxhd8\" (UniqueName: \"kubernetes.io/projected/203847cd-cacb-4011-9ab5-b8f1536957b1-kube-api-access-nxhd8\") pod \"must-gather-g856m\" (UID: \"203847cd-cacb-4011-9ab5-b8f1536957b1\") " pod="openshift-must-gather-csf86/must-gather-g856m"
Oct 31 18:02:10.115554 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:02:10.115462    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/203847cd-cacb-4011-9ab5-b8f1536957b1-must-gather-output\") pod \"must-gather-g856m\" (UID: \"203847cd-cacb-4011-9ab5-b8f1536957b1\") " pod="openshift-must-gather-csf86/must-gather-g856m"
Oct 31 18:02:10.216005 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:02:10.215936    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nxhd8\" (UniqueName: \"kubernetes.io/projected/203847cd-cacb-4011-9ab5-b8f1536957b1-kube-api-access-nxhd8\") pod \"must-gather-g856m\" (UID: \"203847cd-cacb-4011-9ab5-b8f1536957b1\") " pod="openshift-must-gather-csf86/must-gather-g856m"
Oct 31 18:02:10.216115 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:02:10.216019    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/203847cd-cacb-4011-9ab5-b8f1536957b1-must-gather-output\") pod \"must-gather-g856m\" (UID: \"203847cd-cacb-4011-9ab5-b8f1536957b1\") " pod="openshift-must-gather-csf86/must-gather-g856m"
Oct 31 18:02:10.216347 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:02:10.216329    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/203847cd-cacb-4011-9ab5-b8f1536957b1-must-gather-output\") pod \"must-gather-g856m\" (UID: \"203847cd-cacb-4011-9ab5-b8f1536957b1\") " pod="openshift-must-gather-csf86/must-gather-g856m"
Oct 31 18:02:10.231804 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:02:10.231779    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nxhd8\" (UniqueName: \"kubernetes.io/projected/203847cd-cacb-4011-9ab5-b8f1536957b1-kube-api-access-nxhd8\") pod \"must-gather-g856m\" (UID: \"203847cd-cacb-4011-9ab5-b8f1536957b1\") " pod="openshift-must-gather-csf86/must-gather-g856m"
Oct 31 18:02:10.313965 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:02:10.313942    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-csf86/must-gather-g856m"
Oct 31 18:02:10.608614 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:02:10.608586    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-csf86/must-gather-g856m"]
Oct 31 18:02:10.611746 ip-10-0-129-127 kubenswrapper[4276]: W1031 18:02:10.611716    4276 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod203847cd_cacb_4011_9ab5_b8f1536957b1.slice/crio-b4b45464dc1f4a6a362bc8a4425ca8681e10a6817fab7fa77a5be87bc7ef44e0 WatchSource:0}: Error finding container b4b45464dc1f4a6a362bc8a4425ca8681e10a6817fab7fa77a5be87bc7ef44e0: Status 404 returned error can't find the container with id b4b45464dc1f4a6a362bc8a4425ca8681e10a6817fab7fa77a5be87bc7ef44e0
Oct 31 18:02:10.613681 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:02:10.613663    4276 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 31 18:02:10.685517 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:02:10.685485    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 18:02:11.278461 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:02:11.278415    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-csf86/must-gather-g856m" event={"ID":"203847cd-cacb-4011-9ab5-b8f1536957b1","Type":"ContainerStarted","Data":"b4b45464dc1f4a6a362bc8a4425ca8681e10a6817fab7fa77a5be87bc7ef44e0"}
Oct 31 18:02:12.235972 ip-10-0-129-127 kubenswrapper[4276]: E1031 18:02:12.235934    4276 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tg5qk" containerName="extractor"
Oct 31 18:02:12.284935 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:02:12.284903    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-csf86/must-gather-g856m" event={"ID":"203847cd-cacb-4011-9ab5-b8f1536957b1","Type":"ContainerStarted","Data":"98187acf923ee967e7ffb51476d621310a5b6674ea59735fc27ac2b9145ffc4e"}
Oct 31 18:02:12.285107 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:02:12.284946    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-csf86/must-gather-g856m" event={"ID":"203847cd-cacb-4011-9ab5-b8f1536957b1","Type":"ContainerStarted","Data":"dbb699808cc46b459385dfe0d0ccb1b4aa7cf92e058b5973ffcac94ae96cdebf"}
Oct 31 18:02:12.316542 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:02:12.316494    4276 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-must-gather-csf86/must-gather-g856m" podStartSLOduration=2.513647549 podStartE2EDuration="3.316480313s" totalImagesPullingTime="802.832764ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 18:02:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 18:02:12.316272521 +0000 UTC m=+2095.114017449" watchObservedRunningTime="2026-10-31 18:02:12.316480313 +0000 UTC m=+2095.114225240"
Oct 31 18:02:14.326325 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:02:14.326288    4276 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-kmkdh_6d7781d6-8f73-4f1d-bb9a-0212e2eb6e4b/global-pull-secret-syncer/0.log"
Oct 31 18:02:15.252398 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:02:15.250589    4276 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log"
Oct 31 18:02:15.607885 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:02:15.607858    4276 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-129-127.ec2.internal_5136faca2de3de7a06b0d414d9c6fca7/haproxy/0.log"
Oct 31 18:02:15.677907 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:02:15.677877    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 18:02:17.766460 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:02:17.765521    4276 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zpf4d_5819fd64-9ac0-4436-b8fe-dcd3a27827ac/ovn-acl-logging/0.log"
Oct 31 18:02:17.792463 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:02:17.783318    4276 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zpf4d_5819fd64-9ac0-4436-b8fe-dcd3a27827ac/ovn-acl-logging/0.log"
Oct 31 18:02:24.346528 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:02:24.346489    4276 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-must-gather-csf86/perf-node-gather-daemonset-546lh"]
Oct 31 18:02:24.356371 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:02:24.356343    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-csf86/perf-node-gather-daemonset-546lh"
Oct 31 18:02:24.376471 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:02:24.376418    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-csf86/perf-node-gather-daemonset-546lh"]
Oct 31 18:02:24.424106 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:02:24.424067    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/52226300-30bb-4590-a606-d8a1a19ebbad-lib-modules\") pod \"perf-node-gather-daemonset-546lh\" (UID: \"52226300-30bb-4590-a606-d8a1a19ebbad\") " pod="openshift-must-gather-csf86/perf-node-gather-daemonset-546lh"
Oct 31 18:02:24.424295 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:02:24.424117    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/52226300-30bb-4590-a606-d8a1a19ebbad-sys\") pod \"perf-node-gather-daemonset-546lh\" (UID: \"52226300-30bb-4590-a606-d8a1a19ebbad\") " pod="openshift-must-gather-csf86/perf-node-gather-daemonset-546lh"
Oct 31 18:02:24.424295 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:02:24.424146    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lxjtg\" (UniqueName: \"kubernetes.io/projected/52226300-30bb-4590-a606-d8a1a19ebbad-kube-api-access-lxjtg\") pod \"perf-node-gather-daemonset-546lh\" (UID: \"52226300-30bb-4590-a606-d8a1a19ebbad\") " pod="openshift-must-gather-csf86/perf-node-gather-daemonset-546lh"
Oct 31 18:02:24.424295 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:02:24.424199    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/52226300-30bb-4590-a606-d8a1a19ebbad-podres\") pod \"perf-node-gather-daemonset-546lh\" (UID: \"52226300-30bb-4590-a606-d8a1a19ebbad\") " pod="openshift-must-gather-csf86/perf-node-gather-daemonset-546lh"
Oct 31 18:02:24.424295 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:02:24.424253    4276 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/52226300-30bb-4590-a606-d8a1a19ebbad-proc\") pod \"perf-node-gather-daemonset-546lh\" (UID: \"52226300-30bb-4590-a606-d8a1a19ebbad\") " pod="openshift-must-gather-csf86/perf-node-gather-daemonset-546lh"
Oct 31 18:02:24.525389 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:02:24.525347    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/52226300-30bb-4590-a606-d8a1a19ebbad-lib-modules\") pod \"perf-node-gather-daemonset-546lh\" (UID: \"52226300-30bb-4590-a606-d8a1a19ebbad\") " pod="openshift-must-gather-csf86/perf-node-gather-daemonset-546lh"
Oct 31 18:02:24.525594 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:02:24.525401    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/52226300-30bb-4590-a606-d8a1a19ebbad-sys\") pod \"perf-node-gather-daemonset-546lh\" (UID: \"52226300-30bb-4590-a606-d8a1a19ebbad\") " pod="openshift-must-gather-csf86/perf-node-gather-daemonset-546lh"
Oct 31 18:02:24.525594 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:02:24.525434    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lxjtg\" (UniqueName: \"kubernetes.io/projected/52226300-30bb-4590-a606-d8a1a19ebbad-kube-api-access-lxjtg\") pod \"perf-node-gather-daemonset-546lh\" (UID: \"52226300-30bb-4590-a606-d8a1a19ebbad\") " pod="openshift-must-gather-csf86/perf-node-gather-daemonset-546lh"
Oct 31 18:02:24.525594 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:02:24.525507    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/52226300-30bb-4590-a606-d8a1a19ebbad-podres\") pod \"perf-node-gather-daemonset-546lh\" (UID: \"52226300-30bb-4590-a606-d8a1a19ebbad\") " pod="openshift-must-gather-csf86/perf-node-gather-daemonset-546lh"
Oct 31 18:02:24.525594 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:02:24.525543    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/52226300-30bb-4590-a606-d8a1a19ebbad-sys\") pod \"perf-node-gather-daemonset-546lh\" (UID: \"52226300-30bb-4590-a606-d8a1a19ebbad\") " pod="openshift-must-gather-csf86/perf-node-gather-daemonset-546lh"
Oct 31 18:02:24.525594 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:02:24.525560    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/52226300-30bb-4590-a606-d8a1a19ebbad-lib-modules\") pod \"perf-node-gather-daemonset-546lh\" (UID: \"52226300-30bb-4590-a606-d8a1a19ebbad\") " pod="openshift-must-gather-csf86/perf-node-gather-daemonset-546lh"
Oct 31 18:02:24.525594 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:02:24.525565    4276 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/52226300-30bb-4590-a606-d8a1a19ebbad-proc\") pod \"perf-node-gather-daemonset-546lh\" (UID: \"52226300-30bb-4590-a606-d8a1a19ebbad\") " pod="openshift-must-gather-csf86/perf-node-gather-daemonset-546lh"
Oct 31 18:02:24.525938 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:02:24.525618    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/52226300-30bb-4590-a606-d8a1a19ebbad-proc\") pod \"perf-node-gather-daemonset-546lh\" (UID: \"52226300-30bb-4590-a606-d8a1a19ebbad\") " pod="openshift-must-gather-csf86/perf-node-gather-daemonset-546lh"
Oct 31 18:02:24.525938 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:02:24.525666    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/52226300-30bb-4590-a606-d8a1a19ebbad-podres\") pod \"perf-node-gather-daemonset-546lh\" (UID: \"52226300-30bb-4590-a606-d8a1a19ebbad\") " pod="openshift-must-gather-csf86/perf-node-gather-daemonset-546lh"
Oct 31 18:02:24.541524 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:02:24.541496    4276 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lxjtg\" (UniqueName: \"kubernetes.io/projected/52226300-30bb-4590-a606-d8a1a19ebbad-kube-api-access-lxjtg\") pod \"perf-node-gather-daemonset-546lh\" (UID: \"52226300-30bb-4590-a606-d8a1a19ebbad\") " pod="openshift-must-gather-csf86/perf-node-gather-daemonset-546lh"
Oct 31 18:02:24.684855 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:02:24.684769    4276 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-csf86/perf-node-gather-daemonset-546lh"
Oct 31 18:02:24.860290 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:02:24.860222    4276 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mnbmt_2f646206-1186-4fcf-a9ca-fe92371d4d2b/konnectivity-agent/0.log" limit=1024
Oct 31 18:02:24.994150 ip-10-0-129-127 kubenswrapper[4276]: W1031 18:02:24.994119    4276 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod52226300_30bb_4590_a606_d8a1a19ebbad.slice/crio-66fa7af3e6fb7c06ffa400a013a1d4e909e20310f3c00b3732b7497e6254cadf WatchSource:0}: Error finding container 66fa7af3e6fb7c06ffa400a013a1d4e909e20310f3c00b3732b7497e6254cadf: Status 404 returned error can't find the container with id 66fa7af3e6fb7c06ffa400a013a1d4e909e20310f3c00b3732b7497e6254cadf
Oct 31 18:02:25.063718 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:02:25.063653    4276 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-csf86/perf-node-gather-daemonset-546lh"]
Oct 31 18:02:25.391348 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:02:25.391306    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-csf86/perf-node-gather-daemonset-546lh" event={"ID":"52226300-30bb-4590-a606-d8a1a19ebbad","Type":"ContainerStarted","Data":"fd9df0f248a17c357ffbdf9f485f6f95fa822e808bbeeda236ca91ba8719f5fe"}
Oct 31 18:02:25.391869 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:02:25.391369    4276 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-csf86/perf-node-gather-daemonset-546lh" event={"ID":"52226300-30bb-4590-a606-d8a1a19ebbad","Type":"ContainerStarted","Data":"66fa7af3e6fb7c06ffa400a013a1d4e909e20310f3c00b3732b7497e6254cadf"}
Oct 31 18:02:25.391869 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:02:25.391479    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-csf86/perf-node-gather-daemonset-546lh"
Oct 31 18:02:25.426329 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:02:25.426277    4276 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-must-gather-csf86/perf-node-gather-daemonset-546lh" podStartSLOduration=1.426262756 podStartE2EDuration="1.426262756s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-31 18:02:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 18:02:25.423629103 +0000 UTC m=+2108.221374028" watchObservedRunningTime="2026-10-31 18:02:25.426262756 +0000 UTC m=+2108.224007685"
Oct 31 18:02:26.454304 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:02:26.454275    4276 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-mlwjd_2ca5233b-53f1-4251-b43f-fc9c0a14747a/node-exporter/0.log"
Oct 31 18:02:26.571870 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:02:26.571828    4276 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-mlwjd_2ca5233b-53f1-4251-b43f-fc9c0a14747a/kube-rbac-proxy/0.log"
Oct 31 18:02:26.689570 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:02:26.689543    4276 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-mlwjd_2ca5233b-53f1-4251-b43f-fc9c0a14747a/init-textfile/0.log"
Oct 31 18:02:28.369120 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:02:28.369089    4276 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-9cd89b74c-kfd5z_54a68758-d202-43a5-b6e4-cb4287494abf/prometheus-operator/0.log"
Oct 31 18:02:28.485057 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:02:28.485031    4276 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-9cd89b74c-kfd5z_54a68758-d202-43a5-b6e4-cb4287494abf/kube-rbac-proxy/0.log"
Oct 31 18:02:28.604746 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:02:28.604719    4276 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-6489d8bf89-v54cn_45619685-5030-4e78-8c16-09f401ca1692/prometheus-operator-admission-webhook/0.log"
Oct 31 18:02:31.408269 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:02:31.408186    4276 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-csf86/perf-node-gather-daemonset-546lh"
Oct 31 18:02:33.073834 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:02:33.073801    4276 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-dbf8bf5bf-j2wb6_20547067-6a08-4783-84e1-de23febc6893/console/0.log"
Oct 31 18:02:33.202599 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:02:33.202571    4276 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-6dd789c8b7-zgfbl_d7460c3b-edb4-40ce-9b5f-89e46950a7c0/download-server/0.log"
Oct 31 18:02:37.009486 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:02:37.008572    4276 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-59fpc_9ee82620-6497-48e8-bb2c-1bb77a93f7ef/dns/0.log"
Oct 31 18:02:37.125562 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:02:37.125537    4276 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-59fpc_9ee82620-6497-48e8-bb2c-1bb77a93f7ef/kube-rbac-proxy/0.log"
Oct 31 18:02:37.611118 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:02:37.611086    4276 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-7pfr4_b8bbaf33-7cd9-4a30-86f7-6e20ac45b0a0/dns-node-resolver/0.log"
Oct 31 18:02:37.961015 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:02:37.960946    4276 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-77485b4dcc-7nnp2_fc3fdf2a-ea8b-4a6a-a63b-aa5bd7f77486/kube-auth-proxy/0.log"
Oct 31 18:02:38.134693 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:02:38.134656    4276 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-7d9897b54c-98fcj_9421021e-be39-4906-8fae-da1ce9ba4047/istio-proxy/0.log"
Oct 31 18:02:38.265080 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:02:38.265009    4276 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-7d9897b54c-b8gmm_5530ee8f-be36-43d3-a268-051794343d8b/istio-proxy/0.log"
Oct 31 18:02:38.466018 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:02:38.465992    4276 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-7d9897b54c-p4v97_3ffeb078-cb4d-4293-b7fb-eb6fd305d96b/istio-proxy/0.log"
Oct 31 18:02:38.533747 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:02:38.533670    4276 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-7d9897b54c-pfv5k_a0744b4f-5676-4385-b50c-5796ca2365a0/istio-proxy/0.log"
Oct 31 18:02:38.722461 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:02:38.722413    4276 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7b98bb678c-w9z9b_8d0cf02b-9581-4727-aac8-4047b97998c1/payload-pre-processing/0.log"
Oct 31 18:02:38.767978 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:02:38.767930    4276 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-865894c87d-kktxm_765a023c-9406-4774-95f5-5af640086496/registry/0.log"
Oct 31 18:02:38.785220 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:02:38.785167    4276 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-55fbbbf9dc-lbpsv_d8705881-7dcb-40a0-b599-e14596d7697c/payload-processing/0.log"
Oct 31 18:02:39.125438 ip-10-0-129-127 kubenswrapper[4276]: I1031 18:02:39.125411    4276 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-qxnvq_87e61dcd-75f6-4d3f-8ed9-7d35b06a9bb3/node-ca/0.log"