Jul 28 23:19:58.499347 ip-10-0-131-112 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 28 23:19:58.499358 ip-10-0-131-112 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 28 23:19:58.499365 ip-10-0-131-112 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 28 23:19:58.499598 ip-10-0-131-112 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 28 23:20:08.625988 ip-10-0-131-112 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 28 23:20:08.626005 ip-10-0-131-112 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot e13ed8bd940d4e77afca7f0b75072fd8 --
Jul 28 23:22:01.226358 ip-10-0-131-112 systemd[1]: Starting Kubernetes Kubelet...
Jul 28 23:22:01.610402 ip-10-0-131-112 kubenswrapper[2572]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 28 23:22:01.610402 ip-10-0-131-112 kubenswrapper[2572]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 28 23:22:01.610402 ip-10-0-131-112 kubenswrapper[2572]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 28 23:22:01.610402 ip-10-0-131-112 kubenswrapper[2572]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 28 23:22:01.610402 ip-10-0-131-112 kubenswrapper[2572]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 28 23:22:01.611117 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.610468    2572 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 28 23:22:01.615717 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.615700    2572 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 23:22:01.615717 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.615716    2572 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 23:22:01.615717 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.615720    2572 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 23:22:01.615811 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.615723    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 23:22:01.615811 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.615726    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 23:22:01.615811 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.615733    2572 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 23:22:01.615811 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.615736    2572 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 23:22:01.615811 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.615739    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 23:22:01.615811 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.615741    2572 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 23:22:01.615811 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.615744    2572 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 23:22:01.615811 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.615746    2572 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 23:22:01.615811 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.615749    2572 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 23:22:01.615811 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.615751    2572 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 23:22:01.615811 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.615754    2572 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 23:22:01.615811 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.615759    2572 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 23:22:01.615811 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.615763    2572 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 23:22:01.615811 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.615766    2572 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 23:22:01.615811 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.615768    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 23:22:01.615811 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.615771    2572 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 23:22:01.615811 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.615774    2572 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 23:22:01.615811 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.615777    2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 23:22:01.615811 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.615780    2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 23:22:01.616287 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.615782    2572 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 23:22:01.616287 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.615785    2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 23:22:01.616287 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.615787    2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 23:22:01.616287 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.615790    2572 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 23:22:01.616287 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.615792    2572 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 23:22:01.616287 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.615794    2572 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 23:22:01.616287 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.615797    2572 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 23:22:01.616287 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.615799    2572 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 23:22:01.616287 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.615801    2572 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 23:22:01.616287 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.615804    2572 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 23:22:01.616287 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.615806    2572 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 23:22:01.616287 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.615809    2572 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 23:22:01.616287 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.615811    2572 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 23:22:01.616287 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.615814    2572 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 23:22:01.616287 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.615817    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 23:22:01.616287 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.615820    2572 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 23:22:01.616287 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.615824    2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 23:22:01.616287 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.615827    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 23:22:01.616287 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.615829    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 23:22:01.616287 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.615832    2572 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 23:22:01.616776 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.615834    2572 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 23:22:01.616776 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.615837    2572 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 23:22:01.616776 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.615839    2572 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 23:22:01.616776 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.615842    2572 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 23:22:01.616776 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.615845    2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 23:22:01.616776 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.615847    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 23:22:01.616776 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.615849    2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 23:22:01.616776 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.615852    2572 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 23:22:01.616776 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.615854    2572 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 23:22:01.616776 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.615857    2572 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 23:22:01.616776 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.615859    2572 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 23:22:01.616776 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.615862    2572 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 23:22:01.616776 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.615864    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 23:22:01.616776 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.615868    2572 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 23:22:01.616776 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.615871    2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 23:22:01.616776 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.615874    2572 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 23:22:01.616776 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.615878    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 23:22:01.616776 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.615880    2572 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 23:22:01.616776 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.615883    2572 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 23:22:01.616776 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.615886    2572 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 23:22:01.617271 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.615888    2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 23:22:01.617271 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.615891    2572 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 23:22:01.617271 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.615894    2572 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 23:22:01.617271 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.615896    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 23:22:01.617271 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.615899    2572 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 23:22:01.617271 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.615901    2572 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 23:22:01.617271 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.615903    2572 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 23:22:01.617271 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.615906    2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 23:22:01.617271 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.615908    2572 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 23:22:01.617271 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.615911    2572 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 23:22:01.617271 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.615914    2572 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 23:22:01.617271 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.615918    2572 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 23:22:01.617271 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.615923    2572 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 23:22:01.617271 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.615926    2572 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 23:22:01.617271 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.615929    2572 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 23:22:01.617271 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.615933    2572 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 23:22:01.617271 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.615937    2572 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 23:22:01.617271 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.615941    2572 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 23:22:01.617271 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.615944    2572 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 23:22:01.617735 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.615946    2572 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 23:22:01.617735 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.615950    2572 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 23:22:01.617735 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.615953    2572 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 23:22:01.617735 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.615956    2572 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 23:22:01.617735 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.615959    2572 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 23:22:01.617735 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.615962    2572 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 23:22:01.617735 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.615965    2572 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 23:22:01.617735 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.615968    2572 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 23:22:01.617735 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.615971    2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 23:22:01.617735 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.615974    2572 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 23:22:01.617735 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.615976    2572 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 23:22:01.617735 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.615979    2572 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 23:22:01.617735 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.615982    2572 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 23:22:01.617735 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.615984    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 23:22:01.617735 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.615988    2572 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 23:22:01.617735 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.615991    2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 23:22:01.617735 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.615993    2572 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 23:22:01.617735 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.615996    2572 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 23:22:01.617735 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.616709    2572 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 23:22:01.617735 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.616717    2572 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 23:22:01.618243 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.616720    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 23:22:01.618243 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.616723    2572 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 23:22:01.618243 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.616726    2572 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 23:22:01.618243 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.616730    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 23:22:01.618243 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.616733    2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 23:22:01.618243 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.616735    2572 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 23:22:01.618243 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.616739    2572 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 23:22:01.618243 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.616741    2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 23:22:01.618243 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.616744    2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 23:22:01.618243 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.616747    2572 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 23:22:01.618243 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.616750    2572 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 23:22:01.618243 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.616752    2572 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 23:22:01.618243 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.616754    2572 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 23:22:01.618243 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.616758    2572 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 23:22:01.618243 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.616760    2572 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 23:22:01.618243 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.616763    2572 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 23:22:01.618243 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.616765    2572 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 23:22:01.618243 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.616768    2572 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 23:22:01.618243 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.616771    2572 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 23:22:01.618243 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.616774    2572 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 23:22:01.618752 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.616777    2572 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 23:22:01.618752 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.616780    2572 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 23:22:01.618752 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.616783    2572 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 23:22:01.618752 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.616786    2572 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 23:22:01.618752 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.616788    2572 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 23:22:01.618752 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.616791    2572 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 23:22:01.618752 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.616794    2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 23:22:01.618752 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.616796    2572 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 23:22:01.618752 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.616799    2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 23:22:01.618752 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.616801    2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 23:22:01.618752 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.616804    2572 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 23:22:01.618752 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.616807    2572 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 23:22:01.618752 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.616810    2572 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 23:22:01.618752 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.616812    2572 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 23:22:01.618752 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.616815    2572 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 23:22:01.618752 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.616817    2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 23:22:01.618752 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.616821    2572 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 23:22:01.618752 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.616823    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 23:22:01.618752 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.616826    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 23:22:01.618752 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.616828    2572 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 23:22:01.619255 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.616831    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 23:22:01.619255 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.616834    2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 23:22:01.619255 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.616837    2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 23:22:01.619255 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.616839    2572 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 23:22:01.619255 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.616842    2572 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 23:22:01.619255 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.616845    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 23:22:01.619255 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.616847    2572 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 23:22:01.619255 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.616850    2572 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 23:22:01.619255 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.616853    2572 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 23:22:01.619255 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.616855    2572 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 23:22:01.619255 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.616858    2572 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 23:22:01.619255 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.616861    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 23:22:01.619255 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.616864    2572 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 23:22:01.619255 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.616867    2572 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 23:22:01.619255 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.616869    2572 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 23:22:01.619255 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.616872    2572 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 23:22:01.619255 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.616875    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 23:22:01.619255 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.616877    2572 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 23:22:01.619255 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.616880    2572 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 23:22:01.619255 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.616883    2572 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 23:22:01.619748 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.616885    2572 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 23:22:01.619748 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.616889    2572 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 23:22:01.619748 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.616893    2572 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 23:22:01.619748 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.616896    2572 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 23:22:01.619748 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.616899    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 23:22:01.619748 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.616902    2572 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 23:22:01.619748 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.616905    2572 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 23:22:01.619748 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.616907    2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 23:22:01.619748 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.616912    2572 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 23:22:01.619748 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.616917    2572 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 23:22:01.619748 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.616921    2572 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 23:22:01.619748 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.616925    2572 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 23:22:01.619748 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.616928    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 23:22:01.619748 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.616931    2572 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 23:22:01.619748 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.616934    2572 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 23:22:01.619748 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.616936    2572 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 23:22:01.619748 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.616939    2572 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 23:22:01.619748 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.616942    2572 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 23:22:01.619748 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.616945    2572 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 23:22:01.620228 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.616947    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 23:22:01.620228 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.616950    2572 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 23:22:01.620228 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.616952    2572 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 23:22:01.620228 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.616957    2572 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 23:22:01.620228 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.616960    2572 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 23:22:01.620228 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.616962    2572 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 23:22:01.620228 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.616965    2572 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 23:22:01.620228 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.616967    2572 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 23:22:01.620228 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.616970    2572 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 23:22:01.620228 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.616972    2572 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 23:22:01.620228 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.616975    2572 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 23:22:01.620228 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.616978    2572 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 23:22:01.620228 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.616981    2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 23:22:01.620228 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.616984    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 23:22:01.620228 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.616986    2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 23:22:01.620228 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.616989    2572 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 23:22:01.620228 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.616991    2572 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 23:22:01.620228 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.616994    2572 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 23:22:01.620228 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.617401    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 23:22:01.620228 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.617406    2572 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 23:22:01.620714 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.617410    2572 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 23:22:01.620714 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.617412    2572 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 23:22:01.620714 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.617416    2572 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 23:22:01.620714 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.617418    2572 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 23:22:01.620714 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.617421    2572 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 23:22:01.620714 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.617423    2572 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 23:22:01.620714 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.617426    2572 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 23:22:01.620714 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.617429    2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 23:22:01.620714 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.617432    2572 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 23:22:01.620714 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.617434    2572 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 23:22:01.620714 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.617437    2572 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 23:22:01.620714 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.617440    2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 23:22:01.620714 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.617443    2572 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 23:22:01.620714 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.617445    2572 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 23:22:01.620714 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.617448    2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 23:22:01.620714 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.617452    2572 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 23:22:01.620714 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.617455    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 23:22:01.620714 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.617458    2572 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 23:22:01.620714 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.617460    2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 23:22:01.620714 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.617463    2572 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 23:22:01.621236 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.617466    2572 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 23:22:01.621236 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.617468    2572 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 23:22:01.621236 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.617470    2572 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 23:22:01.621236 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.617473    2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 23:22:01.621236 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.617475    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 23:22:01.621236 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.617479    2572 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 23:22:01.621236 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.617481    2572 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 23:22:01.621236 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.617483    2572 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 23:22:01.621236 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.617486    2572 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 23:22:01.621236 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.617488    2572 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 23:22:01.621236 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.617491    2572 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 23:22:01.621236 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.617494    2572 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 23:22:01.621236 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.617496    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 23:22:01.621236 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.617498    2572 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 23:22:01.621236 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.617501    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 23:22:01.621236 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.617504    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 23:22:01.621236 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.617506    2572 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 23:22:01.621236 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.617509    2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 23:22:01.621236 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.617511    2572 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 23:22:01.621236 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.617513    2572 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 23:22:01.621714 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.617517    2572 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 23:22:01.621714 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.617521    2572 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 23:22:01.621714 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.617524    2572 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 23:22:01.621714 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.617527    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 23:22:01.621714 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.617530    2572 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 23:22:01.621714 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.617532    2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 23:22:01.621714 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.617535    2572 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 23:22:01.621714 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.617539    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 23:22:01.621714 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.617541    2572 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 23:22:01.621714 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.617544    2572 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 23:22:01.621714 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.617546    2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 23:22:01.621714 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.617550    2572 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 23:22:01.621714 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.617552    2572 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 23:22:01.621714 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.617555    2572 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 23:22:01.621714 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.617557    2572 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 23:22:01.621714 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.617561    2572 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 23:22:01.621714 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.617564    2572 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 23:22:01.621714 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.617567    2572 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 23:22:01.621714 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.617570    2572 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 23:22:01.621714 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.617573    2572 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 23:22:01.622211 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.617575    2572 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 23:22:01.622211 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.617577    2572 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 23:22:01.622211 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.617580    2572 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 23:22:01.622211 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.617583    2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 23:22:01.622211 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.617585    2572 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 23:22:01.622211 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.617588    2572 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 23:22:01.622211 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.617590    2572 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 23:22:01.622211 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.617593    2572 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 23:22:01.622211 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.617595    2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 23:22:01.622211 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.617598    2572 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 23:22:01.622211 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.617600    2572 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 23:22:01.622211 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.617603    2572 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 23:22:01.622211 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.617606    2572 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 23:22:01.622211 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.617608    2572 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 23:22:01.622211 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.617611    2572 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 23:22:01.622211 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.617614    2572 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 23:22:01.622211 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.617616    2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 23:22:01.622211 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.617618    2572 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 23:22:01.622211 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.617622    2572 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 23:22:01.622729 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.617626    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 23:22:01.622729 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.617629    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 23:22:01.622729 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.617631    2572 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 23:22:01.622729 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.617634    2572 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 23:22:01.622729 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.617636    2572 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 23:22:01.622729 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.617639    2572 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 23:22:01.622729 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.617642    2572 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 23:22:01.622729 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.617644    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 23:22:01.622729 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.617647    2572 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 23:22:01.622729 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.617649    2572 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 23:22:01.622729 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.617652    2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 23:22:01.622729 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.617655    2572 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 23:22:01.622729 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.617659    2572 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 23:22:01.622729 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.617662    2572 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 23:22:01.622729 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.617665    2572 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 23:22:01.622729 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.617667    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 23:22:01.622729 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.617670    2572 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 23:22:01.622729 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.617672    2572 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 23:22:01.622729 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.617734    2572 flags.go:64] FLAG: --address="0.0.0.0"
Jul 28 23:22:01.622729 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.617741    2572 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 28 23:22:01.623240 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.617748    2572 flags.go:64] FLAG: --anonymous-auth="true"
Jul 28 23:22:01.623240 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.617752    2572 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 28 23:22:01.623240 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.617757    2572 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 28 23:22:01.623240 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.617760    2572 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 28 23:22:01.623240 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.617764    2572 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 28 23:22:01.623240 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.617768    2572 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 28 23:22:01.623240 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.617771    2572 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 28 23:22:01.623240 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.617776    2572 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 28 23:22:01.623240 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.617780    2572 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 28 23:22:01.623240 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.617783    2572 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 28 23:22:01.623240 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.617787    2572 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 28 23:22:01.623240 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.617790    2572 flags.go:64] FLAG: --cgroup-root=""
Jul 28 23:22:01.623240 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.617793    2572 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 28 23:22:01.623240 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.617796    2572 flags.go:64] FLAG: --client-ca-file=""
Jul 28 23:22:01.623240 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.617799    2572 flags.go:64] FLAG: --cloud-provider="external"
Jul 28 23:22:01.623240 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.617802    2572 flags.go:64] FLAG: --cluster-dns="[]"
Jul 28 23:22:01.623240 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.617806    2572 flags.go:64] FLAG: --cluster-domain=""
Jul 28 23:22:01.623240 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.617809    2572 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 28 23:22:01.623240 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.617812    2572 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 28 23:22:01.623240 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.617815    2572 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 28 23:22:01.623240 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.617819    2572 flags.go:64] FLAG: --container-log-max-files="5"
Jul 28 23:22:01.623240 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.617823    2572 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 28 23:22:01.623240 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.617825    2572 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 28 23:22:01.623829 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.617829    2572 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 28 23:22:01.623829 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.617832    2572 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 28 23:22:01.623829 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.617835    2572 flags.go:64] FLAG: --contention-profiling="false"
Jul 28 23:22:01.623829 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.617838    2572 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 28 23:22:01.623829 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.617841    2572 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 28 23:22:01.623829 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.617844    2572 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 28 23:22:01.623829 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.617847    2572 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 28 23:22:01.623829 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.617851    2572 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 28 23:22:01.623829 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.617855    2572 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 28 23:22:01.623829 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.617858    2572 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 28 23:22:01.623829 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.617860    2572 flags.go:64] FLAG: --enable-load-reader="false"
Jul 28 23:22:01.623829 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.617863    2572 flags.go:64] FLAG: --enable-server="true"
Jul 28 23:22:01.623829 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.617867    2572 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 28 23:22:01.623829 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.617872    2572 flags.go:64] FLAG: --event-burst="100"
Jul 28 23:22:01.623829 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.617875    2572 flags.go:64] FLAG: --event-qps="50"
Jul 28 23:22:01.623829 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.617878    2572 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 28 23:22:01.623829 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.617882    2572 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 28 23:22:01.623829 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.617885    2572 flags.go:64] FLAG: --eviction-hard=""
Jul 28 23:22:01.623829 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.617889    2572 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 28 23:22:01.623829 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.617892    2572 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 28 23:22:01.623829 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.617894    2572 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 28 23:22:01.623829 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.617898    2572 flags.go:64] FLAG: --eviction-soft=""
Jul 28 23:22:01.623829 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.617901    2572 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 28 23:22:01.623829 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.617905    2572 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 28 23:22:01.623829 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.617908    2572 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 28 23:22:01.624469 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.617911    2572 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 28 23:22:01.624469 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.617914    2572 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 28 23:22:01.624469 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.617917    2572 flags.go:64] FLAG: --fail-swap-on="true"
Jul 28 23:22:01.624469 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.617920    2572 flags.go:64] FLAG: --feature-gates=""
Jul 28 23:22:01.624469 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.617924    2572 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 28 23:22:01.624469 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.617927    2572 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 28 23:22:01.624469 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.617930    2572 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 28 23:22:01.624469 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.617933    2572 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 28 23:22:01.624469 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.617936    2572 flags.go:64] FLAG: --healthz-port="10248"
Jul 28 23:22:01.624469 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.617939    2572 flags.go:64] FLAG: --help="false"
Jul 28 23:22:01.624469 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.617942    2572 flags.go:64] FLAG: --hostname-override="ip-10-0-131-112.ec2.internal"
Jul 28 23:22:01.624469 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.617945    2572 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 28 23:22:01.624469 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.617948    2572 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 28 23:22:01.624469 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.617951    2572 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 28 23:22:01.624469 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.617954    2572 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 28 23:22:01.624469 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.617958    2572 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 28 23:22:01.624469 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.617961    2572 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 28 23:22:01.624469 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.617964    2572 flags.go:64] FLAG: --image-service-endpoint=""
Jul 28 23:22:01.624469 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.617967    2572 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 28 23:22:01.624469 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.617969    2572 flags.go:64] FLAG: --kube-api-burst="100"
Jul 28 23:22:01.624469 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.617973    2572 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 28 23:22:01.624469 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.617976    2572 flags.go:64] FLAG: --kube-api-qps="50"
Jul 28 23:22:01.624469 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.617979    2572 flags.go:64] FLAG: --kube-reserved=""
Jul 28 23:22:01.624469 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.617983    2572 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 28 23:22:01.625054 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.617986    2572 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 28 23:22:01.625054 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.617989    2572 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 28 23:22:01.625054 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.617992    2572 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 28 23:22:01.625054 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.617995    2572 flags.go:64] FLAG: --lock-file=""
Jul 28 23:22:01.625054 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.617998    2572 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 28 23:22:01.625054 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.618001    2572 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 28 23:22:01.625054 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.618004    2572 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 28 23:22:01.625054 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.618009    2572 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 28 23:22:01.625054 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.618012    2572 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 28 23:22:01.625054 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.618015    2572 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 28 23:22:01.625054 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.618018    2572 flags.go:64] FLAG: --logging-format="text"
Jul 28 23:22:01.625054 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.618021    2572 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 28 23:22:01.625054 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.618024    2572 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 28 23:22:01.625054 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.618027    2572 flags.go:64] FLAG: --manifest-url=""
Jul 28 23:22:01.625054 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.618030    2572 flags.go:64] FLAG: --manifest-url-header=""
Jul 28 23:22:01.625054 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.618034    2572 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 28 23:22:01.625054 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.618037    2572 flags.go:64] FLAG: --max-open-files="1000000"
Jul 28 23:22:01.625054 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.618041    2572 flags.go:64] FLAG: --max-pods="110"
Jul 28 23:22:01.625054 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.618044    2572 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 28 23:22:01.625054 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.618047    2572 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 28 23:22:01.625054 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.618050    2572 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 28 23:22:01.625054 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.618053    2572 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 28 23:22:01.625054 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.618068    2572 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 28 23:22:01.625054 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.618071    2572 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 28 23:22:01.625054 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.618074    2572 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 28 23:22:01.625786 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.618081    2572 flags.go:64] FLAG: --node-status-max-images="50"
Jul 28 23:22:01.625786 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.618084    2572 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 28 23:22:01.625786 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.618088    2572 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 28 23:22:01.625786 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.618091    2572 flags.go:64] FLAG: --pod-cidr=""
Jul 28 23:22:01.625786 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.618094    2572 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb"
Jul 28 23:22:01.625786 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.618100    2572 flags.go:64] FLAG: --pod-manifest-path=""
Jul 28 23:22:01.625786 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.618104    2572 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 28 23:22:01.625786 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.618108    2572 flags.go:64] FLAG: --pods-per-core="0"
Jul 28 23:22:01.625786 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.618111    2572 flags.go:64] FLAG: --port="10250"
Jul 28 23:22:01.625786 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.618114    2572 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 28 23:22:01.625786 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.618117    2572 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0f55ea9de439ab84d"
Jul 28 23:22:01.625786 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.618120    2572 flags.go:64] FLAG: --qos-reserved=""
Jul 28 23:22:01.625786 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.618123    2572 flags.go:64] FLAG: --read-only-port="10255"
Jul 28 23:22:01.625786 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.618126    2572 flags.go:64] FLAG: --register-node="true"
Jul 28 23:22:01.625786 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.618129    2572 flags.go:64] FLAG: --register-with-taints=""
Jul 28 23:22:01.625786 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.618132    2572 flags.go:64] FLAG: --registry-burst="10"
Jul 28 23:22:01.625786 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.618135    2572 flags.go:64] FLAG: --registry-qps="5"
Jul 28 23:22:01.625786 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.618138    2572 flags.go:64] FLAG: --reserved-cpus=""
Jul 28 23:22:01.625786 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.618141    2572 flags.go:64] FLAG: --reserved-memory=""
Jul 28 23:22:01.625786 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.618144    2572 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 28 23:22:01.625786 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.618148    2572 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 28 23:22:01.625786 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.618151    2572 flags.go:64] FLAG: --rotate-certificates="false"
Jul 28 23:22:01.625786 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.618154    2572 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 28 23:22:01.625786 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.618157    2572 flags.go:64] FLAG: --runonce="false"
Jul 28 23:22:01.625786 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.618160    2572 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 28 23:22:01.626421 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.618163    2572 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 28 23:22:01.626421 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.618166    2572 flags.go:64] FLAG: --seccomp-default="false"
Jul 28 23:22:01.626421 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.618169    2572 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 28 23:22:01.626421 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.618172    2572 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 28 23:22:01.626421 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.618175    2572 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 28 23:22:01.626421 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.618178    2572 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 28 23:22:01.626421 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.618181    2572 flags.go:64] FLAG: --storage-driver-password="root"
Jul 28 23:22:01.626421 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.618184    2572 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 28 23:22:01.626421 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.618187    2572 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 28 23:22:01.626421 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.618189    2572 flags.go:64] FLAG: --storage-driver-user="root"
Jul 28 23:22:01.626421 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.618192    2572 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 28 23:22:01.626421 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.618195    2572 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 28 23:22:01.626421 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.618199    2572 flags.go:64] FLAG: --system-cgroups=""
Jul 28 23:22:01.626421 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.618203    2572 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 28 23:22:01.626421 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.618208    2572 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 28 23:22:01.626421 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.618213    2572 flags.go:64] FLAG: --tls-cert-file=""
Jul 28 23:22:01.626421 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.618216    2572 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 28 23:22:01.626421 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.618220    2572 flags.go:64] FLAG: --tls-min-version=""
Jul 28 23:22:01.626421 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.618223    2572 flags.go:64] FLAG: --tls-private-key-file=""
Jul 28 23:22:01.626421 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.618226    2572 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 28 23:22:01.626421 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.618229    2572 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 28 23:22:01.626421 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.618231    2572 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 28 23:22:01.626421 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.618234    2572 flags.go:64] FLAG: --v="2"
Jul 28 23:22:01.626421 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.618238    2572 flags.go:64] FLAG: --version="false"
Jul 28 23:22:01.626421 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.618242    2572 flags.go:64] FLAG: --vmodule=""
Jul 28 23:22:01.627040 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.618246    2572 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 28 23:22:01.627040 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.618250    2572 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 28 23:22:01.627040 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.618364    2572 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 23:22:01.627040 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.618368    2572 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 23:22:01.627040 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.618372    2572 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 23:22:01.627040 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.618375    2572 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 23:22:01.627040 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.618378    2572 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 23:22:01.627040 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.618380    2572 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 23:22:01.627040 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.618383    2572 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 23:22:01.627040 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.618387    2572 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 23:22:01.627040 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.618390    2572 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 23:22:01.627040 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.618393    2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 23:22:01.627040 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.618395    2572 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 23:22:01.627040 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.618398    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 23:22:01.627040 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.618401    2572 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 23:22:01.627040 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.618404    2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 23:22:01.627040 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.618407    2572 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 23:22:01.627040 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.618416    2572 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 23:22:01.627040 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.618421    2572 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 23:22:01.627040 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.618424    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 23:22:01.627588 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.618429    2572 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 23:22:01.627588 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.618432    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 23:22:01.627588 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.618436    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 23:22:01.627588 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.618439    2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 23:22:01.627588 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.618441    2572 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 23:22:01.627588 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.618444    2572 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 23:22:01.627588 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.618447    2572 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 23:22:01.627588 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.618449    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 23:22:01.627588 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.618452    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 23:22:01.627588 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.618454    2572 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 23:22:01.627588 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.618457    2572 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 23:22:01.627588 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.618460    2572 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 23:22:01.627588 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.618462    2572 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 23:22:01.627588 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.618465    2572 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 23:22:01.627588 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.618468    2572 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 23:22:01.627588 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.618470    2572 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 23:22:01.627588 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.618473    2572 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 23:22:01.627588 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.618475    2572 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 23:22:01.627588 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.618478    2572 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 23:22:01.627588 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.618480    2572 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 23:22:01.628081 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.618483    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 23:22:01.628081 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.618485    2572 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 23:22:01.628081 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.618488    2572 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 23:22:01.628081 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.618490    2572 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 23:22:01.628081 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.618493    2572 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 23:22:01.628081 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.618495    2572 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 23:22:01.628081 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.618498    2572 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 23:22:01.628081 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.618500    2572 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 23:22:01.628081 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.618503    2572 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 23:22:01.628081 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.618506    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 23:22:01.628081 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.618508    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 23:22:01.628081 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.618511    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 23:22:01.628081 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.618515    2572 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 23:22:01.628081 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.618518    2572 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 23:22:01.628081 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.618522    2572 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 23:22:01.628081 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.618525    2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 23:22:01.628081 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.618527    2572 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 23:22:01.628081 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.618530    2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 23:22:01.628081 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.618532    2572 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 23:22:01.628081 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.618535    2572 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 23:22:01.628572 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.618537    2572 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 23:22:01.628572 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.618540    2572 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 23:22:01.628572 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.618543    2572 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 23:22:01.628572 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.618545    2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 23:22:01.628572 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.618548    2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 23:22:01.628572 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.618551    2572 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 23:22:01.628572 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.618554    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 23:22:01.628572 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.618557    2572 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 23:22:01.628572 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.618559    2572 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 23:22:01.628572 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.618562    2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 23:22:01.628572 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.618564    2572 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 23:22:01.628572 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.618567    2572 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 23:22:01.628572 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.618569    2572 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 23:22:01.628572 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.618572    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 23:22:01.628572 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.618574    2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 23:22:01.628572 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.618577    2572 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 23:22:01.628572 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.618579    2572 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 23:22:01.628572 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.618582    2572 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 23:22:01.628572 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.618584    2572 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 23:22:01.629310 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.618588    2572 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 23:22:01.629310 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.618591    2572 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 23:22:01.629310 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.618595    2572 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 23:22:01.629310 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.618597    2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 23:22:01.629310 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.618600    2572 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 23:22:01.629310 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.618604    2572 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 23:22:01.629310 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.618607    2572 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 23:22:01.629310 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.618611    2572 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 23:22:01.629310 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.618614    2572 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 23:22:01.629310 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.618616    2572 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 23:22:01.629310 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.618619    2572 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 23:22:01.629310 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.618621    2572 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 23:22:01.629310 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.618624    2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 23:22:01.629310 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.618626    2572 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 23:22:01.629310 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.618629    2572 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 23:22:01.629310 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.618633    2572 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 23:22:01.629310 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.618636    2572 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 23:22:01.629310 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.618640    2572 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 23:22:01.629852 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.618643    2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 23:22:01.629852 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.618645    2572 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 23:22:01.629852 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.618648    2572 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 23:22:01.629852 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.618651    2572 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 23:22:01.629852 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.619876    2572 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 28 23:22:01.629852 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.626340    2572 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 28 23:22:01.629852 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.626356    2572 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 28 23:22:01.629852 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.627743    2572 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 28 23:22:01.629852 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.627754    2572 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 28 23:22:01.629852 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.627813    2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 23:22:01.629852 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.627820    2572 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 23:22:01.629852 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.627823    2572 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 23:22:01.629852 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.627828    2572 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 23:22:01.629852 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.627831    2572 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 23:22:01.629852 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.627834    2572 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 23:22:01.630281 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.627837    2572 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 23:22:01.630281 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.627840    2572 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 23:22:01.630281 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.627843    2572 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 23:22:01.630281 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.627846    2572 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 23:22:01.630281 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.627848    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 23:22:01.630281 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.627852    2572 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 23:22:01.630281 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.627855    2572 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 23:22:01.630281 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.627858    2572 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 23:22:01.630281 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.627860    2572 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 23:22:01.630281 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.627865    2572 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 23:22:01.630281 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.627868    2572 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 23:22:01.630281 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.627871    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 23:22:01.630281 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.627874    2572 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 23:22:01.630281 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.627877    2572 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 23:22:01.630281 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.627880    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 23:22:01.630281 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.627882    2572 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 23:22:01.630281 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.627885    2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 23:22:01.630281 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.627888    2572 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 23:22:01.630281 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.627890    2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 23:22:01.630281 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.627893    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 23:22:01.630783 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.627895    2572 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 23:22:01.630783 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.627898    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 23:22:01.630783 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.627900    2572 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 23:22:01.630783 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.627904    2572 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 23:22:01.630783 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.627907    2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 23:22:01.630783 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.627910    2572 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 23:22:01.630783 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.627912    2572 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 23:22:01.630783 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.627915    2572 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 23:22:01.630783 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.627917    2572 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 23:22:01.630783 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.627920    2572 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 23:22:01.630783 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.627922    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 23:22:01.630783 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.627925    2572 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 23:22:01.630783 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.627928    2572 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 23:22:01.630783 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.627931    2572 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 23:22:01.630783 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.627933    2572 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 23:22:01.630783 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.627936    2572 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 23:22:01.630783 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.627939    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 23:22:01.630783 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.627942    2572 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 23:22:01.630783 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.627944    2572 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 23:22:01.630783 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.627947    2572 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 23:22:01.631294 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.627949    2572 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 23:22:01.631294 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.627953    2572 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 23:22:01.631294 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.627955    2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 23:22:01.631294 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.627958    2572 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 23:22:01.631294 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.627960    2572 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 23:22:01.631294 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.627964    2572 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 23:22:01.631294 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.627966    2572 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 23:22:01.631294 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.627968    2572 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 23:22:01.631294 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.627971    2572 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 23:22:01.631294 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.627973    2572 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 23:22:01.631294 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.627976    2572 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 23:22:01.631294 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.627979    2572 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 23:22:01.631294 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.627981    2572 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 23:22:01.631294 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.627984    2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 23:22:01.631294 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.627988    2572 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 23:22:01.631294 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.627992    2572 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 23:22:01.631294 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.627996    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 23:22:01.631294 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.627999    2572 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 23:22:01.631294 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.628002    2572 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 23:22:01.631749 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.628004    2572 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 23:22:01.631749 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.628007    2572 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 23:22:01.631749 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.628009    2572 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 23:22:01.631749 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.628012    2572 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 23:22:01.631749 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.628014    2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 23:22:01.631749 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.628017    2572 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 23:22:01.631749 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.628019    2572 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 23:22:01.631749 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.628022    2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 23:22:01.631749 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.628025    2572 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 23:22:01.631749 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.628029    2572 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 23:22:01.631749 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.628032    2572 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 23:22:01.631749 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.628035    2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 23:22:01.631749 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.628038    2572 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 23:22:01.631749 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.628040    2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 23:22:01.631749 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.628043    2572 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 23:22:01.631749 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.628045    2572 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 23:22:01.631749 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.628048    2572 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 23:22:01.631749 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.628051    2572 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 23:22:01.631749 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.628053    2572 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 23:22:01.632305 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.628071    2572 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 23:22:01.632305 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.628075    2572 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 23:22:01.632305 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.628078    2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 23:22:01.632305 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.628081    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 23:22:01.632305 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.628083    2572 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 23:22:01.632305 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.628086    2572 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 23:22:01.632305 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.628089    2572 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 23:22:01.632305 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.628092    2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 23:22:01.632305 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.628095    2572 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 23:22:01.632305 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.628098    2572 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 23:22:01.632305 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.628100    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 23:22:01.632305 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.628104    2572 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 23:22:01.632305 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.628106    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 23:22:01.632305 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.628109    2572 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 23:22:01.632305 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.628112    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 23:22:01.632666 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.628117    2572 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 28 23:22:01.632666 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.628232    2572 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 23:22:01.632666 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.628238    2572 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 23:22:01.632666 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.628241    2572 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 23:22:01.632666 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.628244    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 23:22:01.632666 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.628247    2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 23:22:01.632666 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.628250    2572 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 23:22:01.632666 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.628253    2572 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 23:22:01.632666 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.628256    2572 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 23:22:01.632666 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.628260    2572 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 23:22:01.632666 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.628262    2572 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 23:22:01.632666 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.628265    2572 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 23:22:01.632666 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.628268    2572 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 23:22:01.632666 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.628271    2572 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 23:22:01.633001 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.628273    2572 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 23:22:01.633001 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.628276    2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 23:22:01.633001 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.628278    2572 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 23:22:01.633001 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.628281    2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 23:22:01.633001 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.628283    2572 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 23:22:01.633001 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.628286    2572 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 23:22:01.633001 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.628289    2572 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 23:22:01.633001 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.628291    2572 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 23:22:01.633001 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.628294    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 23:22:01.633001 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.628297    2572 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 23:22:01.633001 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.628301    2572 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 23:22:01.633001 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.628304    2572 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 23:22:01.633001 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.628307    2572 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 23:22:01.633001 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.628310    2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 23:22:01.633001 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.628313    2572 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 23:22:01.633001 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.628319    2572 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 23:22:01.633001 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.628322    2572 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 23:22:01.633001 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.628324    2572 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 23:22:01.633001 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.628327    2572 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 23:22:01.633001 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.628329    2572 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 23:22:01.633489 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.628332    2572 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 23:22:01.633489 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.628335    2572 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 23:22:01.633489 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.628337    2572 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 23:22:01.633489 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.628340    2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 23:22:01.633489 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.628342    2572 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 23:22:01.633489 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.628345    2572 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 23:22:01.633489 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.628347    2572 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 23:22:01.633489 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.628350    2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 23:22:01.633489 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.628353    2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 23:22:01.633489 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.628355    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 23:22:01.633489 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.628358    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 23:22:01.633489 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.628360    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 23:22:01.633489 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.628363    2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 23:22:01.633489 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.628365    2572 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 23:22:01.633489 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.628368    2572 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 23:22:01.633489 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.628370    2572 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 23:22:01.633489 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.628372    2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 23:22:01.633489 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.628375    2572 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 23:22:01.633489 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.628378    2572 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 23:22:01.633489 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.628381    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 23:22:01.633963 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.628384    2572 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 23:22:01.633963 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.628386    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 23:22:01.633963 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.628389    2572 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 23:22:01.633963 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.628391    2572 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 23:22:01.633963 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.628394    2572 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 23:22:01.633963 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.628396    2572 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 23:22:01.633963 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.628399    2572 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 23:22:01.633963 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.628402    2572 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 23:22:01.633963 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.628406    2572 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 23:22:01.633963 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.628408    2572 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 23:22:01.633963 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.628411    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 23:22:01.633963 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.628414    2572 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 23:22:01.633963 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.628416    2572 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 23:22:01.633963 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.628419    2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 23:22:01.633963 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.628421    2572 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 23:22:01.633963 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.628424    2572 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 23:22:01.633963 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.628427    2572 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 23:22:01.633963 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.628429    2572 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 23:22:01.633963 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.628432    2572 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 23:22:01.633963 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.628434    2572 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 23:22:01.634461 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.628437    2572 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 23:22:01.634461 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.628439    2572 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 23:22:01.634461 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.628441    2572 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 23:22:01.634461 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.628444    2572 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 23:22:01.634461 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.628446    2572 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 23:22:01.634461 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.628449    2572 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 23:22:01.634461 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.628452    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 23:22:01.634461 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.628454    2572 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 23:22:01.634461 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.628457    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 23:22:01.634461 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.628459    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 23:22:01.634461 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.628462    2572 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 23:22:01.634461 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.628465    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 23:22:01.634461 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.628468    2572 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 23:22:01.634461 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.628471    2572 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 23:22:01.634461 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.628473    2572 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 23:22:01.634461 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.628476    2572 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 23:22:01.634461 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.628478    2572 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 23:22:01.634461 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.628481    2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 23:22:01.634461 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.628483    2572 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 23:22:01.634461 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.628486    2572 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 23:22:01.634946 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.628488    2572 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 23:22:01.634946 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.628491    2572 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 23:22:01.634946 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.628494    2572 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 23:22:01.634946 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.628496    2572 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 23:22:01.634946 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.628499    2572 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 23:22:01.634946 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:01.628501    2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 23:22:01.634946 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.628506    2572 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 28 23:22:01.634946 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.630164    2572 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 28 23:22:01.634946 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.632177    2572 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 28 23:22:01.634946 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.633812    2572 server.go:1019] "Starting client certificate rotation"
Jul 28 23:22:01.634946 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.633909    2572 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 28 23:22:01.634946 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.633945    2572 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 28 23:22:01.636378 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.636362    2572 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 28 23:22:01.641186 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.641170    2572 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 28 23:22:01.654213 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.654194    2572 log.go:25] "Validated CRI v1 runtime API"
Jul 28 23:22:01.656290 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.656276    2572 log.go:25] "Validated CRI v1 image API"
Jul 28 23:22:01.657107 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.657090    2572 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 28 23:22:01.659678 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.659650    2572 fs.go:135] Filesystem UUIDs: map[0b4cf6f0-5ea5-4dfe-86c4-93f942a56fb6:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 dbe5b1b9-0816-4908-aa35-7b1f366f4aac:/dev/nvme0n1p4]
Jul 28 23:22:01.659761 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.659677    2572 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 28 23:22:01.662702 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.662679    2572 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 28 23:22:01.666755 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.666633    2572 manager.go:216] Machine: {Timestamp:2026-07-28 23:22:01.665341378 +0000 UTC m=+0.322579444 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3105413 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec249ce6b0f72889addafb2f4c154e3b SystemUUID:ec249ce6-b0f7-2889-adda-fb2f4c154e3b BootID:e13ed8bd-940d-4e77-afca-7f0b75072fd8 Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:42:2d:e8:cd:cb Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:42:2d:e8:cd:cb Speed:0 Mtu:9001} {Name:ovs-system MacAddress:d2:96:30:aa:04:2b Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 28 23:22:01.666755 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.666751    2572 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 28 23:22:01.666863 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.666837    2572 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 28 23:22:01.670838 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.670814    2572 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 28 23:22:01.670986 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.670843    2572 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-131-112.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 28 23:22:01.671477 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.671466    2572 topology_manager.go:138] "Creating topology manager with none policy"
Jul 28 23:22:01.671508 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.671480    2572 container_manager_linux.go:309] "Creating device plugin manager"
Jul 28 23:22:01.671508 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.671488    2572 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 28 23:22:01.672174 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.672164    2572 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 28 23:22:01.672925 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.672915    2572 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 28 23:22:01.675583 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.675572    2572 state_mem.go:36] "Initialized new in-memory state store"
Jul 28 23:22:01.675700 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.675690    2572 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 28 23:22:01.675731 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.675719    2572 kubelet.go:486] "Attempting to sync node with API server"
Jul 28 23:22:01.675731 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.675729    2572 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 28 23:22:01.675792 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.675743    2572 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 28 23:22:01.675792 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.675753    2572 kubelet.go:398] "Adding apiserver pod source"
Jul 28 23:22:01.675792 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.675762    2572 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 28 23:22:01.676180 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.676166    2572 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 28 23:22:01.676269 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.676187    2572 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 28 23:22:01.679434 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.679416    2572 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 28 23:22:01.681102 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.681089    2572 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 28 23:22:01.681150 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.681119    2572 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 28 23:22:01.682685 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.682673    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 28 23:22:01.682723 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.682690    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 28 23:22:01.682723 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.682696    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 28 23:22:01.682723 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.682701    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 28 23:22:01.682723 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.682713    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 28 23:22:01.682723 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.682721    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 28 23:22:01.682864 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.682730    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 28 23:22:01.682864 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.682736    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 28 23:22:01.682864 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.682742    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 28 23:22:01.682864 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.682748    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 28 23:22:01.682864 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.682759    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 28 23:22:01.682864 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.682764    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 28 23:22:01.683490 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.683481    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 28 23:22:01.683490 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.683490    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 28 23:22:01.686088 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.686044    2572 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-qdzrj"
Jul 28 23:22:01.686637 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:22:01.686612    2572 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 28 23:22:01.686637 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.686633    2572 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-131-112.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 28 23:22:01.686724 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:22:01.686677    2572 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-131-112.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 28 23:22:01.686865 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.686854    2572 server.go:1268] "Started kubelet"
Jul 28 23:22:01.686984 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.686933    2572 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 28 23:22:01.687264 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.687162    2572 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 28 23:22:01.687406 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.687300    2572 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 28 23:22:01.687722 ip-10-0-131-112 systemd[1]: Started Kubernetes Kubelet.
Jul 28 23:22:01.689447 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.689413    2572 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 28 23:22:01.689779 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.689767    2572 server.go:310] "Adding debug handlers to kubelet server"
Jul 28 23:22:01.691292 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:22:01.691271    2572 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 28 23:22:01.692934 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.692917    2572 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-qdzrj"
Jul 28 23:22:01.693798 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:22:01.692928    2572 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-112.ec2.internal.18c6978a196ce335  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-112.ec2.internal,UID:ip-10-0-131-112.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-131-112.ec2.internal,},FirstTimestamp:2026-07-28 23:22:01.686827829 +0000 UTC m=+0.344065896,LastTimestamp:2026-07-28 23:22:01.686827829 +0000 UTC m=+0.344065896,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-112.ec2.internal,}"
Jul 28 23:22:01.696854 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.696837    2572 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 28 23:22:01.696854 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.696845    2572 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 28 23:22:01.697024 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:22:01.697005    2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-112.ec2.internal\" not found"
Jul 28 23:22:01.697472 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.697454    2572 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 28 23:22:01.697548 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.697455    2572 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 28 23:22:01.697548 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.697489    2572 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 28 23:22:01.697672 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.697645    2572 reconstruct.go:97] "Volume reconstruction finished"
Jul 28 23:22:01.697672 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.697661    2572 reconciler.go:29] "Reconciler: start to sync state"
Jul 28 23:22:01.698481 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.698432    2572 factory.go:153] Registering CRI-O factory
Jul 28 23:22:01.698481 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.698448    2572 factory.go:223] Registration of the crio container factory successfully
Jul 28 23:22:01.698850 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.698499    2572 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 28 23:22:01.698850 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.698510    2572 factory.go:55] Registering systemd factory
Jul 28 23:22:01.698850 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.698518    2572 factory.go:223] Registration of the systemd container factory successfully
Jul 28 23:22:01.698850 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.698536    2572 factory.go:103] Registering Raw factory
Jul 28 23:22:01.698850 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.698548    2572 manager.go:1196] Started watching for new ooms in manager
Jul 28 23:22:01.699077 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.698959    2572 manager.go:349] Starting recovery of all containers
Jul 28 23:22:01.706389 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.706362    2572 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 23:22:01.710051 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:22:01.709585    2572 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-131-112.ec2.internal\" not found" node="ip-10-0-131-112.ec2.internal"
Jul 28 23:22:01.710205 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.710070    2572 manager.go:354] Recovery completed
Jul 28 23:22:01.711809 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:22:01.711788    2572 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory
Jul 28 23:22:01.714890 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.714874    2572 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 23:22:01.716745 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.716730    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-112.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 23:22:01.716830 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.716752    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-112.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 23:22:01.716830 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.716763    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-112.ec2.internal" event="NodeHasSufficientPID"
Jul 28 23:22:01.717336 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.717319    2572 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 28 23:22:01.717336 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.717334    2572 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 23:22:01.717454 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.717334    2572 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 28 23:22:01.717454 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.717391    2572 state_mem.go:36] "Initialized new in-memory state store"
Jul 28 23:22:01.718027 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.718013    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-112.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 23:22:01.718110 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.718043    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-112.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 23:22:01.718110 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.718071    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-112.ec2.internal" event="NodeHasSufficientPID"
Jul 28 23:22:01.719778 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.719766    2572 policy_none.go:49] "None policy: Start"
Jul 28 23:22:01.719827 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.719783    2572 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 28 23:22:01.719827 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.719792    2572 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 28 23:22:01.721611 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:01.721600    2572 policy_none.go:47] "Start"
Jul 28 23:22:03.732396 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:03.732191    2572 manager.go:337] "Starting Device Plugin manager"
Jul 28 23:22:03.732489 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:22:03.732416    2572 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 28 23:22:03.732489 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:03.732430    2572 server.go:87] "Starting device plugin registration server"
Jul 28 23:22:03.732719 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:03.732707    2572 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 28 23:22:03.732766 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:03.732726    2572 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 28 23:22:03.732854 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:03.732834    2572 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 28 23:22:03.732943 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:03.732930    2572 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 28 23:22:03.732943 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:03.732942    2572 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 28 23:22:03.733480 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:22:03.733460    2572 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 28 23:22:03.733563 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:22:03.733506    2572 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-131-112.ec2.internal\" not found"
Jul 28 23:22:03.833909 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:03.833837    2572 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 23:22:03.835870 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:03.834929    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-112.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 23:22:03.835870 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:03.834956    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-112.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 23:22:03.835870 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:03.834967    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-112.ec2.internal" event="NodeHasSufficientPID"
Jul 28 23:22:03.835870 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:03.835007    2572 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-131-112.ec2.internal"
Jul 28 23:22:03.842533 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:03.842518    2572 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-131-112.ec2.internal"
Jul 28 23:22:03.842679 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:22:03.842541    2572 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-131-112.ec2.internal\": node \"ip-10-0-131-112.ec2.internal\" not found"
Jul 28 23:22:03.848994 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:22:03.848971    2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-112.ec2.internal\" not found"
Jul 28 23:22:03.858006 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:03.857972    2572 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 28 23:22:03.859383 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:03.859364    2572 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 28 23:22:03.859455 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:03.859409    2572 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 28 23:22:03.859455 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:03.859432    2572 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 28 23:22:03.859538 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:22:03.859467    2572 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 28 23:22:03.860753 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:03.860735    2572 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 23:22:03.949099 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:22:03.949062    2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-112.ec2.internal\" not found"
Jul 28 23:22:03.960246 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:03.960213    2572 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-112.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-131-112.ec2.internal"]
Jul 28 23:22:03.960380 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:03.960294    2572 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 23:22:03.961318 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:03.961303    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-112.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 23:22:03.961371 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:03.961335    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-112.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 23:22:03.961371 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:03.961345    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-112.ec2.internal" event="NodeHasSufficientPID"
Jul 28 23:22:03.963488 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:03.963472    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/eb93e6ceeada8d8a42bfd36cc63ee14d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-112.ec2.internal\" (UID: \"eb93e6ceeada8d8a42bfd36cc63ee14d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-112.ec2.internal"
Jul 28 23:22:03.963542 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:03.963498    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/eb93e6ceeada8d8a42bfd36cc63ee14d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-112.ec2.internal\" (UID: \"eb93e6ceeada8d8a42bfd36cc63ee14d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-112.ec2.internal"
Jul 28 23:22:03.963587 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:03.963575    2572 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 23:22:03.963707 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:03.963693    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-112.ec2.internal"
Jul 28 23:22:03.963750 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:03.963727    2572 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 23:22:03.964278 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:03.964262    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-112.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 23:22:03.964352 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:03.964293    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-112.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 23:22:03.964352 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:03.964297    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-112.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 23:22:03.964352 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:03.964305    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-112.ec2.internal" event="NodeHasSufficientPID"
Jul 28 23:22:03.964352 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:03.964316    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-112.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 23:22:03.964352 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:03.964326    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-112.ec2.internal" event="NodeHasSufficientPID"
Jul 28 23:22:03.966411 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:03.966397    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-112.ec2.internal"
Jul 28 23:22:03.966479 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:03.966422    2572 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 23:22:03.967067 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:03.967051    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-112.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 23:22:03.967155 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:03.967080    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-112.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 23:22:03.967155 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:03.967091    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-112.ec2.internal" event="NodeHasSufficientPID"
Jul 28 23:22:03.982203 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:22:03.982183    2572 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-131-112.ec2.internal\" not found" node="ip-10-0-131-112.ec2.internal"
Jul 28 23:22:03.986570 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:22:03.986555    2572 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-131-112.ec2.internal\" not found" node="ip-10-0-131-112.ec2.internal"
Jul 28 23:22:04.049285 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:22:04.049251    2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-112.ec2.internal\" not found"
Jul 28 23:22:04.064601 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.064579    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/eb93e6ceeada8d8a42bfd36cc63ee14d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-112.ec2.internal\" (UID: \"eb93e6ceeada8d8a42bfd36cc63ee14d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-112.ec2.internal"
Jul 28 23:22:04.064709 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.064606    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/eb93e6ceeada8d8a42bfd36cc63ee14d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-112.ec2.internal\" (UID: \"eb93e6ceeada8d8a42bfd36cc63ee14d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-112.ec2.internal"
Jul 28 23:22:04.064709 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.064647    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/48ad21d512355ab82436faafc0eae6b0-config\") pod \"kube-apiserver-proxy-ip-10-0-131-112.ec2.internal\" (UID: \"48ad21d512355ab82436faafc0eae6b0\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-112.ec2.internal"
Jul 28 23:22:04.064709 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.064675    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/eb93e6ceeada8d8a42bfd36cc63ee14d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-112.ec2.internal\" (UID: \"eb93e6ceeada8d8a42bfd36cc63ee14d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-112.ec2.internal"
Jul 28 23:22:04.064709 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.064695    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/eb93e6ceeada8d8a42bfd36cc63ee14d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-112.ec2.internal\" (UID: \"eb93e6ceeada8d8a42bfd36cc63ee14d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-112.ec2.internal"
Jul 28 23:22:04.150083 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:22:04.150014    2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-112.ec2.internal\" not found"
Jul 28 23:22:04.165368 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.165342    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/48ad21d512355ab82436faafc0eae6b0-config\") pod \"kube-apiserver-proxy-ip-10-0-131-112.ec2.internal\" (UID: \"48ad21d512355ab82436faafc0eae6b0\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-112.ec2.internal"
Jul 28 23:22:04.165452 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.165387    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/48ad21d512355ab82436faafc0eae6b0-config\") pod \"kube-apiserver-proxy-ip-10-0-131-112.ec2.internal\" (UID: \"48ad21d512355ab82436faafc0eae6b0\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-112.ec2.internal"
Jul 28 23:22:04.250548 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:22:04.250511    2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-112.ec2.internal\" not found"
Jul 28 23:22:04.284985 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.284962    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-112.ec2.internal"
Jul 28 23:22:04.290720 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.290701    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-112.ec2.internal"
Jul 28 23:22:04.351150 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:22:04.351117    2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-112.ec2.internal\" not found"
Jul 28 23:22:04.451703 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:22:04.451610    2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-112.ec2.internal\" not found"
Jul 28 23:22:04.552194 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:22:04.552164    2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-112.ec2.internal\" not found"
Jul 28 23:22:04.578148 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.578125    2572 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 23:22:04.597345 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.597319    2572 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 28 23:22:04.597498 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.597472    2572 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 28 23:22:04.597539 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.597479    2572 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 28 23:22:04.597539 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.597481    2572 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 28 23:22:04.640703 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.640665    2572 apiserver.go:52] "Watching apiserver"
Jul 28 23:22:04.658208 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.658176    2572 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 28 23:22:04.658566 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.658531    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-node-tuning-operator/tuned-jx5sf","openshift-dns/node-resolver-xqqk5","openshift-multus/multus-l85q7","openshift-network-diagnostics/network-check-target-2svls","openshift-network-operator/iptables-alerter-sltwc","openshift-image-registry/node-ca-5ndfr","openshift-multus/multus-additional-cni-plugins-v6nmr","openshift-multus/network-metrics-daemon-lpj5x","openshift-ovn-kubernetes/ovnkube-node-v84z7","kube-system/global-pull-secret-syncer-8pczd","kube-system/konnectivity-agent-6dwz5","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c59s9"]
Jul 28 23:22:04.660918 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.660899    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-jx5sf"
Jul 28 23:22:04.661794 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.661776    2572 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 28 23:22:04.661794 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.661788    2572 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-112.ec2.internal"
Jul 28 23:22:04.662262 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.662225    2572 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-27 23:17:03 +0000 UTC" deadline="2028-03-27 22:39:10.676858537 +0000 UTC"
Jul 28 23:22:04.662262 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.662258    2572 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14591h17m6.01460276s"
Jul 28 23:22:04.663105 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.663085    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 28 23:22:04.663192 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.663106    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-vvz2b\""
Jul 28 23:22:04.663192 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.663124    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-xqqk5"
Jul 28 23:22:04.663192 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.663088    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 28 23:22:04.665234 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.665217    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-l85q7"
Jul 28 23:22:04.666720 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.666493    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 28 23:22:04.666720 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.666501    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-gw6n9\""
Jul 28 23:22:04.666720 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.666547    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 28 23:22:04.667470 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.667446    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 28 23:22:04.667553 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.667543    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 28 23:22:04.667861 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.667842    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-qlptt\""
Jul 28 23:22:04.667931 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.667890    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 28 23:22:04.668029 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.667888    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/aa22bd20-02b0-4d17-90b8-b0607ef1d16d-etc-sysctl-d\") pod \"tuned-jx5sf\" (UID: \"aa22bd20-02b0-4d17-90b8-b0607ef1d16d\") " pod="openshift-cluster-node-tuning-operator/tuned-jx5sf"
Jul 28 23:22:04.668096 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.668082    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/aa22bd20-02b0-4d17-90b8-b0607ef1d16d-etc-sysconfig\") pod \"tuned-jx5sf\" (UID: \"aa22bd20-02b0-4d17-90b8-b0607ef1d16d\") " pod="openshift-cluster-node-tuning-operator/tuned-jx5sf"
Jul 28 23:22:04.668148 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.668117    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 28 23:22:04.668256 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.668118    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/aa22bd20-02b0-4d17-90b8-b0607ef1d16d-etc-sysctl-conf\") pod \"tuned-jx5sf\" (UID: \"aa22bd20-02b0-4d17-90b8-b0607ef1d16d\") " pod="openshift-cluster-node-tuning-operator/tuned-jx5sf"
Jul 28 23:22:04.668309 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.668277    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/aa22bd20-02b0-4d17-90b8-b0607ef1d16d-etc-systemd\") pod \"tuned-jx5sf\" (UID: \"aa22bd20-02b0-4d17-90b8-b0607ef1d16d\") " pod="openshift-cluster-node-tuning-operator/tuned-jx5sf"
Jul 28 23:22:04.668360 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.668302    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/aa22bd20-02b0-4d17-90b8-b0607ef1d16d-run\") pod \"tuned-jx5sf\" (UID: \"aa22bd20-02b0-4d17-90b8-b0607ef1d16d\") " pod="openshift-cluster-node-tuning-operator/tuned-jx5sf"
Jul 28 23:22:04.668360 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.668332    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/aa22bd20-02b0-4d17-90b8-b0607ef1d16d-sys\") pod \"tuned-jx5sf\" (UID: \"aa22bd20-02b0-4d17-90b8-b0607ef1d16d\") " pod="openshift-cluster-node-tuning-operator/tuned-jx5sf"
Jul 28 23:22:04.668533 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.668362    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/aa22bd20-02b0-4d17-90b8-b0607ef1d16d-etc-tuned\") pod \"tuned-jx5sf\" (UID: \"aa22bd20-02b0-4d17-90b8-b0607ef1d16d\") " pod="openshift-cluster-node-tuning-operator/tuned-jx5sf"
Jul 28 23:22:04.668533 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.668394    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nt5jx\" (UniqueName: \"kubernetes.io/projected/aa22bd20-02b0-4d17-90b8-b0607ef1d16d-kube-api-access-nt5jx\") pod \"tuned-jx5sf\" (UID: \"aa22bd20-02b0-4d17-90b8-b0607ef1d16d\") " pod="openshift-cluster-node-tuning-operator/tuned-jx5sf"
Jul 28 23:22:04.668533 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.668428    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/aa22bd20-02b0-4d17-90b8-b0607ef1d16d-etc-modprobe-d\") pod \"tuned-jx5sf\" (UID: \"aa22bd20-02b0-4d17-90b8-b0607ef1d16d\") " pod="openshift-cluster-node-tuning-operator/tuned-jx5sf"
Jul 28 23:22:04.668533 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.668510    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/aa22bd20-02b0-4d17-90b8-b0607ef1d16d-var-lib-kubelet\") pod \"tuned-jx5sf\" (UID: \"aa22bd20-02b0-4d17-90b8-b0607ef1d16d\") " pod="openshift-cluster-node-tuning-operator/tuned-jx5sf"
Jul 28 23:22:04.668533 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.668525    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2svls"
Jul 28 23:22:04.668835 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.668572    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/aa22bd20-02b0-4d17-90b8-b0607ef1d16d-tmp\") pod \"tuned-jx5sf\" (UID: \"aa22bd20-02b0-4d17-90b8-b0607ef1d16d\") " pod="openshift-cluster-node-tuning-operator/tuned-jx5sf"
Jul 28 23:22:04.668835 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.668602    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/82ce0587-81a8-4b5e-b841-14fed1b2364c-tmp-dir\") pod \"node-resolver-xqqk5\" (UID: \"82ce0587-81a8-4b5e-b841-14fed1b2364c\") " pod="openshift-dns/node-resolver-xqqk5"
Jul 28 23:22:04.668835 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:22:04.668611    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2svls" podUID="ae951cd2-ca5f-4ac4-abaa-3293c8f1ea57"
Jul 28 23:22:04.668835 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.668646    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rfmc5\" (UniqueName: \"kubernetes.io/projected/82ce0587-81a8-4b5e-b841-14fed1b2364c-kube-api-access-rfmc5\") pod \"node-resolver-xqqk5\" (UID: \"82ce0587-81a8-4b5e-b841-14fed1b2364c\") " pod="openshift-dns/node-resolver-xqqk5"
Jul 28 23:22:04.668835 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.668695    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/aa22bd20-02b0-4d17-90b8-b0607ef1d16d-lib-modules\") pod \"tuned-jx5sf\" (UID: \"aa22bd20-02b0-4d17-90b8-b0607ef1d16d\") " pod="openshift-cluster-node-tuning-operator/tuned-jx5sf"
Jul 28 23:22:04.668835 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.668721    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/aa22bd20-02b0-4d17-90b8-b0607ef1d16d-host\") pod \"tuned-jx5sf\" (UID: \"aa22bd20-02b0-4d17-90b8-b0607ef1d16d\") " pod="openshift-cluster-node-tuning-operator/tuned-jx5sf"
Jul 28 23:22:04.668835 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.668748    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/82ce0587-81a8-4b5e-b841-14fed1b2364c-hosts-file\") pod \"node-resolver-xqqk5\" (UID: \"82ce0587-81a8-4b5e-b841-14fed1b2364c\") " pod="openshift-dns/node-resolver-xqqk5"
Jul 28 23:22:04.668835 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.668776    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/aa22bd20-02b0-4d17-90b8-b0607ef1d16d-etc-kubernetes\") pod \"tuned-jx5sf\" (UID: \"aa22bd20-02b0-4d17-90b8-b0607ef1d16d\") " pod="openshift-cluster-node-tuning-operator/tuned-jx5sf"
Jul 28 23:22:04.670643 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.670601    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-sltwc"
Jul 28 23:22:04.670748 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.670730    2572 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 28 23:22:04.672752 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.672733    2572 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 28 23:22:04.672828 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.672801    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 28 23:22:04.672828 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.672812    2572 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-112.ec2.internal"
Jul 28 23:22:04.672920 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.672879    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-5ndfr"
Jul 28 23:22:04.673099 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.672982    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 28 23:22:04.673486 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.673467    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 28 23:22:04.673556 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.673470    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-tnkm8\""
Jul 28 23:22:04.674918 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.674905    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 28 23:22:04.675267 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.675251    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-v6nmr"
Jul 28 23:22:04.675585 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.675400    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 28 23:22:04.675585 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.675420    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 28 23:22:04.675585 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.675426    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-v2hlg\""
Jul 28 23:22:04.677326 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.677312    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-nx6z4\""
Jul 28 23:22:04.677414 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.677396    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 28 23:22:04.677564 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.677542    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lpj5x"
Jul 28 23:22:04.677610 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.677596    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 28 23:22:04.677660 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:22:04.677614    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lpj5x" podUID="df9e50e4-7650-40bb-83eb-becf4ba8c051"
Jul 28 23:22:04.679957 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.679940    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-v84z7"
Jul 28 23:22:04.680355 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.680332    2572 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 28 23:22:04.682004 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.681971    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 28 23:22:04.682237 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.682218    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 28 23:22:04.682305 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.682226    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-gq66v\""
Jul 28 23:22:04.682361 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.682225    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 28 23:22:04.682486 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.682463    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 28 23:22:04.682593 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.682556    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 28 23:22:04.682774 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.682750    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 28 23:22:04.682854 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.682829    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8pczd"
Jul 28 23:22:04.683019 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:22:04.682998    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8pczd" podUID="f0137726-3966-44f4-bcc7-c69cc72143be"
Jul 28 23:22:04.685351 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.685330    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-6dwz5"
Jul 28 23:22:04.687548 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.687524    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 28 23:22:04.687652 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.687591    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-bxrhq\""
Jul 28 23:22:04.687832 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.687819    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 28 23:22:04.688192 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.688135    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-112.ec2.internal"]
Jul 28 23:22:04.688275 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.688199    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c59s9"
Jul 28 23:22:04.688275 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.688252    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-131-112.ec2.internal"]
Jul 28 23:22:04.690346 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.690328    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 28 23:22:04.690346 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.690337    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 28 23:22:04.690499 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.690366    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 28 23:22:04.690572 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.690556    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-xgjcc\""
Jul 28 23:22:04.695913 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.695897    2572 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-zvpnv"
Jul 28 23:22:04.703204 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.703158    2572 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-zvpnv"
Jul 28 23:22:04.762892 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.762871    2572 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 28 23:22:04.769447 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.769427    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/dc10426a-a32b-4910-a2c8-21345f7b691b-host-var-lib-cni-bin\") pod \"multus-l85q7\" (UID: \"dc10426a-a32b-4910-a2c8-21345f7b691b\") " pod="openshift-multus/multus-l85q7"
Jul 28 23:22:04.769527 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.769455    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cmmbl\" (UniqueName: \"kubernetes.io/projected/f220827f-670e-407f-b5c9-664800ba743d-kube-api-access-cmmbl\") pod \"multus-additional-cni-plugins-v6nmr\" (UID: \"f220827f-670e-407f-b5c9-664800ba743d\") " pod="openshift-multus/multus-additional-cni-plugins-v6nmr"
Jul 28 23:22:04.769527 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.769474    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bd3f4ca7-06e3-4857-b42c-32c1a90fd147-host-slash\") pod \"ovnkube-node-v84z7\" (UID: \"bd3f4ca7-06e3-4857-b42c-32c1a90fd147\") " pod="openshift-ovn-kubernetes/ovnkube-node-v84z7"
Jul 28 23:22:04.769527 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.769489    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/bd3f4ca7-06e3-4857-b42c-32c1a90fd147-ovnkube-config\") pod \"ovnkube-node-v84z7\" (UID: \"bd3f4ca7-06e3-4857-b42c-32c1a90fd147\") " pod="openshift-ovn-kubernetes/ovnkube-node-v84z7"
Jul 28 23:22:04.769527 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.769505    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dc10426a-a32b-4910-a2c8-21345f7b691b-etc-kubernetes\") pod \"multus-l85q7\" (UID: \"dc10426a-a32b-4910-a2c8-21345f7b691b\") " pod="openshift-multus/multus-l85q7"
Jul 28 23:22:04.769527 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.769520    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f220827f-670e-407f-b5c9-664800ba743d-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-v6nmr\" (UID: \"f220827f-670e-407f-b5c9-664800ba743d\") " pod="openshift-multus/multus-additional-cni-plugins-v6nmr"
Jul 28 23:22:04.769715 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.769569    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nt5jx\" (UniqueName: \"kubernetes.io/projected/aa22bd20-02b0-4d17-90b8-b0607ef1d16d-kube-api-access-nt5jx\") pod \"tuned-jx5sf\" (UID: \"aa22bd20-02b0-4d17-90b8-b0607ef1d16d\") " pod="openshift-cluster-node-tuning-operator/tuned-jx5sf"
Jul 28 23:22:04.769715 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.769611    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/bd3f4ca7-06e3-4857-b42c-32c1a90fd147-run-ovn\") pod \"ovnkube-node-v84z7\" (UID: \"bd3f4ca7-06e3-4857-b42c-32c1a90fd147\") " pod="openshift-ovn-kubernetes/ovnkube-node-v84z7"
Jul 28 23:22:04.769775 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.769728    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/bd3f4ca7-06e3-4857-b42c-32c1a90fd147-ovn-node-metrics-cert\") pod \"ovnkube-node-v84z7\" (UID: \"bd3f4ca7-06e3-4857-b42c-32c1a90fd147\") " pod="openshift-ovn-kubernetes/ovnkube-node-v84z7"
Jul 28 23:22:04.769775 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.769751    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f220827f-670e-407f-b5c9-664800ba743d-os-release\") pod \"multus-additional-cni-plugins-v6nmr\" (UID: \"f220827f-670e-407f-b5c9-664800ba743d\") " pod="openshift-multus/multus-additional-cni-plugins-v6nmr"
Jul 28 23:22:04.769775 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.769772    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f220827f-670e-407f-b5c9-664800ba743d-cni-binary-copy\") pod \"multus-additional-cni-plugins-v6nmr\" (UID: \"f220827f-670e-407f-b5c9-664800ba743d\") " pod="openshift-multus/multus-additional-cni-plugins-v6nmr"
Jul 28 23:22:04.769880 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.769792    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/aa22bd20-02b0-4d17-90b8-b0607ef1d16d-etc-modprobe-d\") pod \"tuned-jx5sf\" (UID: \"aa22bd20-02b0-4d17-90b8-b0607ef1d16d\") " pod="openshift-cluster-node-tuning-operator/tuned-jx5sf"
Jul 28 23:22:04.769880 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.769809    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/aa22bd20-02b0-4d17-90b8-b0607ef1d16d-var-lib-kubelet\") pod \"tuned-jx5sf\" (UID: \"aa22bd20-02b0-4d17-90b8-b0607ef1d16d\") " pod="openshift-cluster-node-tuning-operator/tuned-jx5sf"
Jul 28 23:22:04.769880 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.769837    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/25a639de-470a-46a0-bad9-c5e646731566-kubelet-dir\") pod \"aws-ebs-csi-driver-node-c59s9\" (UID: \"25a639de-470a-46a0-bad9-c5e646731566\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c59s9"
Jul 28 23:22:04.769880 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.769853    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/dc10426a-a32b-4910-a2c8-21345f7b691b-multus-conf-dir\") pod \"multus-l85q7\" (UID: \"dc10426a-a32b-4910-a2c8-21345f7b691b\") " pod="openshift-multus/multus-l85q7"
Jul 28 23:22:04.770019 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.769883    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j5m76\" (UniqueName: \"kubernetes.io/projected/dc10426a-a32b-4910-a2c8-21345f7b691b-kube-api-access-j5m76\") pod \"multus-l85q7\" (UID: \"dc10426a-a32b-4910-a2c8-21345f7b691b\") " pod="openshift-multus/multus-l85q7"
Jul 28 23:22:04.770019 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.769899    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f220827f-670e-407f-b5c9-664800ba743d-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-v6nmr\" (UID: \"f220827f-670e-407f-b5c9-664800ba743d\") " pod="openshift-multus/multus-additional-cni-plugins-v6nmr"
Jul 28 23:22:04.770019 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.769903    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/aa22bd20-02b0-4d17-90b8-b0607ef1d16d-etc-modprobe-d\") pod \"tuned-jx5sf\" (UID: \"aa22bd20-02b0-4d17-90b8-b0607ef1d16d\") " pod="openshift-cluster-node-tuning-operator/tuned-jx5sf"
Jul 28 23:22:04.770019 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.769905    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/aa22bd20-02b0-4d17-90b8-b0607ef1d16d-var-lib-kubelet\") pod \"tuned-jx5sf\" (UID: \"aa22bd20-02b0-4d17-90b8-b0607ef1d16d\") " pod="openshift-cluster-node-tuning-operator/tuned-jx5sf"
Jul 28 23:22:04.770019 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.769931    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/aa22bd20-02b0-4d17-90b8-b0607ef1d16d-lib-modules\") pod \"tuned-jx5sf\" (UID: \"aa22bd20-02b0-4d17-90b8-b0607ef1d16d\") " pod="openshift-cluster-node-tuning-operator/tuned-jx5sf"
Jul 28 23:22:04.770019 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.769953    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/82ce0587-81a8-4b5e-b841-14fed1b2364c-hosts-file\") pod \"node-resolver-xqqk5\" (UID: \"82ce0587-81a8-4b5e-b841-14fed1b2364c\") " pod="openshift-dns/node-resolver-xqqk5"
Jul 28 23:22:04.770019 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.769973    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/bd3f4ca7-06e3-4857-b42c-32c1a90fd147-host-kubelet\") pod \"ovnkube-node-v84z7\" (UID: \"bd3f4ca7-06e3-4857-b42c-32c1a90fd147\") " pod="openshift-ovn-kubernetes/ovnkube-node-v84z7"
Jul 28 23:22:04.770019 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.770014    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/82ce0587-81a8-4b5e-b841-14fed1b2364c-hosts-file\") pod \"node-resolver-xqqk5\" (UID: \"82ce0587-81a8-4b5e-b841-14fed1b2364c\") " pod="openshift-dns/node-resolver-xqqk5"
Jul 28 23:22:04.770348 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.770018    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9pp2n\" (UniqueName: \"kubernetes.io/projected/bd3f4ca7-06e3-4857-b42c-32c1a90fd147-kube-api-access-9pp2n\") pod \"ovnkube-node-v84z7\" (UID: \"bd3f4ca7-06e3-4857-b42c-32c1a90fd147\") " pod="openshift-ovn-kubernetes/ovnkube-node-v84z7"
Jul 28 23:22:04.770348 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.770063    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/f0137726-3966-44f4-bcc7-c69cc72143be-dbus\") pod \"global-pull-secret-syncer-8pczd\" (UID: \"f0137726-3966-44f4-bcc7-c69cc72143be\") " pod="kube-system/global-pull-secret-syncer-8pczd"
Jul 28 23:22:04.770348 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.770079    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/aa22bd20-02b0-4d17-90b8-b0607ef1d16d-lib-modules\") pod \"tuned-jx5sf\" (UID: \"aa22bd20-02b0-4d17-90b8-b0607ef1d16d\") " pod="openshift-cluster-node-tuning-operator/tuned-jx5sf"
Jul 28 23:22:04.770348 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.770092    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/b923a656-b1ea-41e7-8c65-440a24abcee6-agent-certs\") pod \"konnectivity-agent-6dwz5\" (UID: \"b923a656-b1ea-41e7-8c65-440a24abcee6\") " pod="kube-system/konnectivity-agent-6dwz5"
Jul 28 23:22:04.770348 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.770116    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/dc10426a-a32b-4910-a2c8-21345f7b691b-cnibin\") pod \"multus-l85q7\" (UID: \"dc10426a-a32b-4910-a2c8-21345f7b691b\") " pod="openshift-multus/multus-l85q7"
Jul 28 23:22:04.770348 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.770139    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dc10426a-a32b-4910-a2c8-21345f7b691b-system-cni-dir\") pod \"multus-l85q7\" (UID: \"dc10426a-a32b-4910-a2c8-21345f7b691b\") " pod="openshift-multus/multus-l85q7"
Jul 28 23:22:04.770348 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.770167    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/aa22bd20-02b0-4d17-90b8-b0607ef1d16d-etc-sysctl-d\") pod \"tuned-jx5sf\" (UID: \"aa22bd20-02b0-4d17-90b8-b0607ef1d16d\") " pod="openshift-cluster-node-tuning-operator/tuned-jx5sf"
Jul 28 23:22:04.770348 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.770193    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bd3f4ca7-06e3-4857-b42c-32c1a90fd147-host-cni-bin\") pod \"ovnkube-node-v84z7\" (UID: \"bd3f4ca7-06e3-4857-b42c-32c1a90fd147\") " pod="openshift-ovn-kubernetes/ovnkube-node-v84z7"
Jul 28 23:22:04.770348 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.770222    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/bd3f4ca7-06e3-4857-b42c-32c1a90fd147-host-cni-netd\") pod \"ovnkube-node-v84z7\" (UID: \"bd3f4ca7-06e3-4857-b42c-32c1a90fd147\") " pod="openshift-ovn-kubernetes/ovnkube-node-v84z7"
Jul 28 23:22:04.770348 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.770247    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f0137726-3966-44f4-bcc7-c69cc72143be-original-pull-secret\") pod \"global-pull-secret-syncer-8pczd\" (UID: \"f0137726-3966-44f4-bcc7-c69cc72143be\") " pod="kube-system/global-pull-secret-syncer-8pczd"
Jul 28 23:22:04.770348 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.770277    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/dc10426a-a32b-4910-a2c8-21345f7b691b-hostroot\") pod \"multus-l85q7\" (UID: \"dc10426a-a32b-4910-a2c8-21345f7b691b\") " pod="openshift-multus/multus-l85q7"
Jul 28 23:22:04.770348 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.770302    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/aa22bd20-02b0-4d17-90b8-b0607ef1d16d-etc-sysctl-d\") pod \"tuned-jx5sf\" (UID: \"aa22bd20-02b0-4d17-90b8-b0607ef1d16d\") " pod="openshift-cluster-node-tuning-operator/tuned-jx5sf"
Jul 28 23:22:04.770786 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.770355    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/dc10426a-a32b-4910-a2c8-21345f7b691b-multus-daemon-config\") pod \"multus-l85q7\" (UID: \"dc10426a-a32b-4910-a2c8-21345f7b691b\") " pod="openshift-multus/multus-l85q7"
Jul 28 23:22:04.770786 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.770403    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/aa22bd20-02b0-4d17-90b8-b0607ef1d16d-etc-sysconfig\") pod \"tuned-jx5sf\" (UID: \"aa22bd20-02b0-4d17-90b8-b0607ef1d16d\") " pod="openshift-cluster-node-tuning-operator/tuned-jx5sf"
Jul 28 23:22:04.770786 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.770464    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/aa22bd20-02b0-4d17-90b8-b0607ef1d16d-etc-sysconfig\") pod \"tuned-jx5sf\" (UID: \"aa22bd20-02b0-4d17-90b8-b0607ef1d16d\") " pod="openshift-cluster-node-tuning-operator/tuned-jx5sf"
Jul 28 23:22:04.770786 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.770441    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/aa22bd20-02b0-4d17-90b8-b0607ef1d16d-etc-systemd\") pod \"tuned-jx5sf\" (UID: \"aa22bd20-02b0-4d17-90b8-b0607ef1d16d\") " pod="openshift-cluster-node-tuning-operator/tuned-jx5sf"
Jul 28 23:22:04.770786 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.770523    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/aa22bd20-02b0-4d17-90b8-b0607ef1d16d-run\") pod \"tuned-jx5sf\" (UID: \"aa22bd20-02b0-4d17-90b8-b0607ef1d16d\") " pod="openshift-cluster-node-tuning-operator/tuned-jx5sf"
Jul 28 23:22:04.770786 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.770534    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/aa22bd20-02b0-4d17-90b8-b0607ef1d16d-etc-systemd\") pod \"tuned-jx5sf\" (UID: \"aa22bd20-02b0-4d17-90b8-b0607ef1d16d\") " pod="openshift-cluster-node-tuning-operator/tuned-jx5sf"
Jul 28 23:22:04.770786 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.770542    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/bd3f4ca7-06e3-4857-b42c-32c1a90fd147-ovnkube-script-lib\") pod \"ovnkube-node-v84z7\" (UID: \"bd3f4ca7-06e3-4857-b42c-32c1a90fd147\") " pod="openshift-ovn-kubernetes/ovnkube-node-v84z7"
Jul 28 23:22:04.770786 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.770559    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/25a639de-470a-46a0-bad9-c5e646731566-registration-dir\") pod \"aws-ebs-csi-driver-node-c59s9\" (UID: \"25a639de-470a-46a0-bad9-c5e646731566\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c59s9"
Jul 28 23:22:04.770786 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.770597    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/25a639de-470a-46a0-bad9-c5e646731566-device-dir\") pod \"aws-ebs-csi-driver-node-c59s9\" (UID: \"25a639de-470a-46a0-bad9-c5e646731566\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c59s9"
Jul 28 23:22:04.770786 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.770618    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f220827f-670e-407f-b5c9-664800ba743d-cnibin\") pod \"multus-additional-cni-plugins-v6nmr\" (UID: \"f220827f-670e-407f-b5c9-664800ba743d\") " pod="openshift-multus/multus-additional-cni-plugins-v6nmr"
Jul 28 23:22:04.770786 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.770658    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/aa22bd20-02b0-4d17-90b8-b0607ef1d16d-tmp\") pod \"tuned-jx5sf\" (UID: \"aa22bd20-02b0-4d17-90b8-b0607ef1d16d\") " pod="openshift-cluster-node-tuning-operator/tuned-jx5sf"
Jul 28 23:22:04.770786 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.770684    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/aa22bd20-02b0-4d17-90b8-b0607ef1d16d-run\") pod \"tuned-jx5sf\" (UID: \"aa22bd20-02b0-4d17-90b8-b0607ef1d16d\") " pod="openshift-cluster-node-tuning-operator/tuned-jx5sf"
Jul 28 23:22:04.770786 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.770687    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bd3f4ca7-06e3-4857-b42c-32c1a90fd147-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-v84z7\" (UID: \"bd3f4ca7-06e3-4857-b42c-32c1a90fd147\") " pod="openshift-ovn-kubernetes/ovnkube-node-v84z7"
Jul 28 23:22:04.770786 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.770732    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/dc10426a-a32b-4910-a2c8-21345f7b691b-os-release\") pod \"multus-l85q7\" (UID: \"dc10426a-a32b-4910-a2c8-21345f7b691b\") " pod="openshift-multus/multus-l85q7"
Jul 28 23:22:04.770786 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.770769    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/aa22bd20-02b0-4d17-90b8-b0607ef1d16d-host\") pod \"tuned-jx5sf\" (UID: \"aa22bd20-02b0-4d17-90b8-b0607ef1d16d\") " pod="openshift-cluster-node-tuning-operator/tuned-jx5sf"
Jul 28 23:22:04.770786 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.770792    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/f0137726-3966-44f4-bcc7-c69cc72143be-kubelet-config\") pod \"global-pull-secret-syncer-8pczd\" (UID: \"f0137726-3966-44f4-bcc7-c69cc72143be\") " pod="kube-system/global-pull-secret-syncer-8pczd"
Jul 28 23:22:04.771547 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.770819    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-khr8t\" (UniqueName: \"kubernetes.io/projected/df9e50e4-7650-40bb-83eb-becf4ba8c051-kube-api-access-khr8t\") pod \"network-metrics-daemon-lpj5x\" (UID: \"df9e50e4-7650-40bb-83eb-becf4ba8c051\") " pod="openshift-multus/network-metrics-daemon-lpj5x"
Jul 28 23:22:04.771547 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.770842    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/b923a656-b1ea-41e7-8c65-440a24abcee6-konnectivity-ca\") pod \"konnectivity-agent-6dwz5\" (UID: \"b923a656-b1ea-41e7-8c65-440a24abcee6\") " pod="kube-system/konnectivity-agent-6dwz5"
Jul 28 23:22:04.771547 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.770849    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/aa22bd20-02b0-4d17-90b8-b0607ef1d16d-host\") pod \"tuned-jx5sf\" (UID: \"aa22bd20-02b0-4d17-90b8-b0607ef1d16d\") " pod="openshift-cluster-node-tuning-operator/tuned-jx5sf"
Jul 28 23:22:04.771547 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.770865    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bd3f4ca7-06e3-4857-b42c-32c1a90fd147-host-run-netns\") pod \"ovnkube-node-v84z7\" (UID: \"bd3f4ca7-06e3-4857-b42c-32c1a90fd147\") " pod="openshift-ovn-kubernetes/ovnkube-node-v84z7"
Jul 28 23:22:04.771547 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.770887    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bd3f4ca7-06e3-4857-b42c-32c1a90fd147-var-lib-openvswitch\") pod \"ovnkube-node-v84z7\" (UID: \"bd3f4ca7-06e3-4857-b42c-32c1a90fd147\") " pod="openshift-ovn-kubernetes/ovnkube-node-v84z7"
Jul 28 23:22:04.771547 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.770912    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/bd3f4ca7-06e3-4857-b42c-32c1a90fd147-node-log\") pod \"ovnkube-node-v84z7\" (UID: \"bd3f4ca7-06e3-4857-b42c-32c1a90fd147\") " pod="openshift-ovn-kubernetes/ovnkube-node-v84z7"
Jul 28 23:22:04.771547 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.770934    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n7cs4\" (UniqueName: \"kubernetes.io/projected/6640efb8-6a16-44ee-b831-90f5ee40a30c-kube-api-access-n7cs4\") pod \"iptables-alerter-sltwc\" (UID: \"6640efb8-6a16-44ee-b831-90f5ee40a30c\") " pod="openshift-network-operator/iptables-alerter-sltwc"
Jul 28 23:22:04.771547 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.770959    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/aa22bd20-02b0-4d17-90b8-b0607ef1d16d-etc-tuned\") pod \"tuned-jx5sf\" (UID: \"aa22bd20-02b0-4d17-90b8-b0607ef1d16d\") " pod="openshift-cluster-node-tuning-operator/tuned-jx5sf"
Jul 28 23:22:04.771547 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.771001    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/bd3f4ca7-06e3-4857-b42c-32c1a90fd147-log-socket\") pod \"ovnkube-node-v84z7\" (UID: \"bd3f4ca7-06e3-4857-b42c-32c1a90fd147\") " pod="openshift-ovn-kubernetes/ovnkube-node-v84z7"
Jul 28 23:22:04.771547 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.771016    2572 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 28 23:22:04.771547 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.771048    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/dc10426a-a32b-4910-a2c8-21345f7b691b-host-run-netns\") pod \"multus-l85q7\" (UID: \"dc10426a-a32b-4910-a2c8-21345f7b691b\") " pod="openshift-multus/multus-l85q7"
Jul 28 23:22:04.771547 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.771101    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/6640efb8-6a16-44ee-b831-90f5ee40a30c-iptables-alerter-script\") pod \"iptables-alerter-sltwc\" (UID: \"6640efb8-6a16-44ee-b831-90f5ee40a30c\") " pod="openshift-network-operator/iptables-alerter-sltwc"
Jul 28 23:22:04.771547 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.771157    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6640efb8-6a16-44ee-b831-90f5ee40a30c-host-slash\") pod \"iptables-alerter-sltwc\" (UID: \"6640efb8-6a16-44ee-b831-90f5ee40a30c\") " pod="openshift-network-operator/iptables-alerter-sltwc"
Jul 28 23:22:04.771547 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.771181    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5c9cc37c-daaf-4fa1-b71e-94bcc6732adb-host\") pod \"node-ca-5ndfr\" (UID: \"5c9cc37c-daaf-4fa1-b71e-94bcc6732adb\") " pod="openshift-image-registry/node-ca-5ndfr"
Jul 28 23:22:04.771547 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.771219    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rfmc5\" (UniqueName: \"kubernetes.io/projected/82ce0587-81a8-4b5e-b841-14fed1b2364c-kube-api-access-rfmc5\") pod \"node-resolver-xqqk5\" (UID: \"82ce0587-81a8-4b5e-b841-14fed1b2364c\") " pod="openshift-dns/node-resolver-xqqk5"
Jul 28 23:22:04.771547 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.771252    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/bd3f4ca7-06e3-4857-b42c-32c1a90fd147-systemd-units\") pod \"ovnkube-node-v84z7\" (UID: \"bd3f4ca7-06e3-4857-b42c-32c1a90fd147\") " pod="openshift-ovn-kubernetes/ovnkube-node-v84z7"
Jul 28 23:22:04.771547 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.771349    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n2jnx\" (UniqueName: \"kubernetes.io/projected/ae951cd2-ca5f-4ac4-abaa-3293c8f1ea57-kube-api-access-n2jnx\") pod \"network-check-target-2svls\" (UID: \"ae951cd2-ca5f-4ac4-abaa-3293c8f1ea57\") " pod="openshift-network-diagnostics/network-check-target-2svls"
Jul 28 23:22:04.772279 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.771373    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/25a639de-470a-46a0-bad9-c5e646731566-etc-selinux\") pod \"aws-ebs-csi-driver-node-c59s9\" (UID: \"25a639de-470a-46a0-bad9-c5e646731566\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c59s9"
Jul 28 23:22:04.772279 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.771396    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/dc10426a-a32b-4910-a2c8-21345f7b691b-host-var-lib-cni-multus\") pod \"multus-l85q7\" (UID: \"dc10426a-a32b-4910-a2c8-21345f7b691b\") " pod="openshift-multus/multus-l85q7"
Jul 28 23:22:04.772279 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.771420    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f220827f-670e-407f-b5c9-664800ba743d-system-cni-dir\") pod \"multus-additional-cni-plugins-v6nmr\" (UID: \"f220827f-670e-407f-b5c9-664800ba743d\") " pod="openshift-multus/multus-additional-cni-plugins-v6nmr"
Jul 28 23:22:04.772279 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.771442    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/bd3f4ca7-06e3-4857-b42c-32c1a90fd147-run-systemd\") pod \"ovnkube-node-v84z7\" (UID: \"bd3f4ca7-06e3-4857-b42c-32c1a90fd147\") " pod="openshift-ovn-kubernetes/ovnkube-node-v84z7"
Jul 28 23:22:04.772279 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.771464    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dc10426a-a32b-4910-a2c8-21345f7b691b-multus-cni-dir\") pod \"multus-l85q7\" (UID: \"dc10426a-a32b-4910-a2c8-21345f7b691b\") " pod="openshift-multus/multus-l85q7"
Jul 28 23:22:04.772279 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.771507    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5c9cc37c-daaf-4fa1-b71e-94bcc6732adb-serviceca\") pod \"node-ca-5ndfr\" (UID: \"5c9cc37c-daaf-4fa1-b71e-94bcc6732adb\") " pod="openshift-image-registry/node-ca-5ndfr"
Jul 28 23:22:04.772279 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.771535    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/25a639de-470a-46a0-bad9-c5e646731566-sys-fs\") pod \"aws-ebs-csi-driver-node-c59s9\" (UID: \"25a639de-470a-46a0-bad9-c5e646731566\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c59s9"
Jul 28 23:22:04.772279 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.771564    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/aa22bd20-02b0-4d17-90b8-b0607ef1d16d-etc-kubernetes\") pod \"tuned-jx5sf\" (UID: \"aa22bd20-02b0-4d17-90b8-b0607ef1d16d\") " pod="openshift-cluster-node-tuning-operator/tuned-jx5sf"
Jul 28 23:22:04.772279 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.771594    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bd3f4ca7-06e3-4857-b42c-32c1a90fd147-run-openvswitch\") pod \"ovnkube-node-v84z7\" (UID: \"bd3f4ca7-06e3-4857-b42c-32c1a90fd147\") " pod="openshift-ovn-kubernetes/ovnkube-node-v84z7"
Jul 28 23:22:04.772279 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.771641    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bd3f4ca7-06e3-4857-b42c-32c1a90fd147-host-run-ovn-kubernetes\") pod \"ovnkube-node-v84z7\" (UID: \"bd3f4ca7-06e3-4857-b42c-32c1a90fd147\") " pod="openshift-ovn-kubernetes/ovnkube-node-v84z7"
Jul 28 23:22:04.772279 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.771647    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/aa22bd20-02b0-4d17-90b8-b0607ef1d16d-etc-kubernetes\") pod \"tuned-jx5sf\" (UID: \"aa22bd20-02b0-4d17-90b8-b0607ef1d16d\") " pod="openshift-cluster-node-tuning-operator/tuned-jx5sf"
Jul 28 23:22:04.772279 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.771661    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/df9e50e4-7650-40bb-83eb-becf4ba8c051-metrics-certs\") pod \"network-metrics-daemon-lpj5x\" (UID: \"df9e50e4-7650-40bb-83eb-becf4ba8c051\") " pod="openshift-multus/network-metrics-daemon-lpj5x"
Jul 28 23:22:04.772279 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.771684    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/dc10426a-a32b-4910-a2c8-21345f7b691b-host-run-k8s-cni-cncf-io\") pod \"multus-l85q7\" (UID: \"dc10426a-a32b-4910-a2c8-21345f7b691b\") " pod="openshift-multus/multus-l85q7"
Jul 28 23:22:04.772279 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.771703    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/dc10426a-a32b-4910-a2c8-21345f7b691b-host-run-multus-certs\") pod \"multus-l85q7\" (UID: \"dc10426a-a32b-4910-a2c8-21345f7b691b\") " pod="openshift-multus/multus-l85q7"
Jul 28 23:22:04.772279 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.771723    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f220827f-670e-407f-b5c9-664800ba743d-tuning-conf-dir\") pod \"multus-additional-cni-plugins-v6nmr\" (UID: \"f220827f-670e-407f-b5c9-664800ba743d\") " pod="openshift-multus/multus-additional-cni-plugins-v6nmr"
Jul 28 23:22:04.772279 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.771773    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/aa22bd20-02b0-4d17-90b8-b0607ef1d16d-etc-sysctl-conf\") pod \"tuned-jx5sf\" (UID: \"aa22bd20-02b0-4d17-90b8-b0607ef1d16d\") " pod="openshift-cluster-node-tuning-operator/tuned-jx5sf"
Jul 28 23:22:04.772279 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.771799    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/aa22bd20-02b0-4d17-90b8-b0607ef1d16d-sys\") pod \"tuned-jx5sf\" (UID: \"aa22bd20-02b0-4d17-90b8-b0607ef1d16d\") " pod="openshift-cluster-node-tuning-operator/tuned-jx5sf"
Jul 28 23:22:04.773121 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.771822    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bd3f4ca7-06e3-4857-b42c-32c1a90fd147-etc-openvswitch\") pod \"ovnkube-node-v84z7\" (UID: \"bd3f4ca7-06e3-4857-b42c-32c1a90fd147\") " pod="openshift-ovn-kubernetes/ovnkube-node-v84z7"
Jul 28 23:22:04.773121 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.771844    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/bd3f4ca7-06e3-4857-b42c-32c1a90fd147-env-overrides\") pod \"ovnkube-node-v84z7\" (UID: \"bd3f4ca7-06e3-4857-b42c-32c1a90fd147\") " pod="openshift-ovn-kubernetes/ovnkube-node-v84z7"
Jul 28 23:22:04.773121 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.771865    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/25a639de-470a-46a0-bad9-c5e646731566-socket-dir\") pod \"aws-ebs-csi-driver-node-c59s9\" (UID: \"25a639de-470a-46a0-bad9-c5e646731566\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c59s9"
Jul 28 23:22:04.773121 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.771886    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/aa22bd20-02b0-4d17-90b8-b0607ef1d16d-etc-sysctl-conf\") pod \"tuned-jx5sf\" (UID: \"aa22bd20-02b0-4d17-90b8-b0607ef1d16d\") " pod="openshift-cluster-node-tuning-operator/tuned-jx5sf"
Jul 28 23:22:04.773121 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.771901    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/dc10426a-a32b-4910-a2c8-21345f7b691b-cni-binary-copy\") pod \"multus-l85q7\" (UID: \"dc10426a-a32b-4910-a2c8-21345f7b691b\") " pod="openshift-multus/multus-l85q7"
Jul 28 23:22:04.773121 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.771926    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/dc10426a-a32b-4910-a2c8-21345f7b691b-host-var-lib-kubelet\") pod \"multus-l85q7\" (UID: \"dc10426a-a32b-4910-a2c8-21345f7b691b\") " pod="openshift-multus/multus-l85q7"
Jul 28 23:22:04.773121 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.771905    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/aa22bd20-02b0-4d17-90b8-b0607ef1d16d-sys\") pod \"tuned-jx5sf\" (UID: \"aa22bd20-02b0-4d17-90b8-b0607ef1d16d\") " pod="openshift-cluster-node-tuning-operator/tuned-jx5sf"
Jul 28 23:22:04.773121 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.771952    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lf2sg\" (UniqueName: \"kubernetes.io/projected/5c9cc37c-daaf-4fa1-b71e-94bcc6732adb-kube-api-access-lf2sg\") pod \"node-ca-5ndfr\" (UID: \"5c9cc37c-daaf-4fa1-b71e-94bcc6732adb\") " pod="openshift-image-registry/node-ca-5ndfr"
Jul 28 23:22:04.773121 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.772001    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/82ce0587-81a8-4b5e-b841-14fed1b2364c-tmp-dir\") pod \"node-resolver-xqqk5\" (UID: \"82ce0587-81a8-4b5e-b841-14fed1b2364c\") " pod="openshift-dns/node-resolver-xqqk5"
Jul 28 23:22:04.773121 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.772052    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bdc9z\" (UniqueName: \"kubernetes.io/projected/25a639de-470a-46a0-bad9-c5e646731566-kube-api-access-bdc9z\") pod \"aws-ebs-csi-driver-node-c59s9\" (UID: \"25a639de-470a-46a0-bad9-c5e646731566\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c59s9"
Jul 28 23:22:04.773121 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.772078    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/dc10426a-a32b-4910-a2c8-21345f7b691b-multus-socket-dir-parent\") pod \"multus-l85q7\" (UID: \"dc10426a-a32b-4910-a2c8-21345f7b691b\") " pod="openshift-multus/multus-l85q7"
Jul 28 23:22:04.773121 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.772813    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/82ce0587-81a8-4b5e-b841-14fed1b2364c-tmp-dir\") pod \"node-resolver-xqqk5\" (UID: \"82ce0587-81a8-4b5e-b841-14fed1b2364c\") " pod="openshift-dns/node-resolver-xqqk5"
Jul 28 23:22:04.774250 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.774230    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/aa22bd20-02b0-4d17-90b8-b0607ef1d16d-tmp\") pod \"tuned-jx5sf\" (UID: \"aa22bd20-02b0-4d17-90b8-b0607ef1d16d\") " pod="openshift-cluster-node-tuning-operator/tuned-jx5sf"
Jul 28 23:22:04.774422 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.774400    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/aa22bd20-02b0-4d17-90b8-b0607ef1d16d-etc-tuned\") pod \"tuned-jx5sf\" (UID: \"aa22bd20-02b0-4d17-90b8-b0607ef1d16d\") " pod="openshift-cluster-node-tuning-operator/tuned-jx5sf"
Jul 28 23:22:04.782360 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.782330    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nt5jx\" (UniqueName: \"kubernetes.io/projected/aa22bd20-02b0-4d17-90b8-b0607ef1d16d-kube-api-access-nt5jx\") pod \"tuned-jx5sf\" (UID: \"aa22bd20-02b0-4d17-90b8-b0607ef1d16d\") " pod="openshift-cluster-node-tuning-operator/tuned-jx5sf"
Jul 28 23:22:04.782360 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.782344    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rfmc5\" (UniqueName: \"kubernetes.io/projected/82ce0587-81a8-4b5e-b841-14fed1b2364c-kube-api-access-rfmc5\") pod \"node-resolver-xqqk5\" (UID: \"82ce0587-81a8-4b5e-b841-14fed1b2364c\") " pod="openshift-dns/node-resolver-xqqk5"
Jul 28 23:22:04.862446 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:04.862410    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48ad21d512355ab82436faafc0eae6b0.slice/crio-d515c0c89d5fb11ea86b48b8c4e20286b6309a6fccce864ec3cdc1f65257d141 WatchSource:0}: Error finding container d515c0c89d5fb11ea86b48b8c4e20286b6309a6fccce864ec3cdc1f65257d141: Status 404 returned error can't find the container with id d515c0c89d5fb11ea86b48b8c4e20286b6309a6fccce864ec3cdc1f65257d141
Jul 28 23:22:04.863104 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:04.863076    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb93e6ceeada8d8a42bfd36cc63ee14d.slice/crio-55e663bb96f2d3a2f4e851be4592c94d4940ecc365c136acf255ed4a829baea5 WatchSource:0}: Error finding container 55e663bb96f2d3a2f4e851be4592c94d4940ecc365c136acf255ed4a829baea5: Status 404 returned error can't find the container with id 55e663bb96f2d3a2f4e851be4592c94d4940ecc365c136acf255ed4a829baea5
Jul 28 23:22:04.867557 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.867543    2572 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 23:22:04.872383 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.872363    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bd3f4ca7-06e3-4857-b42c-32c1a90fd147-host-cni-bin\") pod \"ovnkube-node-v84z7\" (UID: \"bd3f4ca7-06e3-4857-b42c-32c1a90fd147\") " pod="openshift-ovn-kubernetes/ovnkube-node-v84z7"
Jul 28 23:22:04.872498 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.872396    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/bd3f4ca7-06e3-4857-b42c-32c1a90fd147-host-cni-netd\") pod \"ovnkube-node-v84z7\" (UID: \"bd3f4ca7-06e3-4857-b42c-32c1a90fd147\") " pod="openshift-ovn-kubernetes/ovnkube-node-v84z7"
Jul 28 23:22:04.872498 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.872420    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f0137726-3966-44f4-bcc7-c69cc72143be-original-pull-secret\") pod \"global-pull-secret-syncer-8pczd\" (UID: \"f0137726-3966-44f4-bcc7-c69cc72143be\") " pod="kube-system/global-pull-secret-syncer-8pczd"
Jul 28 23:22:04.872498 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.872474    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/bd3f4ca7-06e3-4857-b42c-32c1a90fd147-host-cni-netd\") pod \"ovnkube-node-v84z7\" (UID: \"bd3f4ca7-06e3-4857-b42c-32c1a90fd147\") " pod="openshift-ovn-kubernetes/ovnkube-node-v84z7"
Jul 28 23:22:04.872680 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:22:04.872516    2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 23:22:04.872680 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.872518    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/dc10426a-a32b-4910-a2c8-21345f7b691b-hostroot\") pod \"multus-l85q7\" (UID: \"dc10426a-a32b-4910-a2c8-21345f7b691b\") " pod="openshift-multus/multus-l85q7"
Jul 28 23:22:04.872680 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.872534    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bd3f4ca7-06e3-4857-b42c-32c1a90fd147-host-cni-bin\") pod \"ovnkube-node-v84z7\" (UID: \"bd3f4ca7-06e3-4857-b42c-32c1a90fd147\") " pod="openshift-ovn-kubernetes/ovnkube-node-v84z7"
Jul 28 23:22:04.872680 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.872545    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/dc10426a-a32b-4910-a2c8-21345f7b691b-multus-daemon-config\") pod \"multus-l85q7\" (UID: \"dc10426a-a32b-4910-a2c8-21345f7b691b\") " pod="openshift-multus/multus-l85q7"
Jul 28 23:22:04.872680 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:22:04.872590    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f0137726-3966-44f4-bcc7-c69cc72143be-original-pull-secret podName:f0137726-3966-44f4-bcc7-c69cc72143be nodeName:}" failed. No retries permitted until 2026-07-28 23:22:05.372567845 +0000 UTC m=+2.065242351 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f0137726-3966-44f4-bcc7-c69cc72143be-original-pull-secret") pod "global-pull-secret-syncer-8pczd" (UID: "f0137726-3966-44f4-bcc7-c69cc72143be") : object "kube-system"/"original-pull-secret" not registered
Jul 28 23:22:04.872680 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.872592    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/dc10426a-a32b-4910-a2c8-21345f7b691b-hostroot\") pod \"multus-l85q7\" (UID: \"dc10426a-a32b-4910-a2c8-21345f7b691b\") " pod="openshift-multus/multus-l85q7"
Jul 28 23:22:04.872680 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.872610    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/bd3f4ca7-06e3-4857-b42c-32c1a90fd147-ovnkube-script-lib\") pod \"ovnkube-node-v84z7\" (UID: \"bd3f4ca7-06e3-4857-b42c-32c1a90fd147\") " pod="openshift-ovn-kubernetes/ovnkube-node-v84z7"
Jul 28 23:22:04.872680 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.872660    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/25a639de-470a-46a0-bad9-c5e646731566-registration-dir\") pod \"aws-ebs-csi-driver-node-c59s9\" (UID: \"25a639de-470a-46a0-bad9-c5e646731566\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c59s9"
Jul 28 23:22:04.873076 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.872687    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/25a639de-470a-46a0-bad9-c5e646731566-device-dir\") pod \"aws-ebs-csi-driver-node-c59s9\" (UID: \"25a639de-470a-46a0-bad9-c5e646731566\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c59s9"
Jul 28 23:22:04.873076 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.872711    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f220827f-670e-407f-b5c9-664800ba743d-cnibin\") pod \"multus-additional-cni-plugins-v6nmr\" (UID: \"f220827f-670e-407f-b5c9-664800ba743d\") " pod="openshift-multus/multus-additional-cni-plugins-v6nmr"
Jul 28 23:22:04.873076 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.872755    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/25a639de-470a-46a0-bad9-c5e646731566-registration-dir\") pod \"aws-ebs-csi-driver-node-c59s9\" (UID: \"25a639de-470a-46a0-bad9-c5e646731566\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c59s9"
Jul 28 23:22:04.873076 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.872765    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f220827f-670e-407f-b5c9-664800ba743d-cnibin\") pod \"multus-additional-cni-plugins-v6nmr\" (UID: \"f220827f-670e-407f-b5c9-664800ba743d\") " pod="openshift-multus/multus-additional-cni-plugins-v6nmr"
Jul 28 23:22:04.873076 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.872763    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/25a639de-470a-46a0-bad9-c5e646731566-device-dir\") pod \"aws-ebs-csi-driver-node-c59s9\" (UID: \"25a639de-470a-46a0-bad9-c5e646731566\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c59s9"
Jul 28 23:22:04.873076 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.872783    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bd3f4ca7-06e3-4857-b42c-32c1a90fd147-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-v84z7\" (UID: \"bd3f4ca7-06e3-4857-b42c-32c1a90fd147\") " pod="openshift-ovn-kubernetes/ovnkube-node-v84z7"
Jul 28 23:22:04.873076 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.872808    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/dc10426a-a32b-4910-a2c8-21345f7b691b-os-release\") pod \"multus-l85q7\" (UID: \"dc10426a-a32b-4910-a2c8-21345f7b691b\") " pod="openshift-multus/multus-l85q7"
Jul 28 23:22:04.873076 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.872828    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bd3f4ca7-06e3-4857-b42c-32c1a90fd147-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-v84z7\" (UID: \"bd3f4ca7-06e3-4857-b42c-32c1a90fd147\") " pod="openshift-ovn-kubernetes/ovnkube-node-v84z7"
Jul 28 23:22:04.873076 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.872836    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/f0137726-3966-44f4-bcc7-c69cc72143be-kubelet-config\") pod \"global-pull-secret-syncer-8pczd\" (UID: \"f0137726-3966-44f4-bcc7-c69cc72143be\") " pod="kube-system/global-pull-secret-syncer-8pczd"
Jul 28 23:22:04.873076 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.872853    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-khr8t\" (UniqueName: \"kubernetes.io/projected/df9e50e4-7650-40bb-83eb-becf4ba8c051-kube-api-access-khr8t\") pod \"network-metrics-daemon-lpj5x\" (UID: \"df9e50e4-7650-40bb-83eb-becf4ba8c051\") " pod="openshift-multus/network-metrics-daemon-lpj5x"
Jul 28 23:22:04.873076 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.872869    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/b923a656-b1ea-41e7-8c65-440a24abcee6-konnectivity-ca\") pod \"konnectivity-agent-6dwz5\" (UID: \"b923a656-b1ea-41e7-8c65-440a24abcee6\") " pod="kube-system/konnectivity-agent-6dwz5"
Jul 28 23:22:04.873076 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.872883    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bd3f4ca7-06e3-4857-b42c-32c1a90fd147-host-run-netns\") pod \"ovnkube-node-v84z7\" (UID: \"bd3f4ca7-06e3-4857-b42c-32c1a90fd147\") " pod="openshift-ovn-kubernetes/ovnkube-node-v84z7"
Jul 28 23:22:04.873076 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.872895    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/dc10426a-a32b-4910-a2c8-21345f7b691b-os-release\") pod \"multus-l85q7\" (UID: \"dc10426a-a32b-4910-a2c8-21345f7b691b\") " pod="openshift-multus/multus-l85q7"
Jul 28 23:22:04.873076 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.872901    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bd3f4ca7-06e3-4857-b42c-32c1a90fd147-var-lib-openvswitch\") pod \"ovnkube-node-v84z7\" (UID: \"bd3f4ca7-06e3-4857-b42c-32c1a90fd147\") " pod="openshift-ovn-kubernetes/ovnkube-node-v84z7"
Jul 28 23:22:04.873076 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.872928    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/bd3f4ca7-06e3-4857-b42c-32c1a90fd147-node-log\") pod \"ovnkube-node-v84z7\" (UID: \"bd3f4ca7-06e3-4857-b42c-32c1a90fd147\") " pod="openshift-ovn-kubernetes/ovnkube-node-v84z7"
Jul 28 23:22:04.873076 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.872935    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bd3f4ca7-06e3-4857-b42c-32c1a90fd147-var-lib-openvswitch\") pod \"ovnkube-node-v84z7\" (UID: \"bd3f4ca7-06e3-4857-b42c-32c1a90fd147\") " pod="openshift-ovn-kubernetes/ovnkube-node-v84z7"
Jul 28 23:22:04.873076 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.872957    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n7cs4\" (UniqueName: \"kubernetes.io/projected/6640efb8-6a16-44ee-b831-90f5ee40a30c-kube-api-access-n7cs4\") pod \"iptables-alerter-sltwc\" (UID: \"6640efb8-6a16-44ee-b831-90f5ee40a30c\") " pod="openshift-network-operator/iptables-alerter-sltwc"
Jul 28 23:22:04.873775 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.872974    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bd3f4ca7-06e3-4857-b42c-32c1a90fd147-host-run-netns\") pod \"ovnkube-node-v84z7\" (UID: \"bd3f4ca7-06e3-4857-b42c-32c1a90fd147\") " pod="openshift-ovn-kubernetes/ovnkube-node-v84z7"
Jul 28 23:22:04.873775 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.872983    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/bd3f4ca7-06e3-4857-b42c-32c1a90fd147-log-socket\") pod \"ovnkube-node-v84z7\" (UID: \"bd3f4ca7-06e3-4857-b42c-32c1a90fd147\") " pod="openshift-ovn-kubernetes/ovnkube-node-v84z7"
Jul 28 23:22:04.873775 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.872990    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/f0137726-3966-44f4-bcc7-c69cc72143be-kubelet-config\") pod \"global-pull-secret-syncer-8pczd\" (UID: \"f0137726-3966-44f4-bcc7-c69cc72143be\") " pod="kube-system/global-pull-secret-syncer-8pczd"
Jul 28 23:22:04.873775 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.873022    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/bd3f4ca7-06e3-4857-b42c-32c1a90fd147-log-socket\") pod \"ovnkube-node-v84z7\" (UID: \"bd3f4ca7-06e3-4857-b42c-32c1a90fd147\") " pod="openshift-ovn-kubernetes/ovnkube-node-v84z7"
Jul 28 23:22:04.873775 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.873022    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/dc10426a-a32b-4910-a2c8-21345f7b691b-host-run-netns\") pod \"multus-l85q7\" (UID: \"dc10426a-a32b-4910-a2c8-21345f7b691b\") " pod="openshift-multus/multus-l85q7"
Jul 28 23:22:04.873775 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.873053    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/bd3f4ca7-06e3-4857-b42c-32c1a90fd147-node-log\") pod \"ovnkube-node-v84z7\" (UID: \"bd3f4ca7-06e3-4857-b42c-32c1a90fd147\") " pod="openshift-ovn-kubernetes/ovnkube-node-v84z7"
Jul 28 23:22:04.873775 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.873055    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/6640efb8-6a16-44ee-b831-90f5ee40a30c-iptables-alerter-script\") pod \"iptables-alerter-sltwc\" (UID: \"6640efb8-6a16-44ee-b831-90f5ee40a30c\") " pod="openshift-network-operator/iptables-alerter-sltwc"
Jul 28 23:22:04.873775 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.873059    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/dc10426a-a32b-4910-a2c8-21345f7b691b-host-run-netns\") pod \"multus-l85q7\" (UID: \"dc10426a-a32b-4910-a2c8-21345f7b691b\") " pod="openshift-multus/multus-l85q7"
Jul 28 23:22:04.873775 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.873055    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/dc10426a-a32b-4910-a2c8-21345f7b691b-multus-daemon-config\") pod \"multus-l85q7\" (UID: \"dc10426a-a32b-4910-a2c8-21345f7b691b\") " pod="openshift-multus/multus-l85q7"
Jul 28 23:22:04.873775 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.873153    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6640efb8-6a16-44ee-b831-90f5ee40a30c-host-slash\") pod \"iptables-alerter-sltwc\" (UID: \"6640efb8-6a16-44ee-b831-90f5ee40a30c\") " pod="openshift-network-operator/iptables-alerter-sltwc"
Jul 28 23:22:04.873775 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.873181    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5c9cc37c-daaf-4fa1-b71e-94bcc6732adb-host\") pod \"node-ca-5ndfr\" (UID: \"5c9cc37c-daaf-4fa1-b71e-94bcc6732adb\") " pod="openshift-image-registry/node-ca-5ndfr"
Jul 28 23:22:04.873775 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.873189    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6640efb8-6a16-44ee-b831-90f5ee40a30c-host-slash\") pod \"iptables-alerter-sltwc\" (UID: \"6640efb8-6a16-44ee-b831-90f5ee40a30c\") " pod="openshift-network-operator/iptables-alerter-sltwc"
Jul 28 23:22:04.873775 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.873209    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/bd3f4ca7-06e3-4857-b42c-32c1a90fd147-systemd-units\") pod \"ovnkube-node-v84z7\" (UID: \"bd3f4ca7-06e3-4857-b42c-32c1a90fd147\") " pod="openshift-ovn-kubernetes/ovnkube-node-v84z7"
Jul 28 23:22:04.873775 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.873196    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/bd3f4ca7-06e3-4857-b42c-32c1a90fd147-ovnkube-script-lib\") pod \"ovnkube-node-v84z7\" (UID: \"bd3f4ca7-06e3-4857-b42c-32c1a90fd147\") " pod="openshift-ovn-kubernetes/ovnkube-node-v84z7"
Jul 28 23:22:04.873775 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.873243    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/bd3f4ca7-06e3-4857-b42c-32c1a90fd147-systemd-units\") pod \"ovnkube-node-v84z7\" (UID: \"bd3f4ca7-06e3-4857-b42c-32c1a90fd147\") " pod="openshift-ovn-kubernetes/ovnkube-node-v84z7"
Jul 28 23:22:04.873775 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.873250    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5c9cc37c-daaf-4fa1-b71e-94bcc6732adb-host\") pod \"node-ca-5ndfr\" (UID: \"5c9cc37c-daaf-4fa1-b71e-94bcc6732adb\") " pod="openshift-image-registry/node-ca-5ndfr"
Jul 28 23:22:04.873775 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.873243    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n2jnx\" (UniqueName: \"kubernetes.io/projected/ae951cd2-ca5f-4ac4-abaa-3293c8f1ea57-kube-api-access-n2jnx\") pod \"network-check-target-2svls\" (UID: \"ae951cd2-ca5f-4ac4-abaa-3293c8f1ea57\") " pod="openshift-network-diagnostics/network-check-target-2svls"
Jul 28 23:22:04.873775 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.873294    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/25a639de-470a-46a0-bad9-c5e646731566-etc-selinux\") pod \"aws-ebs-csi-driver-node-c59s9\" (UID: \"25a639de-470a-46a0-bad9-c5e646731566\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c59s9"
Jul 28 23:22:04.874640 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.873313    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/dc10426a-a32b-4910-a2c8-21345f7b691b-host-var-lib-cni-multus\") pod \"multus-l85q7\" (UID: \"dc10426a-a32b-4910-a2c8-21345f7b691b\") " pod="openshift-multus/multus-l85q7"
Jul 28 23:22:04.874640 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.873336    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f220827f-670e-407f-b5c9-664800ba743d-system-cni-dir\") pod \"multus-additional-cni-plugins-v6nmr\" (UID: \"f220827f-670e-407f-b5c9-664800ba743d\") " pod="openshift-multus/multus-additional-cni-plugins-v6nmr"
Jul 28 23:22:04.874640 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.873359    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/dc10426a-a32b-4910-a2c8-21345f7b691b-host-var-lib-cni-multus\") pod \"multus-l85q7\" (UID: \"dc10426a-a32b-4910-a2c8-21345f7b691b\") " pod="openshift-multus/multus-l85q7"
Jul 28 23:22:04.874640 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.873389    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/b923a656-b1ea-41e7-8c65-440a24abcee6-konnectivity-ca\") pod \"konnectivity-agent-6dwz5\" (UID: \"b923a656-b1ea-41e7-8c65-440a24abcee6\") " pod="kube-system/konnectivity-agent-6dwz5"
Jul 28 23:22:04.874640 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.873400    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/bd3f4ca7-06e3-4857-b42c-32c1a90fd147-run-systemd\") pod \"ovnkube-node-v84z7\" (UID: \"bd3f4ca7-06e3-4857-b42c-32c1a90fd147\") " pod="openshift-ovn-kubernetes/ovnkube-node-v84z7"
Jul 28 23:22:04.874640 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.873403    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/25a639de-470a-46a0-bad9-c5e646731566-etc-selinux\") pod \"aws-ebs-csi-driver-node-c59s9\" (UID: \"25a639de-470a-46a0-bad9-c5e646731566\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c59s9"
Jul 28 23:22:04.874640 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.873436    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dc10426a-a32b-4910-a2c8-21345f7b691b-multus-cni-dir\") pod \"multus-l85q7\" (UID: \"dc10426a-a32b-4910-a2c8-21345f7b691b\") " pod="openshift-multus/multus-l85q7"
Jul 28 23:22:04.874640 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.873455    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/bd3f4ca7-06e3-4857-b42c-32c1a90fd147-run-systemd\") pod \"ovnkube-node-v84z7\" (UID: \"bd3f4ca7-06e3-4857-b42c-32c1a90fd147\") " pod="openshift-ovn-kubernetes/ovnkube-node-v84z7"
Jul 28 23:22:04.874640 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.873461    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5c9cc37c-daaf-4fa1-b71e-94bcc6732adb-serviceca\") pod \"node-ca-5ndfr\" (UID: \"5c9cc37c-daaf-4fa1-b71e-94bcc6732adb\") " pod="openshift-image-registry/node-ca-5ndfr"
Jul 28 23:22:04.874640 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.873489    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/25a639de-470a-46a0-bad9-c5e646731566-sys-fs\") pod \"aws-ebs-csi-driver-node-c59s9\" (UID: \"25a639de-470a-46a0-bad9-c5e646731566\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c59s9"
Jul 28 23:22:04.874640 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.873516    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bd3f4ca7-06e3-4857-b42c-32c1a90fd147-run-openvswitch\") pod \"ovnkube-node-v84z7\" (UID: \"bd3f4ca7-06e3-4857-b42c-32c1a90fd147\") " pod="openshift-ovn-kubernetes/ovnkube-node-v84z7"
Jul 28 23:22:04.874640 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.873541    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dc10426a-a32b-4910-a2c8-21345f7b691b-multus-cni-dir\") pod \"multus-l85q7\" (UID: \"dc10426a-a32b-4910-a2c8-21345f7b691b\") " pod="openshift-multus/multus-l85q7"
Jul 28 23:22:04.874640 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.873541    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bd3f4ca7-06e3-4857-b42c-32c1a90fd147-host-run-ovn-kubernetes\") pod \"ovnkube-node-v84z7\" (UID: \"bd3f4ca7-06e3-4857-b42c-32c1a90fd147\") " pod="openshift-ovn-kubernetes/ovnkube-node-v84z7"
Jul 28 23:22:04.874640 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.873558    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/25a639de-470a-46a0-bad9-c5e646731566-sys-fs\") pod \"aws-ebs-csi-driver-node-c59s9\" (UID: \"25a639de-470a-46a0-bad9-c5e646731566\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c59s9"
Jul 28 23:22:04.874640 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.873565    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/6640efb8-6a16-44ee-b831-90f5ee40a30c-iptables-alerter-script\") pod \"iptables-alerter-sltwc\" (UID: \"6640efb8-6a16-44ee-b831-90f5ee40a30c\") " pod="openshift-network-operator/iptables-alerter-sltwc"
Jul 28 23:22:04.874640 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.873578    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bd3f4ca7-06e3-4857-b42c-32c1a90fd147-host-run-ovn-kubernetes\") pod \"ovnkube-node-v84z7\" (UID: \"bd3f4ca7-06e3-4857-b42c-32c1a90fd147\") " pod="openshift-ovn-kubernetes/ovnkube-node-v84z7"
Jul 28 23:22:04.874640 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.873584    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/df9e50e4-7650-40bb-83eb-becf4ba8c051-metrics-certs\") pod \"network-metrics-daemon-lpj5x\" (UID: \"df9e50e4-7650-40bb-83eb-becf4ba8c051\") " pod="openshift-multus/network-metrics-daemon-lpj5x"
Jul 28 23:22:04.875325 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.873437    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f220827f-670e-407f-b5c9-664800ba743d-system-cni-dir\") pod \"multus-additional-cni-plugins-v6nmr\" (UID: \"f220827f-670e-407f-b5c9-664800ba743d\") " pod="openshift-multus/multus-additional-cni-plugins-v6nmr"
Jul 28 23:22:04.875325 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.873610    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bd3f4ca7-06e3-4857-b42c-32c1a90fd147-run-openvswitch\") pod \"ovnkube-node-v84z7\" (UID: \"bd3f4ca7-06e3-4857-b42c-32c1a90fd147\") " pod="openshift-ovn-kubernetes/ovnkube-node-v84z7"
Jul 28 23:22:04.875325 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.873664    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/dc10426a-a32b-4910-a2c8-21345f7b691b-host-run-k8s-cni-cncf-io\") pod \"multus-l85q7\" (UID: \"dc10426a-a32b-4910-a2c8-21345f7b691b\") " pod="openshift-multus/multus-l85q7"
Jul 28 23:22:04.875325 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:22:04.873692    2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 23:22:04.875325 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.873724    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/dc10426a-a32b-4910-a2c8-21345f7b691b-host-run-multus-certs\") pod \"multus-l85q7\" (UID: \"dc10426a-a32b-4910-a2c8-21345f7b691b\") " pod="openshift-multus/multus-l85q7"
Jul 28 23:22:04.875325 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:22:04.873733    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df9e50e4-7650-40bb-83eb-becf4ba8c051-metrics-certs podName:df9e50e4-7650-40bb-83eb-becf4ba8c051 nodeName:}" failed. No retries permitted until 2026-07-28 23:22:05.37371825 +0000 UTC m=+2.066392736 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/df9e50e4-7650-40bb-83eb-becf4ba8c051-metrics-certs") pod "network-metrics-daemon-lpj5x" (UID: "df9e50e4-7650-40bb-83eb-becf4ba8c051") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 23:22:04.875325 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.873742    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/dc10426a-a32b-4910-a2c8-21345f7b691b-host-run-k8s-cni-cncf-io\") pod \"multus-l85q7\" (UID: \"dc10426a-a32b-4910-a2c8-21345f7b691b\") " pod="openshift-multus/multus-l85q7"
Jul 28 23:22:04.875325 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.873775    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f220827f-670e-407f-b5c9-664800ba743d-tuning-conf-dir\") pod \"multus-additional-cni-plugins-v6nmr\" (UID: \"f220827f-670e-407f-b5c9-664800ba743d\") " pod="openshift-multus/multus-additional-cni-plugins-v6nmr"
Jul 28 23:22:04.875325 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.873791    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/dc10426a-a32b-4910-a2c8-21345f7b691b-host-run-multus-certs\") pod \"multus-l85q7\" (UID: \"dc10426a-a32b-4910-a2c8-21345f7b691b\") " pod="openshift-multus/multus-l85q7"
Jul 28 23:22:04.875325 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.873816    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5c9cc37c-daaf-4fa1-b71e-94bcc6732adb-serviceca\") pod \"node-ca-5ndfr\" (UID: \"5c9cc37c-daaf-4fa1-b71e-94bcc6732adb\") " pod="openshift-image-registry/node-ca-5ndfr"
Jul 28 23:22:04.875325 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.873839    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bd3f4ca7-06e3-4857-b42c-32c1a90fd147-etc-openvswitch\") pod \"ovnkube-node-v84z7\" (UID: \"bd3f4ca7-06e3-4857-b42c-32c1a90fd147\") " pod="openshift-ovn-kubernetes/ovnkube-node-v84z7"
Jul 28 23:22:04.875325 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.873856    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/bd3f4ca7-06e3-4857-b42c-32c1a90fd147-env-overrides\") pod \"ovnkube-node-v84z7\" (UID: \"bd3f4ca7-06e3-4857-b42c-32c1a90fd147\") " pod="openshift-ovn-kubernetes/ovnkube-node-v84z7"
Jul 28 23:22:04.875325 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.873873    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/25a639de-470a-46a0-bad9-c5e646731566-socket-dir\") pod \"aws-ebs-csi-driver-node-c59s9\" (UID: \"25a639de-470a-46a0-bad9-c5e646731566\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c59s9"
Jul 28 23:22:04.875325 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.873887    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/dc10426a-a32b-4910-a2c8-21345f7b691b-cni-binary-copy\") pod \"multus-l85q7\" (UID: \"dc10426a-a32b-4910-a2c8-21345f7b691b\") " pod="openshift-multus/multus-l85q7"
Jul 28 23:22:04.875325 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.873924    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f220827f-670e-407f-b5c9-664800ba743d-tuning-conf-dir\") pod \"multus-additional-cni-plugins-v6nmr\" (UID: \"f220827f-670e-407f-b5c9-664800ba743d\") " pod="openshift-multus/multus-additional-cni-plugins-v6nmr"
Jul 28 23:22:04.875325 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.873952    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/dc10426a-a32b-4910-a2c8-21345f7b691b-host-var-lib-kubelet\") pod \"multus-l85q7\" (UID: \"dc10426a-a32b-4910-a2c8-21345f7b691b\") " pod="openshift-multus/multus-l85q7"
Jul 28 23:22:04.875325 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.873958    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bd3f4ca7-06e3-4857-b42c-32c1a90fd147-etc-openvswitch\") pod \"ovnkube-node-v84z7\" (UID: \"bd3f4ca7-06e3-4857-b42c-32c1a90fd147\") " pod="openshift-ovn-kubernetes/ovnkube-node-v84z7"
Jul 28 23:22:04.875829 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.873991    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lf2sg\" (UniqueName: \"kubernetes.io/projected/5c9cc37c-daaf-4fa1-b71e-94bcc6732adb-kube-api-access-lf2sg\") pod \"node-ca-5ndfr\" (UID: \"5c9cc37c-daaf-4fa1-b71e-94bcc6732adb\") " pod="openshift-image-registry/node-ca-5ndfr"
Jul 28 23:22:04.875829 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.873997    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/dc10426a-a32b-4910-a2c8-21345f7b691b-host-var-lib-kubelet\") pod \"multus-l85q7\" (UID: \"dc10426a-a32b-4910-a2c8-21345f7b691b\") " pod="openshift-multus/multus-l85q7"
Jul 28 23:22:04.875829 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.874022    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bdc9z\" (UniqueName: \"kubernetes.io/projected/25a639de-470a-46a0-bad9-c5e646731566-kube-api-access-bdc9z\") pod \"aws-ebs-csi-driver-node-c59s9\" (UID: \"25a639de-470a-46a0-bad9-c5e646731566\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c59s9"
Jul 28 23:22:04.875829 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.874047    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/dc10426a-a32b-4910-a2c8-21345f7b691b-multus-socket-dir-parent\") pod \"multus-l85q7\" (UID: \"dc10426a-a32b-4910-a2c8-21345f7b691b\") " pod="openshift-multus/multus-l85q7"
Jul 28 23:22:04.875829 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.874070    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/dc10426a-a32b-4910-a2c8-21345f7b691b-host-var-lib-cni-bin\") pod \"multus-l85q7\" (UID: \"dc10426a-a32b-4910-a2c8-21345f7b691b\") " pod="openshift-multus/multus-l85q7"
Jul 28 23:22:04.875829 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.874023    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/25a639de-470a-46a0-bad9-c5e646731566-socket-dir\") pod \"aws-ebs-csi-driver-node-c59s9\" (UID: \"25a639de-470a-46a0-bad9-c5e646731566\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c59s9"
Jul 28 23:22:04.875829 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.874096    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cmmbl\" (UniqueName: \"kubernetes.io/projected/f220827f-670e-407f-b5c9-664800ba743d-kube-api-access-cmmbl\") pod \"multus-additional-cni-plugins-v6nmr\" (UID: \"f220827f-670e-407f-b5c9-664800ba743d\") " pod="openshift-multus/multus-additional-cni-plugins-v6nmr"
Jul 28 23:22:04.875829 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.874127    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bd3f4ca7-06e3-4857-b42c-32c1a90fd147-host-slash\") pod \"ovnkube-node-v84z7\" (UID: \"bd3f4ca7-06e3-4857-b42c-32c1a90fd147\") " pod="openshift-ovn-kubernetes/ovnkube-node-v84z7"
Jul 28 23:22:04.875829 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.874141    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/dc10426a-a32b-4910-a2c8-21345f7b691b-multus-socket-dir-parent\") pod \"multus-l85q7\" (UID: \"dc10426a-a32b-4910-a2c8-21345f7b691b\") " pod="openshift-multus/multus-l85q7"
Jul 28 23:22:04.875829 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.874151    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/bd3f4ca7-06e3-4857-b42c-32c1a90fd147-ovnkube-config\") pod \"ovnkube-node-v84z7\" (UID: \"bd3f4ca7-06e3-4857-b42c-32c1a90fd147\") " pod="openshift-ovn-kubernetes/ovnkube-node-v84z7"
Jul 28 23:22:04.875829 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.874175    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dc10426a-a32b-4910-a2c8-21345f7b691b-etc-kubernetes\") pod \"multus-l85q7\" (UID: \"dc10426a-a32b-4910-a2c8-21345f7b691b\") " pod="openshift-multus/multus-l85q7"
Jul 28 23:22:04.875829 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.874199    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f220827f-670e-407f-b5c9-664800ba743d-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-v6nmr\" (UID: \"f220827f-670e-407f-b5c9-664800ba743d\") " pod="openshift-multus/multus-additional-cni-plugins-v6nmr"
Jul 28 23:22:04.875829 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.874230    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/bd3f4ca7-06e3-4857-b42c-32c1a90fd147-run-ovn\") pod \"ovnkube-node-v84z7\" (UID: \"bd3f4ca7-06e3-4857-b42c-32c1a90fd147\") " pod="openshift-ovn-kubernetes/ovnkube-node-v84z7"
Jul 28 23:22:04.875829 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.874239    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/bd3f4ca7-06e3-4857-b42c-32c1a90fd147-env-overrides\") pod \"ovnkube-node-v84z7\" (UID: \"bd3f4ca7-06e3-4857-b42c-32c1a90fd147\") " pod="openshift-ovn-kubernetes/ovnkube-node-v84z7"
Jul 28 23:22:04.875829 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.874254    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/bd3f4ca7-06e3-4857-b42c-32c1a90fd147-ovn-node-metrics-cert\") pod \"ovnkube-node-v84z7\" (UID: \"bd3f4ca7-06e3-4857-b42c-32c1a90fd147\") " pod="openshift-ovn-kubernetes/ovnkube-node-v84z7"
Jul 28 23:22:04.875829 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.874202    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bd3f4ca7-06e3-4857-b42c-32c1a90fd147-host-slash\") pod \"ovnkube-node-v84z7\" (UID: \"bd3f4ca7-06e3-4857-b42c-32c1a90fd147\") " pod="openshift-ovn-kubernetes/ovnkube-node-v84z7"
Jul 28 23:22:04.875829 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.874324    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dc10426a-a32b-4910-a2c8-21345f7b691b-etc-kubernetes\") pod \"multus-l85q7\" (UID: \"dc10426a-a32b-4910-a2c8-21345f7b691b\") " pod="openshift-multus/multus-l85q7"
Jul 28 23:22:04.876326 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.874328    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/dc10426a-a32b-4910-a2c8-21345f7b691b-cni-binary-copy\") pod \"multus-l85q7\" (UID: \"dc10426a-a32b-4910-a2c8-21345f7b691b\") " pod="openshift-multus/multus-l85q7"
Jul 28 23:22:04.876326 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.874328    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/bd3f4ca7-06e3-4857-b42c-32c1a90fd147-run-ovn\") pod \"ovnkube-node-v84z7\" (UID: \"bd3f4ca7-06e3-4857-b42c-32c1a90fd147\") " pod="openshift-ovn-kubernetes/ovnkube-node-v84z7"
Jul 28 23:22:04.876326 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.874352    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f220827f-670e-407f-b5c9-664800ba743d-os-release\") pod \"multus-additional-cni-plugins-v6nmr\" (UID: \"f220827f-670e-407f-b5c9-664800ba743d\") " pod="openshift-multus/multus-additional-cni-plugins-v6nmr"
Jul 28 23:22:04.876326 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.874356    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/dc10426a-a32b-4910-a2c8-21345f7b691b-host-var-lib-cni-bin\") pod \"multus-l85q7\" (UID: \"dc10426a-a32b-4910-a2c8-21345f7b691b\") " pod="openshift-multus/multus-l85q7"
Jul 28 23:22:04.876326 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.874379    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f220827f-670e-407f-b5c9-664800ba743d-cni-binary-copy\") pod \"multus-additional-cni-plugins-v6nmr\" (UID: \"f220827f-670e-407f-b5c9-664800ba743d\") " pod="openshift-multus/multus-additional-cni-plugins-v6nmr"
Jul 28 23:22:04.876326 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.874403    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/25a639de-470a-46a0-bad9-c5e646731566-kubelet-dir\") pod \"aws-ebs-csi-driver-node-c59s9\" (UID: \"25a639de-470a-46a0-bad9-c5e646731566\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c59s9"
Jul 28 23:22:04.876326 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.874408    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f220827f-670e-407f-b5c9-664800ba743d-os-release\") pod \"multus-additional-cni-plugins-v6nmr\" (UID: \"f220827f-670e-407f-b5c9-664800ba743d\") " pod="openshift-multus/multus-additional-cni-plugins-v6nmr"
Jul 28 23:22:04.876326 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.874427    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/dc10426a-a32b-4910-a2c8-21345f7b691b-multus-conf-dir\") pod \"multus-l85q7\" (UID: \"dc10426a-a32b-4910-a2c8-21345f7b691b\") " pod="openshift-multus/multus-l85q7"
Jul 28 23:22:04.876326 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.874453    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j5m76\" (UniqueName: \"kubernetes.io/projected/dc10426a-a32b-4910-a2c8-21345f7b691b-kube-api-access-j5m76\") pod \"multus-l85q7\" (UID: \"dc10426a-a32b-4910-a2c8-21345f7b691b\") " pod="openshift-multus/multus-l85q7"
Jul 28 23:22:04.876326 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.874460    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/25a639de-470a-46a0-bad9-c5e646731566-kubelet-dir\") pod \"aws-ebs-csi-driver-node-c59s9\" (UID: \"25a639de-470a-46a0-bad9-c5e646731566\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c59s9"
Jul 28 23:22:04.876326 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.874479    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f220827f-670e-407f-b5c9-664800ba743d-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-v6nmr\" (UID: \"f220827f-670e-407f-b5c9-664800ba743d\") " pod="openshift-multus/multus-additional-cni-plugins-v6nmr"
Jul 28 23:22:04.876326 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.874501    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/dc10426a-a32b-4910-a2c8-21345f7b691b-multus-conf-dir\") pod \"multus-l85q7\" (UID: \"dc10426a-a32b-4910-a2c8-21345f7b691b\") " pod="openshift-multus/multus-l85q7"
Jul 28 23:22:04.876326 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.874512    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/bd3f4ca7-06e3-4857-b42c-32c1a90fd147-host-kubelet\") pod \"ovnkube-node-v84z7\" (UID: \"bd3f4ca7-06e3-4857-b42c-32c1a90fd147\") " pod="openshift-ovn-kubernetes/ovnkube-node-v84z7"
Jul 28 23:22:04.876326 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.874536    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9pp2n\" (UniqueName: \"kubernetes.io/projected/bd3f4ca7-06e3-4857-b42c-32c1a90fd147-kube-api-access-9pp2n\") pod \"ovnkube-node-v84z7\" (UID: \"bd3f4ca7-06e3-4857-b42c-32c1a90fd147\") " pod="openshift-ovn-kubernetes/ovnkube-node-v84z7"
Jul 28 23:22:04.876326 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.874563    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/f0137726-3966-44f4-bcc7-c69cc72143be-dbus\") pod \"global-pull-secret-syncer-8pczd\" (UID: \"f0137726-3966-44f4-bcc7-c69cc72143be\") " pod="kube-system/global-pull-secret-syncer-8pczd"
Jul 28 23:22:04.876326 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.874588    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/b923a656-b1ea-41e7-8c65-440a24abcee6-agent-certs\") pod \"konnectivity-agent-6dwz5\" (UID: \"b923a656-b1ea-41e7-8c65-440a24abcee6\") " pod="kube-system/konnectivity-agent-6dwz5"
Jul 28 23:22:04.876326 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.874610    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/dc10426a-a32b-4910-a2c8-21345f7b691b-cnibin\") pod \"multus-l85q7\" (UID: \"dc10426a-a32b-4910-a2c8-21345f7b691b\") " pod="openshift-multus/multus-l85q7"
Jul 28 23:22:04.876876 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.874659    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dc10426a-a32b-4910-a2c8-21345f7b691b-system-cni-dir\") pod \"multus-l85q7\" (UID: \"dc10426a-a32b-4910-a2c8-21345f7b691b\") " pod="openshift-multus/multus-l85q7"
Jul 28 23:22:04.876876 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.874710    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/bd3f4ca7-06e3-4857-b42c-32c1a90fd147-ovnkube-config\") pod \"ovnkube-node-v84z7\" (UID: \"bd3f4ca7-06e3-4857-b42c-32c1a90fd147\") " pod="openshift-ovn-kubernetes/ovnkube-node-v84z7"
Jul 28 23:22:04.876876 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.874735    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dc10426a-a32b-4910-a2c8-21345f7b691b-system-cni-dir\") pod \"multus-l85q7\" (UID: \"dc10426a-a32b-4910-a2c8-21345f7b691b\") " pod="openshift-multus/multus-l85q7"
Jul 28 23:22:04.876876 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.874760    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f220827f-670e-407f-b5c9-664800ba743d-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-v6nmr\" (UID: \"f220827f-670e-407f-b5c9-664800ba743d\") " pod="openshift-multus/multus-additional-cni-plugins-v6nmr"
Jul 28 23:22:04.876876 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.874777    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/dc10426a-a32b-4910-a2c8-21345f7b691b-cnibin\") pod \"multus-l85q7\" (UID: \"dc10426a-a32b-4910-a2c8-21345f7b691b\") " pod="openshift-multus/multus-l85q7"
Jul 28 23:22:04.876876 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.874765    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/f0137726-3966-44f4-bcc7-c69cc72143be-dbus\") pod \"global-pull-secret-syncer-8pczd\" (UID: \"f0137726-3966-44f4-bcc7-c69cc72143be\") " pod="kube-system/global-pull-secret-syncer-8pczd"
Jul 28 23:22:04.876876 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.874816    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/bd3f4ca7-06e3-4857-b42c-32c1a90fd147-host-kubelet\") pod \"ovnkube-node-v84z7\" (UID: \"bd3f4ca7-06e3-4857-b42c-32c1a90fd147\") " pod="openshift-ovn-kubernetes/ovnkube-node-v84z7"
Jul 28 23:22:04.876876 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.874962    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f220827f-670e-407f-b5c9-664800ba743d-cni-binary-copy\") pod \"multus-additional-cni-plugins-v6nmr\" (UID: \"f220827f-670e-407f-b5c9-664800ba743d\") " pod="openshift-multus/multus-additional-cni-plugins-v6nmr"
Jul 28 23:22:04.876876 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.874988    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f220827f-670e-407f-b5c9-664800ba743d-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-v6nmr\" (UID: \"f220827f-670e-407f-b5c9-664800ba743d\") " pod="openshift-multus/multus-additional-cni-plugins-v6nmr"
Jul 28 23:22:04.877256 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.876966    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/bd3f4ca7-06e3-4857-b42c-32c1a90fd147-ovn-node-metrics-cert\") pod \"ovnkube-node-v84z7\" (UID: \"bd3f4ca7-06e3-4857-b42c-32c1a90fd147\") " pod="openshift-ovn-kubernetes/ovnkube-node-v84z7"
Jul 28 23:22:04.877256 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.877136    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/b923a656-b1ea-41e7-8c65-440a24abcee6-agent-certs\") pod \"konnectivity-agent-6dwz5\" (UID: \"b923a656-b1ea-41e7-8c65-440a24abcee6\") " pod="kube-system/konnectivity-agent-6dwz5"
Jul 28 23:22:04.882539 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:22:04.882521    2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 23:22:04.882539 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:22:04.882541    2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 23:22:04.882690 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:22:04.882554    2572 projected.go:196] Error preparing data for projected volume kube-api-access-n2jnx for pod openshift-network-diagnostics/network-check-target-2svls: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 23:22:04.882690 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:22:04.882639    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ae951cd2-ca5f-4ac4-abaa-3293c8f1ea57-kube-api-access-n2jnx podName:ae951cd2-ca5f-4ac4-abaa-3293c8f1ea57 nodeName:}" failed. No retries permitted until 2026-07-28 23:22:05.382610315 +0000 UTC m=+2.075284815 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-n2jnx" (UniqueName: "kubernetes.io/projected/ae951cd2-ca5f-4ac4-abaa-3293c8f1ea57-kube-api-access-n2jnx") pod "network-check-target-2svls" (UID: "ae951cd2-ca5f-4ac4-abaa-3293c8f1ea57") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 23:22:04.884643 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.884597    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bdc9z\" (UniqueName: \"kubernetes.io/projected/25a639de-470a-46a0-bad9-c5e646731566-kube-api-access-bdc9z\") pod \"aws-ebs-csi-driver-node-c59s9\" (UID: \"25a639de-470a-46a0-bad9-c5e646731566\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c59s9"
Jul 28 23:22:04.885050 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.885017    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9pp2n\" (UniqueName: \"kubernetes.io/projected/bd3f4ca7-06e3-4857-b42c-32c1a90fd147-kube-api-access-9pp2n\") pod \"ovnkube-node-v84z7\" (UID: \"bd3f4ca7-06e3-4857-b42c-32c1a90fd147\") " pod="openshift-ovn-kubernetes/ovnkube-node-v84z7"
Jul 28 23:22:04.885373 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.885354    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lf2sg\" (UniqueName: \"kubernetes.io/projected/5c9cc37c-daaf-4fa1-b71e-94bcc6732adb-kube-api-access-lf2sg\") pod \"node-ca-5ndfr\" (UID: \"5c9cc37c-daaf-4fa1-b71e-94bcc6732adb\") " pod="openshift-image-registry/node-ca-5ndfr"
Jul 28 23:22:04.885571 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.885550    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cmmbl\" (UniqueName: \"kubernetes.io/projected/f220827f-670e-407f-b5c9-664800ba743d-kube-api-access-cmmbl\") pod \"multus-additional-cni-plugins-v6nmr\" (UID: \"f220827f-670e-407f-b5c9-664800ba743d\") " pod="openshift-multus/multus-additional-cni-plugins-v6nmr"
Jul 28 23:22:04.885656 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.885552    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j5m76\" (UniqueName: \"kubernetes.io/projected/dc10426a-a32b-4910-a2c8-21345f7b691b-kube-api-access-j5m76\") pod \"multus-l85q7\" (UID: \"dc10426a-a32b-4910-a2c8-21345f7b691b\") " pod="openshift-multus/multus-l85q7"
Jul 28 23:22:04.885744 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.885727    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n7cs4\" (UniqueName: \"kubernetes.io/projected/6640efb8-6a16-44ee-b831-90f5ee40a30c-kube-api-access-n7cs4\") pod \"iptables-alerter-sltwc\" (UID: \"6640efb8-6a16-44ee-b831-90f5ee40a30c\") " pod="openshift-network-operator/iptables-alerter-sltwc"
Jul 28 23:22:04.886238 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.886220    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-khr8t\" (UniqueName: \"kubernetes.io/projected/df9e50e4-7650-40bb-83eb-becf4ba8c051-kube-api-access-khr8t\") pod \"network-metrics-daemon-lpj5x\" (UID: \"df9e50e4-7650-40bb-83eb-becf4ba8c051\") " pod="openshift-multus/network-metrics-daemon-lpj5x"
Jul 28 23:22:04.988566 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.988541    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-jx5sf"
Jul 28 23:22:04.991608 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:04.991590    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-xqqk5"
Jul 28 23:22:04.998950 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:04.998923    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa22bd20_02b0_4d17_90b8_b0607ef1d16d.slice/crio-ba014dbb794be16759a4ea2b8872d7cdb9e716fab54c0dcbf93cd1426b9244fb WatchSource:0}: Error finding container ba014dbb794be16759a4ea2b8872d7cdb9e716fab54c0dcbf93cd1426b9244fb: Status 404 returned error can't find the container with id ba014dbb794be16759a4ea2b8872d7cdb9e716fab54c0dcbf93cd1426b9244fb
Jul 28 23:22:05.001783 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:05.001758    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82ce0587_81a8_4b5e_b841_14fed1b2364c.slice/crio-f54c35cab58a1882ddd63b9ba2486cc8f52c1d8078284f98dbe6ef3f4c6bd3d2 WatchSource:0}: Error finding container f54c35cab58a1882ddd63b9ba2486cc8f52c1d8078284f98dbe6ef3f4c6bd3d2: Status 404 returned error can't find the container with id f54c35cab58a1882ddd63b9ba2486cc8f52c1d8078284f98dbe6ef3f4c6bd3d2
Jul 28 23:22:05.022396 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:05.022374    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-l85q7"
Jul 28 23:22:05.030860 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:05.030837    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc10426a_a32b_4910_a2c8_21345f7b691b.slice/crio-e4fdf47991571d01cc4b0e138c400fff6f17ea0bf156f06dc5695c6e0e70f50c WatchSource:0}: Error finding container e4fdf47991571d01cc4b0e138c400fff6f17ea0bf156f06dc5695c6e0e70f50c: Status 404 returned error can't find the container with id e4fdf47991571d01cc4b0e138c400fff6f17ea0bf156f06dc5695c6e0e70f50c
Jul 28 23:22:05.038430 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:05.038411    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-sltwc"
Jul 28 23:22:05.046075 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:05.046056    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6640efb8_6a16_44ee_b831_90f5ee40a30c.slice/crio-2f073858dcc1133dcc41cf1ea2d7dabae5f4f093adfe0ca214a2d47573b6c38e WatchSource:0}: Error finding container 2f073858dcc1133dcc41cf1ea2d7dabae5f4f093adfe0ca214a2d47573b6c38e: Status 404 returned error can't find the container with id 2f073858dcc1133dcc41cf1ea2d7dabae5f4f093adfe0ca214a2d47573b6c38e
Jul 28 23:22:05.052682 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:05.052665    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-5ndfr"
Jul 28 23:22:05.060127 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:05.060108    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c9cc37c_daaf_4fa1_b71e_94bcc6732adb.slice/crio-65853ad2ade305ab0cce44e2ebfdd9f87425873503bc3bdd889c7c28788c937a WatchSource:0}: Error finding container 65853ad2ade305ab0cce44e2ebfdd9f87425873503bc3bdd889c7c28788c937a: Status 404 returned error can't find the container with id 65853ad2ade305ab0cce44e2ebfdd9f87425873503bc3bdd889c7c28788c937a
Jul 28 23:22:05.073764 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:05.073742    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-v6nmr"
Jul 28 23:22:05.081728 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:05.081701    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf220827f_670e_407f_b5c9_664800ba743d.slice/crio-5b133010880c1074cb930bd1a430304d65e03e2d18112f44c57d3c7647770094 WatchSource:0}: Error finding container 5b133010880c1074cb930bd1a430304d65e03e2d18112f44c57d3c7647770094: Status 404 returned error can't find the container with id 5b133010880c1074cb930bd1a430304d65e03e2d18112f44c57d3c7647770094
Jul 28 23:22:05.095832 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:05.095810    2572 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 23:22:05.099649 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:05.099615    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-v84z7"
Jul 28 23:22:05.107541 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:05.107521    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd3f4ca7_06e3_4857_b42c_32c1a90fd147.slice/crio-b1bf9e7d4908931f71e8bf29ae61f5d8d76db70815a7f412e1e0a3cd3635b9b2 WatchSource:0}: Error finding container b1bf9e7d4908931f71e8bf29ae61f5d8d76db70815a7f412e1e0a3cd3635b9b2: Status 404 returned error can't find the container with id b1bf9e7d4908931f71e8bf29ae61f5d8d76db70815a7f412e1e0a3cd3635b9b2
Jul 28 23:22:05.121337 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:05.121321    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-6dwz5"
Jul 28 23:22:05.127178 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:05.127157    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c59s9"
Jul 28 23:22:05.129554 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:05.129534    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb923a656_b1ea_41e7_8c65_440a24abcee6.slice/crio-de1b154ae5b5e5a85bc34817b733bad336d590a4a32346c53c3deeeda506555f WatchSource:0}: Error finding container de1b154ae5b5e5a85bc34817b733bad336d590a4a32346c53c3deeeda506555f: Status 404 returned error can't find the container with id de1b154ae5b5e5a85bc34817b733bad336d590a4a32346c53c3deeeda506555f
Jul 28 23:22:05.134857 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:05.134837    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod25a639de_470a_46a0_bad9_c5e646731566.slice/crio-4ca6b8164e0b75cf43a0f6d234aaff4e136db31fd78f12c29765b8fcf4b156a1 WatchSource:0}: Error finding container 4ca6b8164e0b75cf43a0f6d234aaff4e136db31fd78f12c29765b8fcf4b156a1: Status 404 returned error can't find the container with id 4ca6b8164e0b75cf43a0f6d234aaff4e136db31fd78f12c29765b8fcf4b156a1
Jul 28 23:22:05.378269 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:05.378190    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f0137726-3966-44f4-bcc7-c69cc72143be-original-pull-secret\") pod \"global-pull-secret-syncer-8pczd\" (UID: \"f0137726-3966-44f4-bcc7-c69cc72143be\") " pod="kube-system/global-pull-secret-syncer-8pczd"
Jul 28 23:22:05.378420 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:05.378281    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/df9e50e4-7650-40bb-83eb-becf4ba8c051-metrics-certs\") pod \"network-metrics-daemon-lpj5x\" (UID: \"df9e50e4-7650-40bb-83eb-becf4ba8c051\") " pod="openshift-multus/network-metrics-daemon-lpj5x"
Jul 28 23:22:05.378539 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:22:05.378429    2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 23:22:05.378539 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:22:05.378490    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df9e50e4-7650-40bb-83eb-becf4ba8c051-metrics-certs podName:df9e50e4-7650-40bb-83eb-becf4ba8c051 nodeName:}" failed. No retries permitted until 2026-07-28 23:22:06.378470423 +0000 UTC m=+3.071144926 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/df9e50e4-7650-40bb-83eb-becf4ba8c051-metrics-certs") pod "network-metrics-daemon-lpj5x" (UID: "df9e50e4-7650-40bb-83eb-becf4ba8c051") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 23:22:05.378896 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:22:05.378800    2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 23:22:05.378896 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:22:05.378875    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f0137726-3966-44f4-bcc7-c69cc72143be-original-pull-secret podName:f0137726-3966-44f4-bcc7-c69cc72143be nodeName:}" failed. No retries permitted until 2026-07-28 23:22:06.378858488 +0000 UTC m=+3.071532980 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f0137726-3966-44f4-bcc7-c69cc72143be-original-pull-secret") pod "global-pull-secret-syncer-8pczd" (UID: "f0137726-3966-44f4-bcc7-c69cc72143be") : object "kube-system"/"original-pull-secret" not registered
Jul 28 23:22:05.479107 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:05.479066    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n2jnx\" (UniqueName: \"kubernetes.io/projected/ae951cd2-ca5f-4ac4-abaa-3293c8f1ea57-kube-api-access-n2jnx\") pod \"network-check-target-2svls\" (UID: \"ae951cd2-ca5f-4ac4-abaa-3293c8f1ea57\") " pod="openshift-network-diagnostics/network-check-target-2svls"
Jul 28 23:22:05.479353 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:22:05.479334    2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 23:22:05.479412 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:22:05.479362    2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 23:22:05.479412 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:22:05.479377    2572 projected.go:196] Error preparing data for projected volume kube-api-access-n2jnx for pod openshift-network-diagnostics/network-check-target-2svls: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 23:22:05.479529 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:22:05.479448    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ae951cd2-ca5f-4ac4-abaa-3293c8f1ea57-kube-api-access-n2jnx podName:ae951cd2-ca5f-4ac4-abaa-3293c8f1ea57 nodeName:}" failed. No retries permitted until 2026-07-28 23:22:06.479426971 +0000 UTC m=+3.172101458 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-n2jnx" (UniqueName: "kubernetes.io/projected/ae951cd2-ca5f-4ac4-abaa-3293c8f1ea57-kube-api-access-n2jnx") pod "network-check-target-2svls" (UID: "ae951cd2-ca5f-4ac4-abaa-3293c8f1ea57") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 23:22:05.707134 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:05.706964    2572 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-27 23:17:04 +0000 UTC" deadline="2028-04-06 01:24:26.785896454 +0000 UTC"
Jul 28 23:22:05.707134 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:05.707033    2572 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14810h2m21.078867897s"
Jul 28 23:22:05.846698 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:05.846672    2572 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 23:22:05.861138 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:05.860694    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8pczd"
Jul 28 23:22:05.861138 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:22:05.860826    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8pczd" podUID="f0137726-3966-44f4-bcc7-c69cc72143be"
Jul 28 23:22:05.873240 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:05.873178    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-v84z7" event={"ID":"bd3f4ca7-06e3-4857-b42c-32c1a90fd147","Type":"ContainerStarted","Data":"b1bf9e7d4908931f71e8bf29ae61f5d8d76db70815a7f412e1e0a3cd3635b9b2"}
Jul 28 23:22:05.881672 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:05.881639    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-sltwc" event={"ID":"6640efb8-6a16-44ee-b831-90f5ee40a30c","Type":"ContainerStarted","Data":"2f073858dcc1133dcc41cf1ea2d7dabae5f4f093adfe0ca214a2d47573b6c38e"}
Jul 28 23:22:05.886574 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:05.886509    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-xqqk5" event={"ID":"82ce0587-81a8-4b5e-b841-14fed1b2364c","Type":"ContainerStarted","Data":"f54c35cab58a1882ddd63b9ba2486cc8f52c1d8078284f98dbe6ef3f4c6bd3d2"}
Jul 28 23:22:05.888142 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:05.888081    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-112.ec2.internal" event={"ID":"eb93e6ceeada8d8a42bfd36cc63ee14d","Type":"ContainerStarted","Data":"55e663bb96f2d3a2f4e851be4592c94d4940ecc365c136acf255ed4a829baea5"}
Jul 28 23:22:05.903470 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:05.903432    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c59s9" event={"ID":"25a639de-470a-46a0-bad9-c5e646731566","Type":"ContainerStarted","Data":"4ca6b8164e0b75cf43a0f6d234aaff4e136db31fd78f12c29765b8fcf4b156a1"}
Jul 28 23:22:05.915107 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:05.915071    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-6dwz5" event={"ID":"b923a656-b1ea-41e7-8c65-440a24abcee6","Type":"ContainerStarted","Data":"de1b154ae5b5e5a85bc34817b733bad336d590a4a32346c53c3deeeda506555f"}
Jul 28 23:22:05.928556 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:05.928505    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-v6nmr" event={"ID":"f220827f-670e-407f-b5c9-664800ba743d","Type":"ContainerStarted","Data":"5b133010880c1074cb930bd1a430304d65e03e2d18112f44c57d3c7647770094"}
Jul 28 23:22:05.938764 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:05.938733    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-5ndfr" event={"ID":"5c9cc37c-daaf-4fa1-b71e-94bcc6732adb","Type":"ContainerStarted","Data":"65853ad2ade305ab0cce44e2ebfdd9f87425873503bc3bdd889c7c28788c937a"}
Jul 28 23:22:05.951696 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:05.951640    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-l85q7" event={"ID":"dc10426a-a32b-4910-a2c8-21345f7b691b","Type":"ContainerStarted","Data":"e4fdf47991571d01cc4b0e138c400fff6f17ea0bf156f06dc5695c6e0e70f50c"}
Jul 28 23:22:05.958431 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:05.958317    2572 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 23:22:05.969678 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:05.969641    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-jx5sf" event={"ID":"aa22bd20-02b0-4d17-90b8-b0607ef1d16d","Type":"ContainerStarted","Data":"ba014dbb794be16759a4ea2b8872d7cdb9e716fab54c0dcbf93cd1426b9244fb"}
Jul 28 23:22:05.984672 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:05.984610    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-112.ec2.internal" event={"ID":"48ad21d512355ab82436faafc0eae6b0","Type":"ContainerStarted","Data":"d515c0c89d5fb11ea86b48b8c4e20286b6309a6fccce864ec3cdc1f65257d141"}
Jul 28 23:22:06.387024 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:06.386970    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f0137726-3966-44f4-bcc7-c69cc72143be-original-pull-secret\") pod \"global-pull-secret-syncer-8pczd\" (UID: \"f0137726-3966-44f4-bcc7-c69cc72143be\") " pod="kube-system/global-pull-secret-syncer-8pczd"
Jul 28 23:22:06.387215 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:06.387064    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/df9e50e4-7650-40bb-83eb-becf4ba8c051-metrics-certs\") pod \"network-metrics-daemon-lpj5x\" (UID: \"df9e50e4-7650-40bb-83eb-becf4ba8c051\") " pod="openshift-multus/network-metrics-daemon-lpj5x"
Jul 28 23:22:06.387215 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:22:06.387184    2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 23:22:06.387323 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:22:06.387248    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df9e50e4-7650-40bb-83eb-becf4ba8c051-metrics-certs podName:df9e50e4-7650-40bb-83eb-becf4ba8c051 nodeName:}" failed. No retries permitted until 2026-07-28 23:22:08.387228535 +0000 UTC m=+5.079903028 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/df9e50e4-7650-40bb-83eb-becf4ba8c051-metrics-certs") pod "network-metrics-daemon-lpj5x" (UID: "df9e50e4-7650-40bb-83eb-becf4ba8c051") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 23:22:06.387693 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:22:06.387647    2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 23:22:06.387810 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:22:06.387703    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f0137726-3966-44f4-bcc7-c69cc72143be-original-pull-secret podName:f0137726-3966-44f4-bcc7-c69cc72143be nodeName:}" failed. No retries permitted until 2026-07-28 23:22:08.387686962 +0000 UTC m=+5.080361454 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f0137726-3966-44f4-bcc7-c69cc72143be-original-pull-secret") pod "global-pull-secret-syncer-8pczd" (UID: "f0137726-3966-44f4-bcc7-c69cc72143be") : object "kube-system"/"original-pull-secret" not registered
Jul 28 23:22:06.488425 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:06.488388    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n2jnx\" (UniqueName: \"kubernetes.io/projected/ae951cd2-ca5f-4ac4-abaa-3293c8f1ea57-kube-api-access-n2jnx\") pod \"network-check-target-2svls\" (UID: \"ae951cd2-ca5f-4ac4-abaa-3293c8f1ea57\") " pod="openshift-network-diagnostics/network-check-target-2svls"
Jul 28 23:22:06.488619 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:22:06.488601    2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 23:22:06.488619 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:22:06.488640    2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 23:22:06.488791 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:22:06.488655    2572 projected.go:196] Error preparing data for projected volume kube-api-access-n2jnx for pod openshift-network-diagnostics/network-check-target-2svls: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 23:22:06.488791 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:22:06.488713    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ae951cd2-ca5f-4ac4-abaa-3293c8f1ea57-kube-api-access-n2jnx podName:ae951cd2-ca5f-4ac4-abaa-3293c8f1ea57 nodeName:}" failed. No retries permitted until 2026-07-28 23:22:08.488693009 +0000 UTC m=+5.181367503 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-n2jnx" (UniqueName: "kubernetes.io/projected/ae951cd2-ca5f-4ac4-abaa-3293c8f1ea57-kube-api-access-n2jnx") pod "network-check-target-2svls" (UID: "ae951cd2-ca5f-4ac4-abaa-3293c8f1ea57") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 23:22:06.489333 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:06.489311    2572 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 23:22:06.708070 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:06.707975    2572 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-27 23:17:04 +0000 UTC" deadline="2028-04-01 14:03:03.227325292 +0000 UTC"
Jul 28 23:22:06.708070 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:06.708014    2572 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14702h40m56.519316234s"
Jul 28 23:22:06.860237 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:06.860208    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lpj5x"
Jul 28 23:22:06.860431 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:22:06.860351    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lpj5x" podUID="df9e50e4-7650-40bb-83eb-becf4ba8c051"
Jul 28 23:22:06.860855 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:06.860832    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2svls"
Jul 28 23:22:06.860959 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:22:06.860928    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2svls" podUID="ae951cd2-ca5f-4ac4-abaa-3293c8f1ea57"
Jul 28 23:22:07.860618 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:07.860582    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8pczd"
Jul 28 23:22:07.861084 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:22:07.860734    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8pczd" podUID="f0137726-3966-44f4-bcc7-c69cc72143be"
Jul 28 23:22:08.406449 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:08.406412    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f0137726-3966-44f4-bcc7-c69cc72143be-original-pull-secret\") pod \"global-pull-secret-syncer-8pczd\" (UID: \"f0137726-3966-44f4-bcc7-c69cc72143be\") " pod="kube-system/global-pull-secret-syncer-8pczd"
Jul 28 23:22:08.406641 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:08.406507    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/df9e50e4-7650-40bb-83eb-becf4ba8c051-metrics-certs\") pod \"network-metrics-daemon-lpj5x\" (UID: \"df9e50e4-7650-40bb-83eb-becf4ba8c051\") " pod="openshift-multus/network-metrics-daemon-lpj5x"
Jul 28 23:22:08.406641 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:22:08.406543    2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 23:22:08.406641 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:22:08.406613    2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 23:22:08.406641 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:22:08.406641    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f0137726-3966-44f4-bcc7-c69cc72143be-original-pull-secret podName:f0137726-3966-44f4-bcc7-c69cc72143be nodeName:}" failed. No retries permitted until 2026-07-28 23:22:12.406607321 +0000 UTC m=+9.099281811 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f0137726-3966-44f4-bcc7-c69cc72143be-original-pull-secret") pod "global-pull-secret-syncer-8pczd" (UID: "f0137726-3966-44f4-bcc7-c69cc72143be") : object "kube-system"/"original-pull-secret" not registered
Jul 28 23:22:08.406870 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:22:08.406681    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df9e50e4-7650-40bb-83eb-becf4ba8c051-metrics-certs podName:df9e50e4-7650-40bb-83eb-becf4ba8c051 nodeName:}" failed. No retries permitted until 2026-07-28 23:22:12.406663351 +0000 UTC m=+9.099337840 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/df9e50e4-7650-40bb-83eb-becf4ba8c051-metrics-certs") pod "network-metrics-daemon-lpj5x" (UID: "df9e50e4-7650-40bb-83eb-becf4ba8c051") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 23:22:08.507785 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:08.507742    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n2jnx\" (UniqueName: \"kubernetes.io/projected/ae951cd2-ca5f-4ac4-abaa-3293c8f1ea57-kube-api-access-n2jnx\") pod \"network-check-target-2svls\" (UID: \"ae951cd2-ca5f-4ac4-abaa-3293c8f1ea57\") " pod="openshift-network-diagnostics/network-check-target-2svls"
Jul 28 23:22:08.507954 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:22:08.507905    2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 23:22:08.507954 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:22:08.507927    2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 23:22:08.507954 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:22:08.507942    2572 projected.go:196] Error preparing data for projected volume kube-api-access-n2jnx for pod openshift-network-diagnostics/network-check-target-2svls: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 23:22:08.508117 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:22:08.508003    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ae951cd2-ca5f-4ac4-abaa-3293c8f1ea57-kube-api-access-n2jnx podName:ae951cd2-ca5f-4ac4-abaa-3293c8f1ea57 nodeName:}" failed. No retries permitted until 2026-07-28 23:22:12.50798307 +0000 UTC m=+9.200657562 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-n2jnx" (UniqueName: "kubernetes.io/projected/ae951cd2-ca5f-4ac4-abaa-3293c8f1ea57-kube-api-access-n2jnx") pod "network-check-target-2svls" (UID: "ae951cd2-ca5f-4ac4-abaa-3293c8f1ea57") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 23:22:08.860350 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:08.860318    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lpj5x"
Jul 28 23:22:08.860546 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:22:08.860459    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lpj5x" podUID="df9e50e4-7650-40bb-83eb-becf4ba8c051"
Jul 28 23:22:08.860546 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:08.860528    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2svls"
Jul 28 23:22:08.860702 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:22:08.860605    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2svls" podUID="ae951cd2-ca5f-4ac4-abaa-3293c8f1ea57"
Jul 28 23:22:09.864284 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:09.863742    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8pczd"
Jul 28 23:22:09.864284 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:22:09.863874    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8pczd" podUID="f0137726-3966-44f4-bcc7-c69cc72143be"
Jul 28 23:22:10.859766 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:10.859729    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lpj5x"
Jul 28 23:22:10.859934 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:22:10.859884    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lpj5x" podUID="df9e50e4-7650-40bb-83eb-becf4ba8c051"
Jul 28 23:22:10.860300 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:10.860280    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2svls"
Jul 28 23:22:10.860394 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:22:10.860372    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2svls" podUID="ae951cd2-ca5f-4ac4-abaa-3293c8f1ea57"
Jul 28 23:22:11.859807 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:11.859723    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8pczd"
Jul 28 23:22:11.860259 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:22:11.859864    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8pczd" podUID="f0137726-3966-44f4-bcc7-c69cc72143be"
Jul 28 23:22:12.442737 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:12.442703    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/df9e50e4-7650-40bb-83eb-becf4ba8c051-metrics-certs\") pod \"network-metrics-daemon-lpj5x\" (UID: \"df9e50e4-7650-40bb-83eb-becf4ba8c051\") " pod="openshift-multus/network-metrics-daemon-lpj5x"
Jul 28 23:22:12.442935 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:12.442772    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f0137726-3966-44f4-bcc7-c69cc72143be-original-pull-secret\") pod \"global-pull-secret-syncer-8pczd\" (UID: \"f0137726-3966-44f4-bcc7-c69cc72143be\") " pod="kube-system/global-pull-secret-syncer-8pczd"
Jul 28 23:22:12.442935 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:22:12.442879    2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 23:22:12.442935 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:22:12.442907    2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 23:22:12.443094 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:22:12.442939    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df9e50e4-7650-40bb-83eb-becf4ba8c051-metrics-certs podName:df9e50e4-7650-40bb-83eb-becf4ba8c051 nodeName:}" failed. No retries permitted until 2026-07-28 23:22:20.442924223 +0000 UTC m=+17.135598710 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/df9e50e4-7650-40bb-83eb-becf4ba8c051-metrics-certs") pod "network-metrics-daemon-lpj5x" (UID: "df9e50e4-7650-40bb-83eb-becf4ba8c051") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 23:22:12.443094 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:22:12.442955    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f0137726-3966-44f4-bcc7-c69cc72143be-original-pull-secret podName:f0137726-3966-44f4-bcc7-c69cc72143be nodeName:}" failed. No retries permitted until 2026-07-28 23:22:20.442948647 +0000 UTC m=+17.135623135 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f0137726-3966-44f4-bcc7-c69cc72143be-original-pull-secret") pod "global-pull-secret-syncer-8pczd" (UID: "f0137726-3966-44f4-bcc7-c69cc72143be") : object "kube-system"/"original-pull-secret" not registered
Jul 28 23:22:12.543450 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:12.543359    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n2jnx\" (UniqueName: \"kubernetes.io/projected/ae951cd2-ca5f-4ac4-abaa-3293c8f1ea57-kube-api-access-n2jnx\") pod \"network-check-target-2svls\" (UID: \"ae951cd2-ca5f-4ac4-abaa-3293c8f1ea57\") " pod="openshift-network-diagnostics/network-check-target-2svls"
Jul 28 23:22:12.543619 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:22:12.543566    2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 23:22:12.543619 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:22:12.543595    2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 23:22:12.543619 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:22:12.543610    2572 projected.go:196] Error preparing data for projected volume kube-api-access-n2jnx for pod openshift-network-diagnostics/network-check-target-2svls: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 23:22:12.543809 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:22:12.543702    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ae951cd2-ca5f-4ac4-abaa-3293c8f1ea57-kube-api-access-n2jnx podName:ae951cd2-ca5f-4ac4-abaa-3293c8f1ea57 nodeName:}" failed. No retries permitted until 2026-07-28 23:22:20.543679068 +0000 UTC m=+17.236353560 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-n2jnx" (UniqueName: "kubernetes.io/projected/ae951cd2-ca5f-4ac4-abaa-3293c8f1ea57-kube-api-access-n2jnx") pod "network-check-target-2svls" (UID: "ae951cd2-ca5f-4ac4-abaa-3293c8f1ea57") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 23:22:12.860733 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:12.860260    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lpj5x"
Jul 28 23:22:12.860733 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:12.860286    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2svls"
Jul 28 23:22:12.860733 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:22:12.860398    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lpj5x" podUID="df9e50e4-7650-40bb-83eb-becf4ba8c051"
Jul 28 23:22:12.860733 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:22:12.860446    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2svls" podUID="ae951cd2-ca5f-4ac4-abaa-3293c8f1ea57"
Jul 28 23:22:13.861395 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:13.861361    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8pczd"
Jul 28 23:22:13.861871 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:22:13.861477    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8pczd" podUID="f0137726-3966-44f4-bcc7-c69cc72143be"
Jul 28 23:22:14.859776 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:14.859650    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lpj5x"
Jul 28 23:22:14.859996 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:22:14.859797    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lpj5x" podUID="df9e50e4-7650-40bb-83eb-becf4ba8c051"
Jul 28 23:22:14.860177 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:14.860150    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2svls"
Jul 28 23:22:14.860299 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:22:14.860264    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2svls" podUID="ae951cd2-ca5f-4ac4-abaa-3293c8f1ea57"
Jul 28 23:22:15.860139 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:15.860105    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8pczd"
Jul 28 23:22:15.860562 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:22:15.860223    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8pczd" podUID="f0137726-3966-44f4-bcc7-c69cc72143be"
Jul 28 23:22:16.859766 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:16.859682    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lpj5x"
Jul 28 23:22:16.859766 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:16.859704    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2svls"
Jul 28 23:22:16.859950 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:22:16.859809    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lpj5x" podUID="df9e50e4-7650-40bb-83eb-becf4ba8c051"
Jul 28 23:22:16.860004 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:22:16.859948    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2svls" podUID="ae951cd2-ca5f-4ac4-abaa-3293c8f1ea57"
Jul 28 23:22:17.860103 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:17.859889    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8pczd"
Jul 28 23:22:17.860571 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:22:17.860218    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8pczd" podUID="f0137726-3966-44f4-bcc7-c69cc72143be"
Jul 28 23:22:18.859824 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:18.859787    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lpj5x"
Jul 28 23:22:18.860019 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:18.859838    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2svls"
Jul 28 23:22:18.860019 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:22:18.859936    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lpj5x" podUID="df9e50e4-7650-40bb-83eb-becf4ba8c051"
Jul 28 23:22:18.860136 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:22:18.860083    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2svls" podUID="ae951cd2-ca5f-4ac4-abaa-3293c8f1ea57"
Jul 28 23:22:19.859918 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:19.859887    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8pczd"
Jul 28 23:22:19.860081 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:22:19.860034    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8pczd" podUID="f0137726-3966-44f4-bcc7-c69cc72143be"
Jul 28 23:22:20.499991 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:20.499955    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f0137726-3966-44f4-bcc7-c69cc72143be-original-pull-secret\") pod \"global-pull-secret-syncer-8pczd\" (UID: \"f0137726-3966-44f4-bcc7-c69cc72143be\") " pod="kube-system/global-pull-secret-syncer-8pczd"
Jul 28 23:22:20.500434 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:20.500025    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/df9e50e4-7650-40bb-83eb-becf4ba8c051-metrics-certs\") pod \"network-metrics-daemon-lpj5x\" (UID: \"df9e50e4-7650-40bb-83eb-becf4ba8c051\") " pod="openshift-multus/network-metrics-daemon-lpj5x"
Jul 28 23:22:20.500434 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:22:20.500111    2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 23:22:20.500434 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:22:20.500172    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df9e50e4-7650-40bb-83eb-becf4ba8c051-metrics-certs podName:df9e50e4-7650-40bb-83eb-becf4ba8c051 nodeName:}" failed. No retries permitted until 2026-07-28 23:22:36.500156752 +0000 UTC m=+33.192831239 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/df9e50e4-7650-40bb-83eb-becf4ba8c051-metrics-certs") pod "network-metrics-daemon-lpj5x" (UID: "df9e50e4-7650-40bb-83eb-becf4ba8c051") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 23:22:20.500434 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:22:20.500111    2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 23:22:20.500434 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:22:20.500237    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f0137726-3966-44f4-bcc7-c69cc72143be-original-pull-secret podName:f0137726-3966-44f4-bcc7-c69cc72143be nodeName:}" failed. No retries permitted until 2026-07-28 23:22:36.500223156 +0000 UTC m=+33.192897653 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f0137726-3966-44f4-bcc7-c69cc72143be-original-pull-secret") pod "global-pull-secret-syncer-8pczd" (UID: "f0137726-3966-44f4-bcc7-c69cc72143be") : object "kube-system"/"original-pull-secret" not registered
Jul 28 23:22:20.601241 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:20.601205    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n2jnx\" (UniqueName: \"kubernetes.io/projected/ae951cd2-ca5f-4ac4-abaa-3293c8f1ea57-kube-api-access-n2jnx\") pod \"network-check-target-2svls\" (UID: \"ae951cd2-ca5f-4ac4-abaa-3293c8f1ea57\") " pod="openshift-network-diagnostics/network-check-target-2svls"
Jul 28 23:22:20.601425 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:22:20.601404    2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 23:22:20.601490 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:22:20.601435    2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 23:22:20.601490 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:22:20.601450    2572 projected.go:196] Error preparing data for projected volume kube-api-access-n2jnx for pod openshift-network-diagnostics/network-check-target-2svls: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 23:22:20.601582 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:22:20.601523    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ae951cd2-ca5f-4ac4-abaa-3293c8f1ea57-kube-api-access-n2jnx podName:ae951cd2-ca5f-4ac4-abaa-3293c8f1ea57 nodeName:}" failed. No retries permitted until 2026-07-28 23:22:36.601503368 +0000 UTC m=+33.294177858 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-n2jnx" (UniqueName: "kubernetes.io/projected/ae951cd2-ca5f-4ac4-abaa-3293c8f1ea57-kube-api-access-n2jnx") pod "network-check-target-2svls" (UID: "ae951cd2-ca5f-4ac4-abaa-3293c8f1ea57") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 23:22:20.859691 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:20.859592    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2svls"
Jul 28 23:22:20.859872 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:22:20.859743    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2svls" podUID="ae951cd2-ca5f-4ac4-abaa-3293c8f1ea57"
Jul 28 23:22:20.859872 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:20.859592    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lpj5x"
Jul 28 23:22:20.860058 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:22:20.859897    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lpj5x" podUID="df9e50e4-7650-40bb-83eb-becf4ba8c051"
Jul 28 23:22:21.860069 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:21.860032    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8pczd"
Jul 28 23:22:21.860515 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:22:21.860147    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8pczd" podUID="f0137726-3966-44f4-bcc7-c69cc72143be"
Jul 28 23:22:22.860682 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:22.860650    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lpj5x"
Jul 28 23:22:22.861125 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:22.860650    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2svls"
Jul 28 23:22:22.861125 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:22:22.860786    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lpj5x" podUID="df9e50e4-7650-40bb-83eb-becf4ba8c051"
Jul 28 23:22:22.861125 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:22:22.860873    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2svls" podUID="ae951cd2-ca5f-4ac4-abaa-3293c8f1ea57"
Jul 28 23:22:23.861005 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:23.860967    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8pczd"
Jul 28 23:22:23.861415 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:22:23.861074    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8pczd" podUID="f0137726-3966-44f4-bcc7-c69cc72143be"
Jul 28 23:22:24.859661 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:24.859639    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lpj5x"
Jul 28 23:22:24.859723 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:24.859667    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2svls"
Jul 28 23:22:24.859794 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:22:24.859772    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lpj5x" podUID="df9e50e4-7650-40bb-83eb-becf4ba8c051"
Jul 28 23:22:24.859899 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:22:24.859881    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2svls" podUID="ae951cd2-ca5f-4ac4-abaa-3293c8f1ea57"
Jul 28 23:22:25.862873 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:25.862706    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8pczd"
Jul 28 23:22:25.863514 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:22:25.862941    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8pczd" podUID="f0137726-3966-44f4-bcc7-c69cc72143be"
Jul 28 23:22:26.034458 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:26.034421    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c59s9" event={"ID":"25a639de-470a-46a0-bad9-c5e646731566","Type":"ContainerStarted","Data":"c544c77221db60040cd61d1c63ee4d230fb52bc85ef3faf6e7574666cad6eba1"}
Jul 28 23:22:26.035981 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:26.035952    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-6dwz5" event={"ID":"b923a656-b1ea-41e7-8c65-440a24abcee6","Type":"ContainerStarted","Data":"ef40c9738c1de301024bf2cfc43ab97af83a418d247ce09d52675893a701bef6"}
Jul 28 23:22:26.038325 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:26.038297    2572 generic.go:356] "Generic (PLEG): container finished" podID="f220827f-670e-407f-b5c9-664800ba743d" containerID="6bd3ee061ef9f85e29b646dddcab1943e656fca0eb9988cd612620fef8ddc0a7" exitCode=0
Jul 28 23:22:26.038420 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:26.038382    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-v6nmr" event={"ID":"f220827f-670e-407f-b5c9-664800ba743d","Type":"ContainerDied","Data":"6bd3ee061ef9f85e29b646dddcab1943e656fca0eb9988cd612620fef8ddc0a7"}
Jul 28 23:22:26.040884 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:26.040858    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-5ndfr" event={"ID":"5c9cc37c-daaf-4fa1-b71e-94bcc6732adb","Type":"ContainerStarted","Data":"b2ba92027658bfd27a184ec1ff63ccb8ede20835ce3385b5cb3e61aee61c94cf"}
Jul 28 23:22:26.042561 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:26.042535    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-l85q7" event={"ID":"dc10426a-a32b-4910-a2c8-21345f7b691b","Type":"ContainerStarted","Data":"75fe8d305af5d929c89c4d29175ab4a4b3c34d03a1d751ffe112a5d92c7c9434"}
Jul 28 23:22:26.044135 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:26.044089    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-jx5sf" event={"ID":"aa22bd20-02b0-4d17-90b8-b0607ef1d16d","Type":"ContainerStarted","Data":"4102a760066c57d69bd5f0fc85c1e2f0d8a3f19a5d87d4381e644a35ed5bdd4b"}
Jul 28 23:22:26.045486 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:26.045465    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-112.ec2.internal" event={"ID":"48ad21d512355ab82436faafc0eae6b0","Type":"ContainerStarted","Data":"9a6287165e28bc5034bc379919008f6be8be82b09a9067ca5a8661ba9d8eaa28"}
Jul 28 23:22:26.048060 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:26.048041    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-v84z7" event={"ID":"bd3f4ca7-06e3-4857-b42c-32c1a90fd147","Type":"ContainerStarted","Data":"c25f39ba0a48d769d33dc14d477ee29887fcaa32a9a94ca64880b5dafc8809ef"}
Jul 28 23:22:26.048136 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:26.048107    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-v84z7" event={"ID":"bd3f4ca7-06e3-4857-b42c-32c1a90fd147","Type":"ContainerStarted","Data":"6ed8f8c11402861704f2c58a6f54cdb45e023c1bf8c85e337f9ee3403a33de20"}
Jul 28 23:22:26.048173 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:26.048138    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-v84z7" event={"ID":"bd3f4ca7-06e3-4857-b42c-32c1a90fd147","Type":"ContainerStarted","Data":"aed93e34141be4c74438913fa10a3ea0b94b30c987f4e8273152b51cf61700f6"}
Jul 28 23:22:26.048206 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:26.048180    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-v84z7" event={"ID":"bd3f4ca7-06e3-4857-b42c-32c1a90fd147","Type":"ContainerStarted","Data":"3262e39e8a25f551e400a62049df5ce7e9cd3bce4032e15cb2d08b0287fb4717"}
Jul 28 23:22:26.048250 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:26.048216    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-v84z7" event={"ID":"bd3f4ca7-06e3-4857-b42c-32c1a90fd147","Type":"ContainerStarted","Data":"07e925d79fa4e3a5a83d00ac24e5947e435b10b3bc263a28d0e6ffdecfe0bd33"}
Jul 28 23:22:26.048299 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:26.048263    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-v84z7" event={"ID":"bd3f4ca7-06e3-4857-b42c-32c1a90fd147","Type":"ContainerStarted","Data":"1f1f5d4e068d2aeadf2070ad3f8076e7b317c65c3e6458032e8aaba8ed8e870d"}
Jul 28 23:22:26.049336 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:26.049318    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-xqqk5" event={"ID":"82ce0587-81a8-4b5e-b841-14fed1b2364c","Type":"ContainerStarted","Data":"9937e5f40bbf66a2e0f45b0903eec396d3b191fd7e799f1a5f50e57778fde27b"}
Jul 28 23:22:26.053059 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:26.050693    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-6dwz5" podStartSLOduration=3.358131464 podStartE2EDuration="23.050674404s" totalImagesPullingTime="19.69254294s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 23:22:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 23:22:26.049765627 +0000 UTC m=+22.742440136" watchObservedRunningTime="2026-07-28 23:22:26.050674404 +0000 UTC m=+22.743348913"
Jul 28 23:22:26.053685 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:26.053659    2572 generic.go:356] "Generic (PLEG): container finished" podID="eb93e6ceeada8d8a42bfd36cc63ee14d" containerID="14d3e5399b7bc43aeccc1aea090da487d9f067bdafd7dbd66cdc5a8a13a5faca" exitCode=0
Jul 28 23:22:26.053783 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:26.053695    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-112.ec2.internal" event={"ID":"eb93e6ceeada8d8a42bfd36cc63ee14d","Type":"ContainerDied","Data":"14d3e5399b7bc43aeccc1aea090da487d9f067bdafd7dbd66cdc5a8a13a5faca"}
Jul 28 23:22:26.078846 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:26.078805    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-jx5sf" podStartSLOduration=3.253296542 podStartE2EDuration="23.078791209s" totalImagesPullingTime="19.825494667s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 23:22:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 23:22:26.065155984 +0000 UTC m=+22.757830493" watchObservedRunningTime="2026-07-28 23:22:26.078791209 +0000 UTC m=+22.771465718"
Jul 28 23:22:26.078948 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:26.078891    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-xqqk5" podStartSLOduration=3.2805430859999998 podStartE2EDuration="23.07888695s" totalImagesPullingTime="19.798343864s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 23:22:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 23:22:26.078584112 +0000 UTC m=+22.771258621" watchObservedRunningTime="2026-07-28 23:22:26.07888695 +0000 UTC m=+22.771561458"
Jul 28 23:22:26.092151 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:26.091945    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-5ndfr" podStartSLOduration=11.454693098 podStartE2EDuration="23.091926402s" totalImagesPullingTime="11.637233304s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 23:22:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 23:22:26.091110198 +0000 UTC m=+22.783784708" watchObservedRunningTime="2026-07-28 23:22:26.091926402 +0000 UTC m=+22.784600911"
Jul 28 23:22:26.126828 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:26.126717    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-112.ec2.internal" podStartSLOduration=22.12670146 podStartE2EDuration="22.12670146s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 23:22:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 23:22:26.126408089 +0000 UTC m=+22.819082599" watchObservedRunningTime="2026-07-28 23:22:26.12670146 +0000 UTC m=+22.819375969"
Jul 28 23:22:26.146059 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:26.146002    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-l85q7" podStartSLOduration=3.063508613 podStartE2EDuration="23.145979172s" totalImagesPullingTime="20.082470559s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 23:22:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 23:22:26.145134091 +0000 UTC m=+22.837808623" watchObservedRunningTime="2026-07-28 23:22:26.145979172 +0000 UTC m=+22.838653682"
Jul 28 23:22:26.705931 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:26.705891    2572 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 28 23:22:26.743904 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:26.743789    2572 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-28T23:22:26.705910005Z","UUID":"b928d9ba-b638-48c9-9f7c-faec773838f1","Handler":null,"Name":"","Endpoint":""}
Jul 28 23:22:26.745494 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:26.745472    2572 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 28 23:22:26.745653 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:26.745512    2572 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 28 23:22:26.859838 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:26.859763    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lpj5x"
Jul 28 23:22:26.859838 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:26.859788    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2svls"
Jul 28 23:22:26.860053 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:22:26.859882    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lpj5x" podUID="df9e50e4-7650-40bb-83eb-becf4ba8c051"
Jul 28 23:22:26.860053 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:22:26.860026    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2svls" podUID="ae951cd2-ca5f-4ac4-abaa-3293c8f1ea57"
Jul 28 23:22:27.058421 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:27.058382    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c59s9" event={"ID":"25a639de-470a-46a0-bad9-c5e646731566","Type":"ContainerStarted","Data":"fbcc06c5182a2b43a9fa79a0bfb0724a953228a29e3802304a2f93268a2a537b"}
Jul 28 23:22:27.060173 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:27.060148    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-sltwc" event={"ID":"6640efb8-6a16-44ee-b831-90f5ee40a30c","Type":"ContainerStarted","Data":"0650a7243710155e6a902d6a11e1d19886b49fa9e5fca634df928751249a3c3d"}
Jul 28 23:22:27.062202 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:27.062162    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-112.ec2.internal" event={"ID":"eb93e6ceeada8d8a42bfd36cc63ee14d","Type":"ContainerStarted","Data":"94c34b7c32d50d5233f2a09c4251fad75a643bcf604a5e26d80a391e147afb76"}
Jul 28 23:22:27.074260 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:27.074209    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-sltwc" podStartSLOduration=4.297878055 podStartE2EDuration="24.074191756s" totalImagesPullingTime="19.776313701s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 23:22:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 23:22:27.073742729 +0000 UTC m=+23.766417239" watchObservedRunningTime="2026-07-28 23:22:27.074191756 +0000 UTC m=+23.766866264"
Jul 28 23:22:27.089434 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:27.089392    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-112.ec2.internal" podStartSLOduration=23.089377765 podStartE2EDuration="23.089377765s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 23:22:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 23:22:27.089023636 +0000 UTC m=+23.781698146" watchObservedRunningTime="2026-07-28 23:22:27.089377765 +0000 UTC m=+23.782052274"
Jul 28 23:22:27.368194 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:27.368164    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-6dwz5"
Jul 28 23:22:27.368780 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:27.368762    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-6dwz5"
Jul 28 23:22:27.859978 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:27.859943    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8pczd"
Jul 28 23:22:27.860192 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:22:27.860078    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8pczd" podUID="f0137726-3966-44f4-bcc7-c69cc72143be"
Jul 28 23:22:28.069112 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:28.069075    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-v84z7" event={"ID":"bd3f4ca7-06e3-4857-b42c-32c1a90fd147","Type":"ContainerStarted","Data":"d0d7105cc993191132a02870478afabc3e71ee1f25fa033160656c1d18da229e"}
Jul 28 23:22:28.859656 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:28.859608    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lpj5x"
Jul 28 23:22:28.859938 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:28.859608    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2svls"
Jul 28 23:22:28.859938 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:22:28.859762    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lpj5x" podUID="df9e50e4-7650-40bb-83eb-becf4ba8c051"
Jul 28 23:22:28.859938 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:22:28.859830    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2svls" podUID="ae951cd2-ca5f-4ac4-abaa-3293c8f1ea57"
Jul 28 23:22:29.074157 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:29.074128    2572 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 28 23:22:29.074157 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:29.074138    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c59s9" event={"ID":"25a639de-470a-46a0-bad9-c5e646731566","Type":"ContainerStarted","Data":"440006e9794ff13b7a5682b7a4c29dc17d384cdaf35f8261048b33102e2baceb"}
Jul 28 23:22:29.092569 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:29.092519    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c59s9" podStartSLOduration=3.686654853 podStartE2EDuration="26.092504103s" totalImagesPullingTime="22.40584925s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 23:22:03 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 23:22:29.090982621 +0000 UTC m=+25.783657131" watchObservedRunningTime="2026-07-28 23:22:29.092504103 +0000 UTC m=+25.785178611"
Jul 28 23:22:29.860271 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:29.860230    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8pczd"
Jul 28 23:22:29.860459 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:22:29.860358    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8pczd" podUID="f0137726-3966-44f4-bcc7-c69cc72143be"
Jul 28 23:22:30.860292 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:30.860119    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lpj5x"
Jul 28 23:22:30.860887 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:30.860124    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2svls"
Jul 28 23:22:30.860887 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:22:30.860368    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lpj5x" podUID="df9e50e4-7650-40bb-83eb-becf4ba8c051"
Jul 28 23:22:30.860887 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:22:30.860456    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2svls" podUID="ae951cd2-ca5f-4ac4-abaa-3293c8f1ea57"
Jul 28 23:22:31.080843 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:31.080810    2572 generic.go:356] "Generic (PLEG): container finished" podID="f220827f-670e-407f-b5c9-664800ba743d" containerID="5618783e889c6146ecf21b73c1516405cb49d51178ea4647f8978075660b3592" exitCode=0
Jul 28 23:22:31.081067 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:31.080901    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-v6nmr" event={"ID":"f220827f-670e-407f-b5c9-664800ba743d","Type":"ContainerDied","Data":"5618783e889c6146ecf21b73c1516405cb49d51178ea4647f8978075660b3592"}
Jul 28 23:22:31.084638 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:31.084602    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-v84z7" event={"ID":"bd3f4ca7-06e3-4857-b42c-32c1a90fd147","Type":"ContainerStarted","Data":"a93331a73c3c8fa9ec4e808045bd7e2a7d03c9c54ce1966494517da0849e600c"}
Jul 28 23:22:31.084896 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:31.084876    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-v84z7"
Jul 28 23:22:31.085010 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:31.084950    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-v84z7"
Jul 28 23:22:31.099699 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:31.099680    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-v84z7"
Jul 28 23:22:31.123794 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:31.123718    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-v84z7" podStartSLOduration=8.364686526 podStartE2EDuration="28.123703303s" totalImagesPullingTime="19.759016777s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 23:22:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 23:22:31.122889674 +0000 UTC m=+27.815564183" watchObservedRunningTime="2026-07-28 23:22:31.123703303 +0000 UTC m=+27.816377811"
Jul 28 23:22:31.862794 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:31.862717    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8pczd"
Jul 28 23:22:31.863321 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:22:31.862839    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8pczd" podUID="f0137726-3966-44f4-bcc7-c69cc72143be"
Jul 28 23:22:32.087441 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:32.087399    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-v84z7"
Jul 28 23:22:32.101329 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:32.101306    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-v84z7"
Jul 28 23:22:32.412378 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:32.412124    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-lpj5x"]
Jul 28 23:22:32.412539 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:32.412453    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lpj5x"
Jul 28 23:22:32.412605 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:22:32.412579    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lpj5x" podUID="df9e50e4-7650-40bb-83eb-becf4ba8c051"
Jul 28 23:22:32.417411 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:32.417384    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-8pczd"]
Jul 28 23:22:32.417546 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:32.417493    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8pczd"
Jul 28 23:22:32.417614 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:22:32.417579    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8pczd" podUID="f0137726-3966-44f4-bcc7-c69cc72143be"
Jul 28 23:22:32.418104 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:32.418079    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-2svls"]
Jul 28 23:22:32.418221 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:32.418186    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2svls"
Jul 28 23:22:32.418297 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:22:32.418271    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2svls" podUID="ae951cd2-ca5f-4ac4-abaa-3293c8f1ea57"
Jul 28 23:22:33.091563 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:33.091529    2572 generic.go:356] "Generic (PLEG): container finished" podID="f220827f-670e-407f-b5c9-664800ba743d" containerID="61fb5b8c037b94589cc8fc65a03828f2842f2e3d8b4978096072a37745d67d2c" exitCode=0
Jul 28 23:22:33.092035 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:33.091610    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-v6nmr" event={"ID":"f220827f-670e-407f-b5c9-664800ba743d","Type":"ContainerDied","Data":"61fb5b8c037b94589cc8fc65a03828f2842f2e3d8b4978096072a37745d67d2c"}
Jul 28 23:22:33.860970 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:33.860934    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8pczd"
Jul 28 23:22:33.861177 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:22:33.861033    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8pczd" podUID="f0137726-3966-44f4-bcc7-c69cc72143be"
Jul 28 23:22:33.861177 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:33.861120    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2svls"
Jul 28 23:22:33.861307 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:33.861220    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lpj5x"
Jul 28 23:22:33.861307 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:22:33.861233    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2svls" podUID="ae951cd2-ca5f-4ac4-abaa-3293c8f1ea57"
Jul 28 23:22:33.861392 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:22:33.861318    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lpj5x" podUID="df9e50e4-7650-40bb-83eb-becf4ba8c051"
Jul 28 23:22:34.742800 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:34.742773    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-6dwz5"
Jul 28 23:22:34.743177 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:34.742915    2572 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 28 23:22:34.743476 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:34.743459    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-6dwz5"
Jul 28 23:22:35.097488 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:35.097399    2572 generic.go:356] "Generic (PLEG): container finished" podID="f220827f-670e-407f-b5c9-664800ba743d" containerID="fd286c6bfa66418f11e28e708e4ba181b8dc98be70ab348454f4c0c4d251bde2" exitCode=0
Jul 28 23:22:35.097488 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:35.097460    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-v6nmr" event={"ID":"f220827f-670e-407f-b5c9-664800ba743d","Type":"ContainerDied","Data":"fd286c6bfa66418f11e28e708e4ba181b8dc98be70ab348454f4c0c4d251bde2"}
Jul 28 23:22:35.860731 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:35.860701    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2svls"
Jul 28 23:22:35.861302 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:35.860746    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8pczd"
Jul 28 23:22:35.861302 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:35.860700    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lpj5x"
Jul 28 23:22:35.861302 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:22:35.860822    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2svls" podUID="ae951cd2-ca5f-4ac4-abaa-3293c8f1ea57"
Jul 28 23:22:35.861302 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:22:35.860889    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lpj5x" podUID="df9e50e4-7650-40bb-83eb-becf4ba8c051"
Jul 28 23:22:35.861302 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:22:35.860964    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8pczd" podUID="f0137726-3966-44f4-bcc7-c69cc72143be"
Jul 28 23:22:36.519110 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:36.519024    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/df9e50e4-7650-40bb-83eb-becf4ba8c051-metrics-certs\") pod \"network-metrics-daemon-lpj5x\" (UID: \"df9e50e4-7650-40bb-83eb-becf4ba8c051\") " pod="openshift-multus/network-metrics-daemon-lpj5x"
Jul 28 23:22:36.519110 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:36.519083    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f0137726-3966-44f4-bcc7-c69cc72143be-original-pull-secret\") pod \"global-pull-secret-syncer-8pczd\" (UID: \"f0137726-3966-44f4-bcc7-c69cc72143be\") " pod="kube-system/global-pull-secret-syncer-8pczd"
Jul 28 23:22:36.519319 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:22:36.519180    2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 23:22:36.519319 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:22:36.519228    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f0137726-3966-44f4-bcc7-c69cc72143be-original-pull-secret podName:f0137726-3966-44f4-bcc7-c69cc72143be nodeName:}" failed. No retries permitted until 2026-07-28 23:23:08.519215412 +0000 UTC m=+65.211889899 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f0137726-3966-44f4-bcc7-c69cc72143be-original-pull-secret") pod "global-pull-secret-syncer-8pczd" (UID: "f0137726-3966-44f4-bcc7-c69cc72143be") : object "kube-system"/"original-pull-secret" not registered
Jul 28 23:22:36.519319 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:22:36.519180    2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 23:22:36.519319 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:22:36.519317    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df9e50e4-7650-40bb-83eb-becf4ba8c051-metrics-certs podName:df9e50e4-7650-40bb-83eb-becf4ba8c051 nodeName:}" failed. No retries permitted until 2026-07-28 23:23:08.519300316 +0000 UTC m=+65.211974810 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/df9e50e4-7650-40bb-83eb-becf4ba8c051-metrics-certs") pod "network-metrics-daemon-lpj5x" (UID: "df9e50e4-7650-40bb-83eb-becf4ba8c051") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 23:22:36.620154 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:36.620120    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n2jnx\" (UniqueName: \"kubernetes.io/projected/ae951cd2-ca5f-4ac4-abaa-3293c8f1ea57-kube-api-access-n2jnx\") pod \"network-check-target-2svls\" (UID: \"ae951cd2-ca5f-4ac4-abaa-3293c8f1ea57\") " pod="openshift-network-diagnostics/network-check-target-2svls"
Jul 28 23:22:36.620325 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:22:36.620248    2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 23:22:36.620325 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:22:36.620262    2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 23:22:36.620325 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:22:36.620271    2572 projected.go:196] Error preparing data for projected volume kube-api-access-n2jnx for pod openshift-network-diagnostics/network-check-target-2svls: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 23:22:36.620325 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:22:36.620322    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ae951cd2-ca5f-4ac4-abaa-3293c8f1ea57-kube-api-access-n2jnx podName:ae951cd2-ca5f-4ac4-abaa-3293c8f1ea57 nodeName:}" failed. No retries permitted until 2026-07-28 23:23:08.620304305 +0000 UTC m=+65.312978791 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-n2jnx" (UniqueName: "kubernetes.io/projected/ae951cd2-ca5f-4ac4-abaa-3293c8f1ea57-kube-api-access-n2jnx") pod "network-check-target-2svls" (UID: "ae951cd2-ca5f-4ac4-abaa-3293c8f1ea57") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 23:22:37.629761 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:37.629729    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-112.ec2.internal" event="NodeReady"
Jul 28 23:22:37.630185 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:37.629885    2572 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 28 23:22:37.673205 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:37.673171    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-8k4xj"]
Jul 28 23:22:37.687743 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:37.687709    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-gwrtv"]
Jul 28 23:22:37.687938 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:37.687918    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-8k4xj"
Jul 28 23:22:37.690329 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:37.690301    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 28 23:22:37.691123 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:37.691101    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 28 23:22:37.691244 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:37.691194    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-7kmkd\""
Jul 28 23:22:37.695977 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:37.695957    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-gwrtv"]
Jul 28 23:22:37.696074 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:37.695988    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-8k4xj"]
Jul 28 23:22:37.696074 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:37.696068    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-gwrtv"
Jul 28 23:22:37.698292 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:37.698260    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 28 23:22:37.698392 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:37.698348    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-dpgd4\""
Jul 28 23:22:37.698392 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:37.698382    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 28 23:22:37.698605 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:37.698584    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 28 23:22:37.829975 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:37.829883    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3d670d49-7bd6-47f9-befe-d76cce20834a-tmp-dir\") pod \"dns-default-8k4xj\" (UID: \"3d670d49-7bd6-47f9-befe-d76cce20834a\") " pod="openshift-dns/dns-default-8k4xj"
Jul 28 23:22:37.829975 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:37.829957    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ad652290-064c-4b42-8419-bdd779497c86-cert\") pod \"ingress-canary-gwrtv\" (UID: \"ad652290-064c-4b42-8419-bdd779497c86\") " pod="openshift-ingress-canary/ingress-canary-gwrtv"
Jul 28 23:22:37.830193 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:37.829979    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/3d670d49-7bd6-47f9-befe-d76cce20834a-config-volume\") pod \"dns-default-8k4xj\" (UID: \"3d670d49-7bd6-47f9-befe-d76cce20834a\") " pod="openshift-dns/dns-default-8k4xj"
Jul 28 23:22:37.830193 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:37.830090    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3d670d49-7bd6-47f9-befe-d76cce20834a-metrics-tls\") pod \"dns-default-8k4xj\" (UID: \"3d670d49-7bd6-47f9-befe-d76cce20834a\") " pod="openshift-dns/dns-default-8k4xj"
Jul 28 23:22:37.830193 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:37.830122    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2c7mn\" (UniqueName: \"kubernetes.io/projected/3d670d49-7bd6-47f9-befe-d76cce20834a-kube-api-access-2c7mn\") pod \"dns-default-8k4xj\" (UID: \"3d670d49-7bd6-47f9-befe-d76cce20834a\") " pod="openshift-dns/dns-default-8k4xj"
Jul 28 23:22:37.830193 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:37.830148    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n8td8\" (UniqueName: \"kubernetes.io/projected/ad652290-064c-4b42-8419-bdd779497c86-kube-api-access-n8td8\") pod \"ingress-canary-gwrtv\" (UID: \"ad652290-064c-4b42-8419-bdd779497c86\") " pod="openshift-ingress-canary/ingress-canary-gwrtv"
Jul 28 23:22:37.859950 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:37.859917    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2svls"
Jul 28 23:22:37.860105 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:37.859917    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8pczd"
Jul 28 23:22:37.860523 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:37.860501    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lpj5x"
Jul 28 23:22:37.862886 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:37.862865    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 28 23:22:37.862886 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:37.862880    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 28 23:22:37.863106 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:37.862952    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 28 23:22:37.863187 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:37.863158    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-plrk7\""
Jul 28 23:22:37.863247 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:37.863186    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-bhxv7\""
Jul 28 23:22:37.863247 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:37.863230    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 28 23:22:37.931301 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:37.931266    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/3d670d49-7bd6-47f9-befe-d76cce20834a-config-volume\") pod \"dns-default-8k4xj\" (UID: \"3d670d49-7bd6-47f9-befe-d76cce20834a\") " pod="openshift-dns/dns-default-8k4xj"
Jul 28 23:22:37.931465 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:37.931346    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3d670d49-7bd6-47f9-befe-d76cce20834a-metrics-tls\") pod \"dns-default-8k4xj\" (UID: \"3d670d49-7bd6-47f9-befe-d76cce20834a\") " pod="openshift-dns/dns-default-8k4xj"
Jul 28 23:22:37.931465 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:37.931366    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2c7mn\" (UniqueName: \"kubernetes.io/projected/3d670d49-7bd6-47f9-befe-d76cce20834a-kube-api-access-2c7mn\") pod \"dns-default-8k4xj\" (UID: \"3d670d49-7bd6-47f9-befe-d76cce20834a\") " pod="openshift-dns/dns-default-8k4xj"
Jul 28 23:22:37.931465 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:37.931383    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n8td8\" (UniqueName: \"kubernetes.io/projected/ad652290-064c-4b42-8419-bdd779497c86-kube-api-access-n8td8\") pod \"ingress-canary-gwrtv\" (UID: \"ad652290-064c-4b42-8419-bdd779497c86\") " pod="openshift-ingress-canary/ingress-canary-gwrtv"
Jul 28 23:22:37.931465 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:37.931406    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3d670d49-7bd6-47f9-befe-d76cce20834a-tmp-dir\") pod \"dns-default-8k4xj\" (UID: \"3d670d49-7bd6-47f9-befe-d76cce20834a\") " pod="openshift-dns/dns-default-8k4xj"
Jul 28 23:22:37.931465 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:37.931444    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ad652290-064c-4b42-8419-bdd779497c86-cert\") pod \"ingress-canary-gwrtv\" (UID: \"ad652290-064c-4b42-8419-bdd779497c86\") " pod="openshift-ingress-canary/ingress-canary-gwrtv"
Jul 28 23:22:37.931727 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:22:37.931517    2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 23:22:37.931727 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:22:37.931582    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3d670d49-7bd6-47f9-befe-d76cce20834a-metrics-tls podName:3d670d49-7bd6-47f9-befe-d76cce20834a nodeName:}" failed. No retries permitted until 2026-07-28 23:22:38.431560845 +0000 UTC m=+35.124235334 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3d670d49-7bd6-47f9-befe-d76cce20834a-metrics-tls") pod "dns-default-8k4xj" (UID: "3d670d49-7bd6-47f9-befe-d76cce20834a") : secret "dns-default-metrics-tls" not found
Jul 28 23:22:37.931988 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:37.931964    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3d670d49-7bd6-47f9-befe-d76cce20834a-tmp-dir\") pod \"dns-default-8k4xj\" (UID: \"3d670d49-7bd6-47f9-befe-d76cce20834a\") " pod="openshift-dns/dns-default-8k4xj"
Jul 28 23:22:37.932141 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:37.932125    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/3d670d49-7bd6-47f9-befe-d76cce20834a-config-volume\") pod \"dns-default-8k4xj\" (UID: \"3d670d49-7bd6-47f9-befe-d76cce20834a\") " pod="openshift-dns/dns-default-8k4xj"
Jul 28 23:22:37.936058 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:37.936033    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ad652290-064c-4b42-8419-bdd779497c86-cert\") pod \"ingress-canary-gwrtv\" (UID: \"ad652290-064c-4b42-8419-bdd779497c86\") " pod="openshift-ingress-canary/ingress-canary-gwrtv"
Jul 28 23:22:37.939911 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:37.939861    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2c7mn\" (UniqueName: \"kubernetes.io/projected/3d670d49-7bd6-47f9-befe-d76cce20834a-kube-api-access-2c7mn\") pod \"dns-default-8k4xj\" (UID: \"3d670d49-7bd6-47f9-befe-d76cce20834a\") " pod="openshift-dns/dns-default-8k4xj"
Jul 28 23:22:37.940021 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:37.939960    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n8td8\" (UniqueName: \"kubernetes.io/projected/ad652290-064c-4b42-8419-bdd779497c86-kube-api-access-n8td8\") pod \"ingress-canary-gwrtv\" (UID: \"ad652290-064c-4b42-8419-bdd779497c86\") " pod="openshift-ingress-canary/ingress-canary-gwrtv"
Jul 28 23:22:38.007937 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:38.007899    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-gwrtv"
Jul 28 23:22:38.279904 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:38.279871    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-4p55m"]
Jul 28 23:22:38.299076 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:38.299045    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-4p55m"]
Jul 28 23:22:38.299229 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:38.299142    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-4p55m"
Jul 28 23:22:38.301720 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:38.301538    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 28 23:22:38.301720 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:38.301555    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-xh4g8\""
Jul 28 23:22:38.301720 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:38.301538    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 28 23:22:38.301720 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:38.301639    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 28 23:22:38.301980 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:38.301931    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 28 23:22:38.436277 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:38.436240    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2e08e8ce-573f-4068-8c3b-96315b3d60ce-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4p55m\" (UID: \"2e08e8ce-573f-4068-8c3b-96315b3d60ce\") " pod="openshift-insights/insights-runtime-extractor-4p55m"
Jul 28 23:22:38.436442 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:38.436286    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2e08e8ce-573f-4068-8c3b-96315b3d60ce-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4p55m\" (UID: \"2e08e8ce-573f-4068-8c3b-96315b3d60ce\") " pod="openshift-insights/insights-runtime-extractor-4p55m"
Jul 28 23:22:38.436442 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:38.436378    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/2e08e8ce-573f-4068-8c3b-96315b3d60ce-data-volume\") pod \"insights-runtime-extractor-4p55m\" (UID: \"2e08e8ce-573f-4068-8c3b-96315b3d60ce\") " pod="openshift-insights/insights-runtime-extractor-4p55m"
Jul 28 23:22:38.436564 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:38.436444    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3d670d49-7bd6-47f9-befe-d76cce20834a-metrics-tls\") pod \"dns-default-8k4xj\" (UID: \"3d670d49-7bd6-47f9-befe-d76cce20834a\") " pod="openshift-dns/dns-default-8k4xj"
Jul 28 23:22:38.436564 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:38.436472    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8n5xw\" (UniqueName: \"kubernetes.io/projected/2e08e8ce-573f-4068-8c3b-96315b3d60ce-kube-api-access-8n5xw\") pod \"insights-runtime-extractor-4p55m\" (UID: \"2e08e8ce-573f-4068-8c3b-96315b3d60ce\") " pod="openshift-insights/insights-runtime-extractor-4p55m"
Jul 28 23:22:38.436564 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:38.436507    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/2e08e8ce-573f-4068-8c3b-96315b3d60ce-crio-socket\") pod \"insights-runtime-extractor-4p55m\" (UID: \"2e08e8ce-573f-4068-8c3b-96315b3d60ce\") " pod="openshift-insights/insights-runtime-extractor-4p55m"
Jul 28 23:22:38.436725 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:22:38.436672    2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 23:22:38.436771 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:22:38.436747    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3d670d49-7bd6-47f9-befe-d76cce20834a-metrics-tls podName:3d670d49-7bd6-47f9-befe-d76cce20834a nodeName:}" failed. No retries permitted until 2026-07-28 23:22:39.436723449 +0000 UTC m=+36.129397936 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3d670d49-7bd6-47f9-befe-d76cce20834a-metrics-tls") pod "dns-default-8k4xj" (UID: "3d670d49-7bd6-47f9-befe-d76cce20834a") : secret "dns-default-metrics-tls" not found
Jul 28 23:22:38.537488 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:38.537399    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/2e08e8ce-573f-4068-8c3b-96315b3d60ce-crio-socket\") pod \"insights-runtime-extractor-4p55m\" (UID: \"2e08e8ce-573f-4068-8c3b-96315b3d60ce\") " pod="openshift-insights/insights-runtime-extractor-4p55m"
Jul 28 23:22:38.537488 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:38.537469    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2e08e8ce-573f-4068-8c3b-96315b3d60ce-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4p55m\" (UID: \"2e08e8ce-573f-4068-8c3b-96315b3d60ce\") " pod="openshift-insights/insights-runtime-extractor-4p55m"
Jul 28 23:22:38.537755 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:38.537500    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2e08e8ce-573f-4068-8c3b-96315b3d60ce-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4p55m\" (UID: \"2e08e8ce-573f-4068-8c3b-96315b3d60ce\") " pod="openshift-insights/insights-runtime-extractor-4p55m"
Jul 28 23:22:38.537755 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:38.537550    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/2e08e8ce-573f-4068-8c3b-96315b3d60ce-data-volume\") pod \"insights-runtime-extractor-4p55m\" (UID: \"2e08e8ce-573f-4068-8c3b-96315b3d60ce\") " pod="openshift-insights/insights-runtime-extractor-4p55m"
Jul 28 23:22:38.537755 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:38.537609    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8n5xw\" (UniqueName: \"kubernetes.io/projected/2e08e8ce-573f-4068-8c3b-96315b3d60ce-kube-api-access-8n5xw\") pod \"insights-runtime-extractor-4p55m\" (UID: \"2e08e8ce-573f-4068-8c3b-96315b3d60ce\") " pod="openshift-insights/insights-runtime-extractor-4p55m"
Jul 28 23:22:38.537755 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:22:38.537665    2572 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 23:22:38.537755 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:38.537637    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/2e08e8ce-573f-4068-8c3b-96315b3d60ce-crio-socket\") pod \"insights-runtime-extractor-4p55m\" (UID: \"2e08e8ce-573f-4068-8c3b-96315b3d60ce\") " pod="openshift-insights/insights-runtime-extractor-4p55m"
Jul 28 23:22:38.537755 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:22:38.537740    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2e08e8ce-573f-4068-8c3b-96315b3d60ce-insights-runtime-extractor-tls podName:2e08e8ce-573f-4068-8c3b-96315b3d60ce nodeName:}" failed. No retries permitted until 2026-07-28 23:22:39.037720206 +0000 UTC m=+35.730394717 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/2e08e8ce-573f-4068-8c3b-96315b3d60ce-insights-runtime-extractor-tls") pod "insights-runtime-extractor-4p55m" (UID: "2e08e8ce-573f-4068-8c3b-96315b3d60ce") : secret "insights-runtime-extractor-tls" not found
Jul 28 23:22:38.538048 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:22:38.537665    2572 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 28 23:22:38.538048 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:22:38.537811    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2e08e8ce-573f-4068-8c3b-96315b3d60ce-kube-rbac-proxy-cm podName:2e08e8ce-573f-4068-8c3b-96315b3d60ce nodeName:}" failed. No retries permitted until 2026-07-28 23:22:39.037798111 +0000 UTC m=+35.730472636 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/2e08e8ce-573f-4068-8c3b-96315b3d60ce-kube-rbac-proxy-cm") pod "insights-runtime-extractor-4p55m" (UID: "2e08e8ce-573f-4068-8c3b-96315b3d60ce") : configmap "kube-rbac-proxy" not found
Jul 28 23:22:38.538048 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:38.537958    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/2e08e8ce-573f-4068-8c3b-96315b3d60ce-data-volume\") pod \"insights-runtime-extractor-4p55m\" (UID: \"2e08e8ce-573f-4068-8c3b-96315b3d60ce\") " pod="openshift-insights/insights-runtime-extractor-4p55m"
Jul 28 23:22:38.546996 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:38.546973    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8n5xw\" (UniqueName: \"kubernetes.io/projected/2e08e8ce-573f-4068-8c3b-96315b3d60ce-kube-api-access-8n5xw\") pod \"insights-runtime-extractor-4p55m\" (UID: \"2e08e8ce-573f-4068-8c3b-96315b3d60ce\") " pod="openshift-insights/insights-runtime-extractor-4p55m"
Jul 28 23:22:39.040938 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:39.040901    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2e08e8ce-573f-4068-8c3b-96315b3d60ce-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4p55m\" (UID: \"2e08e8ce-573f-4068-8c3b-96315b3d60ce\") " pod="openshift-insights/insights-runtime-extractor-4p55m"
Jul 28 23:22:39.040938 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:39.040943    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2e08e8ce-573f-4068-8c3b-96315b3d60ce-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4p55m\" (UID: \"2e08e8ce-573f-4068-8c3b-96315b3d60ce\") " pod="openshift-insights/insights-runtime-extractor-4p55m"
Jul 28 23:22:39.041556 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:22:39.041034    2572 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 28 23:22:39.041556 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:22:39.041052    2572 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 23:22:39.041556 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:22:39.041093    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2e08e8ce-573f-4068-8c3b-96315b3d60ce-kube-rbac-proxy-cm podName:2e08e8ce-573f-4068-8c3b-96315b3d60ce nodeName:}" failed. No retries permitted until 2026-07-28 23:22:40.041077868 +0000 UTC m=+36.733752376 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/2e08e8ce-573f-4068-8c3b-96315b3d60ce-kube-rbac-proxy-cm") pod "insights-runtime-extractor-4p55m" (UID: "2e08e8ce-573f-4068-8c3b-96315b3d60ce") : configmap "kube-rbac-proxy" not found
Jul 28 23:22:39.041556 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:22:39.041107    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2e08e8ce-573f-4068-8c3b-96315b3d60ce-insights-runtime-extractor-tls podName:2e08e8ce-573f-4068-8c3b-96315b3d60ce nodeName:}" failed. No retries permitted until 2026-07-28 23:22:40.041101142 +0000 UTC m=+36.733775629 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/2e08e8ce-573f-4068-8c3b-96315b3d60ce-insights-runtime-extractor-tls") pod "insights-runtime-extractor-4p55m" (UID: "2e08e8ce-573f-4068-8c3b-96315b3d60ce") : secret "insights-runtime-extractor-tls" not found
Jul 28 23:22:39.251591 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:39.251560    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-gwrtv"]
Jul 28 23:22:39.336725 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:39.336653    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad652290_064c_4b42_8419_bdd779497c86.slice/crio-24299900cc0728e0da22c055ae2747763df3759dc7a1758b215fc51f4b68d928 WatchSource:0}: Error finding container 24299900cc0728e0da22c055ae2747763df3759dc7a1758b215fc51f4b68d928: Status 404 returned error can't find the container with id 24299900cc0728e0da22c055ae2747763df3759dc7a1758b215fc51f4b68d928
Jul 28 23:22:39.444834 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:39.444803    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3d670d49-7bd6-47f9-befe-d76cce20834a-metrics-tls\") pod \"dns-default-8k4xj\" (UID: \"3d670d49-7bd6-47f9-befe-d76cce20834a\") " pod="openshift-dns/dns-default-8k4xj"
Jul 28 23:22:39.444981 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:22:39.444960    2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 23:22:39.445036 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:22:39.445026    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3d670d49-7bd6-47f9-befe-d76cce20834a-metrics-tls podName:3d670d49-7bd6-47f9-befe-d76cce20834a nodeName:}" failed. No retries permitted until 2026-07-28 23:22:41.445010576 +0000 UTC m=+38.137685062 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3d670d49-7bd6-47f9-befe-d76cce20834a-metrics-tls") pod "dns-default-8k4xj" (UID: "3d670d49-7bd6-47f9-befe-d76cce20834a") : secret "dns-default-metrics-tls" not found
Jul 28 23:22:40.050556 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:40.050328    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2e08e8ce-573f-4068-8c3b-96315b3d60ce-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4p55m\" (UID: \"2e08e8ce-573f-4068-8c3b-96315b3d60ce\") " pod="openshift-insights/insights-runtime-extractor-4p55m"
Jul 28 23:22:40.051245 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:40.050597    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2e08e8ce-573f-4068-8c3b-96315b3d60ce-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4p55m\" (UID: \"2e08e8ce-573f-4068-8c3b-96315b3d60ce\") " pod="openshift-insights/insights-runtime-extractor-4p55m"
Jul 28 23:22:40.051245 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:22:40.050507    2572 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 23:22:40.051245 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:22:40.050715    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2e08e8ce-573f-4068-8c3b-96315b3d60ce-insights-runtime-extractor-tls podName:2e08e8ce-573f-4068-8c3b-96315b3d60ce nodeName:}" failed. No retries permitted until 2026-07-28 23:22:42.050692786 +0000 UTC m=+38.743367286 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/2e08e8ce-573f-4068-8c3b-96315b3d60ce-insights-runtime-extractor-tls") pod "insights-runtime-extractor-4p55m" (UID: "2e08e8ce-573f-4068-8c3b-96315b3d60ce") : secret "insights-runtime-extractor-tls" not found
Jul 28 23:22:40.051245 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:22:40.050721    2572 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 28 23:22:40.051245 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:22:40.050764    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2e08e8ce-573f-4068-8c3b-96315b3d60ce-kube-rbac-proxy-cm podName:2e08e8ce-573f-4068-8c3b-96315b3d60ce nodeName:}" failed. No retries permitted until 2026-07-28 23:22:42.050750824 +0000 UTC m=+38.743425311 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/2e08e8ce-573f-4068-8c3b-96315b3d60ce-kube-rbac-proxy-cm") pod "insights-runtime-extractor-4p55m" (UID: "2e08e8ce-573f-4068-8c3b-96315b3d60ce") : configmap "kube-rbac-proxy" not found
Jul 28 23:22:40.113082 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:40.113046    2572 generic.go:356] "Generic (PLEG): container finished" podID="f220827f-670e-407f-b5c9-664800ba743d" containerID="96cef1e1f7d838a9eb44a7ed876bdacc593e49c5ab05a67ab7d1707fafff390f" exitCode=0
Jul 28 23:22:40.113267 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:40.113137    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-v6nmr" event={"ID":"f220827f-670e-407f-b5c9-664800ba743d","Type":"ContainerDied","Data":"96cef1e1f7d838a9eb44a7ed876bdacc593e49c5ab05a67ab7d1707fafff390f"}
Jul 28 23:22:40.114388 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:40.114365    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-gwrtv" event={"ID":"ad652290-064c-4b42-8419-bdd779497c86","Type":"ContainerStarted","Data":"24299900cc0728e0da22c055ae2747763df3759dc7a1758b215fc51f4b68d928"}
Jul 28 23:22:40.155424 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:40.155333    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-gvg5f"]
Jul 28 23:22:40.158012 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:40.157990    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-gvg5f"
Jul 28 23:22:40.160305 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:40.160278    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Jul 28 23:22:40.160572 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:40.160552    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Jul 28 23:22:40.160729 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:40.160711    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-gstg9\""
Jul 28 23:22:40.167048 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:40.165922    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-gvg5f"]
Jul 28 23:22:40.251829 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:40.251798    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tshb7\" (UniqueName: \"kubernetes.io/projected/de30b99d-4a8d-4307-8e38-ec8ca095c1e0-kube-api-access-tshb7\") pod \"migrator-6865c9967-gvg5f\" (UID: \"de30b99d-4a8d-4307-8e38-ec8ca095c1e0\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-gvg5f"
Jul 28 23:22:40.261898 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:40.261870    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-xqqk5_82ce0587-81a8-4b5e-b841-14fed1b2364c/dns-node-resolver/0.log"
Jul 28 23:22:40.352531 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:40.352417    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tshb7\" (UniqueName: \"kubernetes.io/projected/de30b99d-4a8d-4307-8e38-ec8ca095c1e0-kube-api-access-tshb7\") pod \"migrator-6865c9967-gvg5f\" (UID: \"de30b99d-4a8d-4307-8e38-ec8ca095c1e0\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-gvg5f"
Jul 28 23:22:40.360914 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:40.360881    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tshb7\" (UniqueName: \"kubernetes.io/projected/de30b99d-4a8d-4307-8e38-ec8ca095c1e0-kube-api-access-tshb7\") pod \"migrator-6865c9967-gvg5f\" (UID: \"de30b99d-4a8d-4307-8e38-ec8ca095c1e0\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-gvg5f"
Jul 28 23:22:40.470918 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:40.470883    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-gvg5f"
Jul 28 23:22:40.621601 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:40.621515    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-gvg5f"]
Jul 28 23:22:40.625174 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:40.625140    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde30b99d_4a8d_4307_8e38_ec8ca095c1e0.slice/crio-4a1318a9a909e726443ba5330e6e074051e7a103f51f66cb3762075935c157c8 WatchSource:0}: Error finding container 4a1318a9a909e726443ba5330e6e074051e7a103f51f66cb3762075935c157c8: Status 404 returned error can't find the container with id 4a1318a9a909e726443ba5330e6e074051e7a103f51f66cb3762075935c157c8
Jul 28 23:22:41.066721 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:41.066695    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-5ndfr_5c9cc37c-daaf-4fa1-b71e-94bcc6732adb/node-ca/0.log"
Jul 28 23:22:41.118714 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:41.118669    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-gvg5f" event={"ID":"de30b99d-4a8d-4307-8e38-ec8ca095c1e0","Type":"ContainerStarted","Data":"4a1318a9a909e726443ba5330e6e074051e7a103f51f66cb3762075935c157c8"}
Jul 28 23:22:41.122520 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:41.122491    2572 generic.go:356] "Generic (PLEG): container finished" podID="f220827f-670e-407f-b5c9-664800ba743d" containerID="71447778f8c7a72f1da92aedf5e0b6bce0a684025782219029e80c7f9b6de672" exitCode=0
Jul 28 23:22:41.122707 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:41.122535    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-v6nmr" event={"ID":"f220827f-670e-407f-b5c9-664800ba743d","Type":"ContainerDied","Data":"71447778f8c7a72f1da92aedf5e0b6bce0a684025782219029e80c7f9b6de672"}
Jul 28 23:22:41.465612 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:41.465508    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3d670d49-7bd6-47f9-befe-d76cce20834a-metrics-tls\") pod \"dns-default-8k4xj\" (UID: \"3d670d49-7bd6-47f9-befe-d76cce20834a\") " pod="openshift-dns/dns-default-8k4xj"
Jul 28 23:22:41.465805 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:22:41.465697    2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 23:22:41.465805 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:22:41.465780    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3d670d49-7bd6-47f9-befe-d76cce20834a-metrics-tls podName:3d670d49-7bd6-47f9-befe-d76cce20834a nodeName:}" failed. No retries permitted until 2026-07-28 23:22:45.465756084 +0000 UTC m=+42.158430594 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3d670d49-7bd6-47f9-befe-d76cce20834a-metrics-tls") pod "dns-default-8k4xj" (UID: "3d670d49-7bd6-47f9-befe-d76cce20834a") : secret "dns-default-metrics-tls" not found
Jul 28 23:22:42.071691 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:42.071459    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2e08e8ce-573f-4068-8c3b-96315b3d60ce-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4p55m\" (UID: \"2e08e8ce-573f-4068-8c3b-96315b3d60ce\") " pod="openshift-insights/insights-runtime-extractor-4p55m"
Jul 28 23:22:42.072149 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:22:42.071596    2572 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 28 23:22:42.072149 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:22:42.071829    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2e08e8ce-573f-4068-8c3b-96315b3d60ce-kube-rbac-proxy-cm podName:2e08e8ce-573f-4068-8c3b-96315b3d60ce nodeName:}" failed. No retries permitted until 2026-07-28 23:22:46.071808747 +0000 UTC m=+42.764483253 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/2e08e8ce-573f-4068-8c3b-96315b3d60ce-kube-rbac-proxy-cm") pod "insights-runtime-extractor-4p55m" (UID: "2e08e8ce-573f-4068-8c3b-96315b3d60ce") : configmap "kube-rbac-proxy" not found
Jul 28 23:22:42.072149 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:42.071854    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2e08e8ce-573f-4068-8c3b-96315b3d60ce-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4p55m\" (UID: \"2e08e8ce-573f-4068-8c3b-96315b3d60ce\") " pod="openshift-insights/insights-runtime-extractor-4p55m"
Jul 28 23:22:42.072149 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:22:42.071959    2572 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 23:22:42.072149 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:22:42.072005    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2e08e8ce-573f-4068-8c3b-96315b3d60ce-insights-runtime-extractor-tls podName:2e08e8ce-573f-4068-8c3b-96315b3d60ce nodeName:}" failed. No retries permitted until 2026-07-28 23:22:46.07199296 +0000 UTC m=+42.764667453 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/2e08e8ce-573f-4068-8c3b-96315b3d60ce-insights-runtime-extractor-tls") pod "insights-runtime-extractor-4p55m" (UID: "2e08e8ce-573f-4068-8c3b-96315b3d60ce") : secret "insights-runtime-extractor-tls" not found
Jul 28 23:22:42.128035 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:42.127995    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-v6nmr" event={"ID":"f220827f-670e-407f-b5c9-664800ba743d","Type":"ContainerStarted","Data":"00e1ab0a084c3a74efd2f2589e646522d5a5bac29adbf3f9463799336ff6cbba"}
Jul 28 23:22:42.129568 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:42.129543    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-gwrtv" event={"ID":"ad652290-064c-4b42-8419-bdd779497c86","Type":"ContainerStarted","Data":"a5e30077dbe678dcea8dff1eee86b2bd9a7e8b30036e539b842976195d8bdf39"}
Jul 28 23:22:42.149266 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:42.149213    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-v6nmr" podStartSLOduration=8.988683599 podStartE2EDuration="39.149198056s" totalImagesPullingTime="30.160514457s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 23:22:03 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 23:22:42.147588128 +0000 UTC m=+38.840262636" watchObservedRunningTime="2026-07-28 23:22:42.149198056 +0000 UTC m=+38.841872591"
Jul 28 23:22:42.161357 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:42.161301    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-gwrtv" podStartSLOduration=2.837877209 podStartE2EDuration="5.161287268s" totalImagesPullingTime="2.323410059s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 23:22:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 23:22:42.160952877 +0000 UTC m=+38.853627386" watchObservedRunningTime="2026-07-28 23:22:42.161287268 +0000 UTC m=+38.853961807"
Jul 28 23:22:42.247316 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:42.247283    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-658d7b7c78-64sgn"]
Jul 28 23:22:42.249401 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:42.249073    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-658d7b7c78-64sgn"
Jul 28 23:22:42.251305 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:42.251286    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Jul 28 23:22:42.251490 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:42.251446    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Jul 28 23:22:42.251490 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:42.251465    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Jul 28 23:22:42.251685 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:42.251672    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Jul 28 23:22:42.251781 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:42.251761    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-hjmpb\""
Jul 28 23:22:42.257219 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:42.257100    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-658d7b7c78-64sgn"]
Jul 28 23:22:42.375184 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:42.375104    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/298a891a-3bd6-460f-9896-8acc79426d2e-signing-key\") pod \"service-ca-658d7b7c78-64sgn\" (UID: \"298a891a-3bd6-460f-9896-8acc79426d2e\") " pod="openshift-service-ca/service-ca-658d7b7c78-64sgn"
Jul 28 23:22:42.375337 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:42.375202    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/298a891a-3bd6-460f-9896-8acc79426d2e-signing-cabundle\") pod \"service-ca-658d7b7c78-64sgn\" (UID: \"298a891a-3bd6-460f-9896-8acc79426d2e\") " pod="openshift-service-ca/service-ca-658d7b7c78-64sgn"
Jul 28 23:22:42.375337 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:42.375273    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fwn2h\" (UniqueName: \"kubernetes.io/projected/298a891a-3bd6-460f-9896-8acc79426d2e-kube-api-access-fwn2h\") pod \"service-ca-658d7b7c78-64sgn\" (UID: \"298a891a-3bd6-460f-9896-8acc79426d2e\") " pod="openshift-service-ca/service-ca-658d7b7c78-64sgn"
Jul 28 23:22:42.475653 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:42.475614    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fwn2h\" (UniqueName: \"kubernetes.io/projected/298a891a-3bd6-460f-9896-8acc79426d2e-kube-api-access-fwn2h\") pod \"service-ca-658d7b7c78-64sgn\" (UID: \"298a891a-3bd6-460f-9896-8acc79426d2e\") " pod="openshift-service-ca/service-ca-658d7b7c78-64sgn"
Jul 28 23:22:42.475728 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:42.475667    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/298a891a-3bd6-460f-9896-8acc79426d2e-signing-key\") pod \"service-ca-658d7b7c78-64sgn\" (UID: \"298a891a-3bd6-460f-9896-8acc79426d2e\") " pod="openshift-service-ca/service-ca-658d7b7c78-64sgn"
Jul 28 23:22:42.475822 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:42.475735    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/298a891a-3bd6-460f-9896-8acc79426d2e-signing-cabundle\") pod \"service-ca-658d7b7c78-64sgn\" (UID: \"298a891a-3bd6-460f-9896-8acc79426d2e\") " pod="openshift-service-ca/service-ca-658d7b7c78-64sgn"
Jul 28 23:22:42.476400 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:42.476380    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/298a891a-3bd6-460f-9896-8acc79426d2e-signing-cabundle\") pod \"service-ca-658d7b7c78-64sgn\" (UID: \"298a891a-3bd6-460f-9896-8acc79426d2e\") " pod="openshift-service-ca/service-ca-658d7b7c78-64sgn"
Jul 28 23:22:42.479291 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:42.479269    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/298a891a-3bd6-460f-9896-8acc79426d2e-signing-key\") pod \"service-ca-658d7b7c78-64sgn\" (UID: \"298a891a-3bd6-460f-9896-8acc79426d2e\") " pod="openshift-service-ca/service-ca-658d7b7c78-64sgn"
Jul 28 23:22:42.484101 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:42.484076    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fwn2h\" (UniqueName: \"kubernetes.io/projected/298a891a-3bd6-460f-9896-8acc79426d2e-kube-api-access-fwn2h\") pod \"service-ca-658d7b7c78-64sgn\" (UID: \"298a891a-3bd6-460f-9896-8acc79426d2e\") " pod="openshift-service-ca/service-ca-658d7b7c78-64sgn"
Jul 28 23:22:42.559442 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:42.559418    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-658d7b7c78-64sgn"
Jul 28 23:22:42.676831 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:42.676797    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-658d7b7c78-64sgn"]
Jul 28 23:22:42.680124 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:42.680098    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod298a891a_3bd6_460f_9896_8acc79426d2e.slice/crio-db5bbb47eefe779368606bd6770a59c2b03a5a1d392c7ec7a07ceae905cf3ff8 WatchSource:0}: Error finding container db5bbb47eefe779368606bd6770a59c2b03a5a1d392c7ec7a07ceae905cf3ff8: Status 404 returned error can't find the container with id db5bbb47eefe779368606bd6770a59c2b03a5a1d392c7ec7a07ceae905cf3ff8
Jul 28 23:22:43.133925 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:43.133880    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-gvg5f" event={"ID":"de30b99d-4a8d-4307-8e38-ec8ca095c1e0","Type":"ContainerStarted","Data":"50594857f19cf1c5870d7a48568a3d334b663360e529a1dabb7a49058e4b9f39"}
Jul 28 23:22:43.134478 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:43.133946    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-gvg5f" event={"ID":"de30b99d-4a8d-4307-8e38-ec8ca095c1e0","Type":"ContainerStarted","Data":"a5a66a31d51ac77902b3187a0b7d10f7e1d62baa29da367c9e301dc4985cd07a"}
Jul 28 23:22:43.134987 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:43.134961    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-658d7b7c78-64sgn" event={"ID":"298a891a-3bd6-460f-9896-8acc79426d2e","Type":"ContainerStarted","Data":"db5bbb47eefe779368606bd6770a59c2b03a5a1d392c7ec7a07ceae905cf3ff8"}
Jul 28 23:22:43.149419 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:43.149354    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-gvg5f" podStartSLOduration=1.301936223 podStartE2EDuration="3.149336501s" totalImagesPullingTime="1.847400278s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 23:22:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 23:22:43.149024555 +0000 UTC m=+39.841699063" watchObservedRunningTime="2026-07-28 23:22:43.149336501 +0000 UTC m=+39.842011010"
Jul 28 23:22:45.142678 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:45.142638    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-658d7b7c78-64sgn" event={"ID":"298a891a-3bd6-460f-9896-8acc79426d2e","Type":"ContainerStarted","Data":"84480ef57acccd92ffe23c569c0609141742bfafa3edd31905039f9890248fae"}
Jul 28 23:22:45.157769 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:45.157720    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-658d7b7c78-64sgn" podStartSLOduration=1.2971591949999999 podStartE2EDuration="3.15770522s" totalImagesPullingTime="1.860546025s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 23:22:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 23:22:45.155910366 +0000 UTC m=+41.848584874" watchObservedRunningTime="2026-07-28 23:22:45.15770522 +0000 UTC m=+41.850379762"
Jul 28 23:22:45.500368 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:45.500329    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3d670d49-7bd6-47f9-befe-d76cce20834a-metrics-tls\") pod \"dns-default-8k4xj\" (UID: \"3d670d49-7bd6-47f9-befe-d76cce20834a\") " pod="openshift-dns/dns-default-8k4xj"
Jul 28 23:22:45.500548 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:22:45.500504    2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 23:22:45.500609 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:22:45.500583    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3d670d49-7bd6-47f9-befe-d76cce20834a-metrics-tls podName:3d670d49-7bd6-47f9-befe-d76cce20834a nodeName:}" failed. No retries permitted until 2026-07-28 23:22:53.50056108 +0000 UTC m=+50.193235573 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3d670d49-7bd6-47f9-befe-d76cce20834a-metrics-tls") pod "dns-default-8k4xj" (UID: "3d670d49-7bd6-47f9-befe-d76cce20834a") : secret "dns-default-metrics-tls" not found
Jul 28 23:22:46.105704 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:46.105664    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2e08e8ce-573f-4068-8c3b-96315b3d60ce-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4p55m\" (UID: \"2e08e8ce-573f-4068-8c3b-96315b3d60ce\") " pod="openshift-insights/insights-runtime-extractor-4p55m"
Jul 28 23:22:46.105912 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:46.105811    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2e08e8ce-573f-4068-8c3b-96315b3d60ce-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4p55m\" (UID: \"2e08e8ce-573f-4068-8c3b-96315b3d60ce\") " pod="openshift-insights/insights-runtime-extractor-4p55m"
Jul 28 23:22:46.105912 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:22:46.105828    2572 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 28 23:22:46.105912 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:22:46.105909    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2e08e8ce-573f-4068-8c3b-96315b3d60ce-kube-rbac-proxy-cm podName:2e08e8ce-573f-4068-8c3b-96315b3d60ce nodeName:}" failed. No retries permitted until 2026-07-28 23:22:54.105888352 +0000 UTC m=+50.798562844 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/2e08e8ce-573f-4068-8c3b-96315b3d60ce-kube-rbac-proxy-cm") pod "insights-runtime-extractor-4p55m" (UID: "2e08e8ce-573f-4068-8c3b-96315b3d60ce") : configmap "kube-rbac-proxy" not found
Jul 28 23:22:46.106077 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:22:46.105930    2572 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 23:22:46.106077 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:22:46.105982    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2e08e8ce-573f-4068-8c3b-96315b3d60ce-insights-runtime-extractor-tls podName:2e08e8ce-573f-4068-8c3b-96315b3d60ce nodeName:}" failed. No retries permitted until 2026-07-28 23:22:54.105966353 +0000 UTC m=+50.798640847 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/2e08e8ce-573f-4068-8c3b-96315b3d60ce-insights-runtime-extractor-tls") pod "insights-runtime-extractor-4p55m" (UID: "2e08e8ce-573f-4068-8c3b-96315b3d60ce") : secret "insights-runtime-extractor-tls" not found
Jul 28 23:22:53.564406 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:53.564365    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3d670d49-7bd6-47f9-befe-d76cce20834a-metrics-tls\") pod \"dns-default-8k4xj\" (UID: \"3d670d49-7bd6-47f9-befe-d76cce20834a\") " pod="openshift-dns/dns-default-8k4xj"
Jul 28 23:22:53.566986 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:53.566951    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3d670d49-7bd6-47f9-befe-d76cce20834a-metrics-tls\") pod \"dns-default-8k4xj\" (UID: \"3d670d49-7bd6-47f9-befe-d76cce20834a\") " pod="openshift-dns/dns-default-8k4xj"
Jul 28 23:22:53.600109 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:53.600079    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-8k4xj"
Jul 28 23:22:53.736871 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:53.736838    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-8k4xj"]
Jul 28 23:22:53.745875 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:22:53.745847    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d670d49_7bd6_47f9_befe_d76cce20834a.slice/crio-2b3d9d70e358ef41f30023610d9494f3e3707a90e48db1d2eadd38bcc46d99b5 WatchSource:0}: Error finding container 2b3d9d70e358ef41f30023610d9494f3e3707a90e48db1d2eadd38bcc46d99b5: Status 404 returned error can't find the container with id 2b3d9d70e358ef41f30023610d9494f3e3707a90e48db1d2eadd38bcc46d99b5
Jul 28 23:22:54.168637 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:54.168583    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-8k4xj" event={"ID":"3d670d49-7bd6-47f9-befe-d76cce20834a","Type":"ContainerStarted","Data":"2b3d9d70e358ef41f30023610d9494f3e3707a90e48db1d2eadd38bcc46d99b5"}
Jul 28 23:22:54.168637 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:54.168635    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2e08e8ce-573f-4068-8c3b-96315b3d60ce-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4p55m\" (UID: \"2e08e8ce-573f-4068-8c3b-96315b3d60ce\") " pod="openshift-insights/insights-runtime-extractor-4p55m"
Jul 28 23:22:54.168880 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:54.168667    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2e08e8ce-573f-4068-8c3b-96315b3d60ce-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4p55m\" (UID: \"2e08e8ce-573f-4068-8c3b-96315b3d60ce\") " pod="openshift-insights/insights-runtime-extractor-4p55m"
Jul 28 23:22:54.168880 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:22:54.168756    2572 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 23:22:54.168880 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:22:54.168815    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2e08e8ce-573f-4068-8c3b-96315b3d60ce-insights-runtime-extractor-tls podName:2e08e8ce-573f-4068-8c3b-96315b3d60ce nodeName:}" failed. No retries permitted until 2026-07-28 23:23:10.168799172 +0000 UTC m=+66.861473659 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/2e08e8ce-573f-4068-8c3b-96315b3d60ce-insights-runtime-extractor-tls") pod "insights-runtime-extractor-4p55m" (UID: "2e08e8ce-573f-4068-8c3b-96315b3d60ce") : secret "insights-runtime-extractor-tls" not found
Jul 28 23:22:54.169340 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:54.169322    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2e08e8ce-573f-4068-8c3b-96315b3d60ce-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4p55m\" (UID: \"2e08e8ce-573f-4068-8c3b-96315b3d60ce\") " pod="openshift-insights/insights-runtime-extractor-4p55m"
Jul 28 23:22:56.176193 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:56.176156    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-8k4xj" event={"ID":"3d670d49-7bd6-47f9-befe-d76cce20834a","Type":"ContainerStarted","Data":"751f193af1ea69770550b10ea0b065cfa87c82e94bdfd83ea4ba8ce67158cb14"}
Jul 28 23:22:56.176686 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:56.176219    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-8k4xj" event={"ID":"3d670d49-7bd6-47f9-befe-d76cce20834a","Type":"ContainerStarted","Data":"a3d00883be55b236cf27ab5c6c9c3fb0ccbcd1c2073061ca7b26cfc9dce4f828"}
Jul 28 23:22:56.176686 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:56.176264    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-8k4xj"
Jul 28 23:22:56.192660 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:22:56.192588    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-8k4xj" podStartSLOduration=17.173120162 podStartE2EDuration="19.192573553s" totalImagesPullingTime="2.019453391s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 23:22:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 23:22:56.192213136 +0000 UTC m=+52.884887645" watchObservedRunningTime="2026-07-28 23:22:56.192573553 +0000 UTC m=+52.885248061"
Jul 28 23:23:01.582820 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:23:01.582779    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5cb487cc5c-ctcl2"]
Jul 28 23:23:01.585360 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:23:01.585341    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7b6747f5b8-blhkd"]
Jul 28 23:23:01.585503 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:23:01.585484    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5cb487cc5c-ctcl2"
Jul 28 23:23:01.587352 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:23:01.587329    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7b6747f5b8-blhkd"
Jul 28 23:23:01.587861 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:23:01.587841    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Jul 28 23:23:01.587962 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:23:01.587848    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Jul 28 23:23:01.587962 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:23:01.587848    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Jul 28 23:23:01.588047 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:23:01.587966    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 28 23:23:01.588920 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:23:01.588903    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 28 23:23:01.589000 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:23:01.588928    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 28 23:23:01.589000 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:23:01.588903    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Jul 28 23:23:01.589455 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:23:01.589437    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Jul 28 23:23:01.589507 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:23:01.589462    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-dkdk7\""
Jul 28 23:23:01.595693 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:23:01.595668    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7b6747f5b8-blhkd"]
Jul 28 23:23:01.598893 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:23:01.598873    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5cb487cc5c-ctcl2"]
Jul 28 23:23:01.621996 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:23:01.621969    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/cf9e10b1-7c21-41a9-bc28-1f4d63718a79-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7b6747f5b8-blhkd\" (UID: \"cf9e10b1-7c21-41a9-bc28-1f4d63718a79\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7b6747f5b8-blhkd"
Jul 28 23:23:01.622104 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:23:01.622000    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/aae0a6e3-f19b-468a-8419-040f90e07d19-ca\") pod \"cluster-proxy-proxy-agent-5cb487cc5c-ctcl2\" (UID: \"aae0a6e3-f19b-468a-8419-040f90e07d19\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5cb487cc5c-ctcl2"
Jul 28 23:23:01.622104 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:23:01.622018    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/aae0a6e3-f19b-468a-8419-040f90e07d19-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-5cb487cc5c-ctcl2\" (UID: \"aae0a6e3-f19b-468a-8419-040f90e07d19\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5cb487cc5c-ctcl2"
Jul 28 23:23:01.622178 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:23:01.622104    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/aae0a6e3-f19b-468a-8419-040f90e07d19-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-5cb487cc5c-ctcl2\" (UID: \"aae0a6e3-f19b-468a-8419-040f90e07d19\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5cb487cc5c-ctcl2"
Jul 28 23:23:01.622178 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:23:01.622128    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/aae0a6e3-f19b-468a-8419-040f90e07d19-additional-service-ca\") pod \"cluster-proxy-proxy-agent-5cb487cc5c-ctcl2\" (UID: \"aae0a6e3-f19b-468a-8419-040f90e07d19\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5cb487cc5c-ctcl2"
Jul 28 23:23:01.622240 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:23:01.622172    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-df29f\" (UniqueName: \"kubernetes.io/projected/aae0a6e3-f19b-468a-8419-040f90e07d19-kube-api-access-df29f\") pod \"cluster-proxy-proxy-agent-5cb487cc5c-ctcl2\" (UID: \"aae0a6e3-f19b-468a-8419-040f90e07d19\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5cb487cc5c-ctcl2"
Jul 28 23:23:01.622273 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:23:01.622240    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/aae0a6e3-f19b-468a-8419-040f90e07d19-hub\") pod \"cluster-proxy-proxy-agent-5cb487cc5c-ctcl2\" (UID: \"aae0a6e3-f19b-468a-8419-040f90e07d19\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5cb487cc5c-ctcl2"
Jul 28 23:23:01.622273 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:23:01.622264    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vfxks\" (UniqueName: \"kubernetes.io/projected/cf9e10b1-7c21-41a9-bc28-1f4d63718a79-kube-api-access-vfxks\") pod \"managed-serviceaccount-addon-agent-7b6747f5b8-blhkd\" (UID: \"cf9e10b1-7c21-41a9-bc28-1f4d63718a79\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7b6747f5b8-blhkd"
Jul 28 23:23:01.723302 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:23:01.723266    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/aae0a6e3-f19b-468a-8419-040f90e07d19-hub\") pod \"cluster-proxy-proxy-agent-5cb487cc5c-ctcl2\" (UID: \"aae0a6e3-f19b-468a-8419-040f90e07d19\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5cb487cc5c-ctcl2"
Jul 28 23:23:01.723302 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:23:01.723300    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vfxks\" (UniqueName: \"kubernetes.io/projected/cf9e10b1-7c21-41a9-bc28-1f4d63718a79-kube-api-access-vfxks\") pod \"managed-serviceaccount-addon-agent-7b6747f5b8-blhkd\" (UID: \"cf9e10b1-7c21-41a9-bc28-1f4d63718a79\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7b6747f5b8-blhkd"
Jul 28 23:23:01.723550 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:23:01.723321    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/cf9e10b1-7c21-41a9-bc28-1f4d63718a79-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7b6747f5b8-blhkd\" (UID: \"cf9e10b1-7c21-41a9-bc28-1f4d63718a79\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7b6747f5b8-blhkd"
Jul 28 23:23:01.723550 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:23:01.723337    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/aae0a6e3-f19b-468a-8419-040f90e07d19-ca\") pod \"cluster-proxy-proxy-agent-5cb487cc5c-ctcl2\" (UID: \"aae0a6e3-f19b-468a-8419-040f90e07d19\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5cb487cc5c-ctcl2"
Jul 28 23:23:01.723550 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:23:01.723362    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/aae0a6e3-f19b-468a-8419-040f90e07d19-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-5cb487cc5c-ctcl2\" (UID: \"aae0a6e3-f19b-468a-8419-040f90e07d19\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5cb487cc5c-ctcl2"
Jul 28 23:23:01.723751 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:23:01.723547    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/aae0a6e3-f19b-468a-8419-040f90e07d19-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-5cb487cc5c-ctcl2\" (UID: \"aae0a6e3-f19b-468a-8419-040f90e07d19\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5cb487cc5c-ctcl2"
Jul 28 23:23:01.723751 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:23:01.723584    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/aae0a6e3-f19b-468a-8419-040f90e07d19-additional-service-ca\") pod \"cluster-proxy-proxy-agent-5cb487cc5c-ctcl2\" (UID: \"aae0a6e3-f19b-468a-8419-040f90e07d19\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5cb487cc5c-ctcl2"
Jul 28 23:23:01.723751 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:23:01.723617    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-df29f\" (UniqueName: \"kubernetes.io/projected/aae0a6e3-f19b-468a-8419-040f90e07d19-kube-api-access-df29f\") pod \"cluster-proxy-proxy-agent-5cb487cc5c-ctcl2\" (UID: \"aae0a6e3-f19b-468a-8419-040f90e07d19\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5cb487cc5c-ctcl2"
Jul 28 23:23:01.724325 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:23:01.724305    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/aae0a6e3-f19b-468a-8419-040f90e07d19-additional-service-ca\") pod \"cluster-proxy-proxy-agent-5cb487cc5c-ctcl2\" (UID: \"aae0a6e3-f19b-468a-8419-040f90e07d19\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5cb487cc5c-ctcl2"
Jul 28 23:23:01.725977 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:23:01.725954    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/aae0a6e3-f19b-468a-8419-040f90e07d19-hub\") pod \"cluster-proxy-proxy-agent-5cb487cc5c-ctcl2\" (UID: \"aae0a6e3-f19b-468a-8419-040f90e07d19\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5cb487cc5c-ctcl2"
Jul 28 23:23:01.726386 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:23:01.726357    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/aae0a6e3-f19b-468a-8419-040f90e07d19-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-5cb487cc5c-ctcl2\" (UID: \"aae0a6e3-f19b-468a-8419-040f90e07d19\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5cb487cc5c-ctcl2"
Jul 28 23:23:01.726386 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:23:01.726383    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/aae0a6e3-f19b-468a-8419-040f90e07d19-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-5cb487cc5c-ctcl2\" (UID: \"aae0a6e3-f19b-468a-8419-040f90e07d19\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5cb487cc5c-ctcl2"
Jul 28 23:23:01.726497 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:23:01.726447    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/aae0a6e3-f19b-468a-8419-040f90e07d19-ca\") pod \"cluster-proxy-proxy-agent-5cb487cc5c-ctcl2\" (UID: \"aae0a6e3-f19b-468a-8419-040f90e07d19\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5cb487cc5c-ctcl2"
Jul 28 23:23:01.726536 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:23:01.726506    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/cf9e10b1-7c21-41a9-bc28-1f4d63718a79-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7b6747f5b8-blhkd\" (UID: \"cf9e10b1-7c21-41a9-bc28-1f4d63718a79\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7b6747f5b8-blhkd"
Jul 28 23:23:01.733446 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:23:01.733423    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vfxks\" (UniqueName: \"kubernetes.io/projected/cf9e10b1-7c21-41a9-bc28-1f4d63718a79-kube-api-access-vfxks\") pod \"managed-serviceaccount-addon-agent-7b6747f5b8-blhkd\" (UID: \"cf9e10b1-7c21-41a9-bc28-1f4d63718a79\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7b6747f5b8-blhkd"
Jul 28 23:23:01.733540 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:23:01.733503    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-df29f\" (UniqueName: \"kubernetes.io/projected/aae0a6e3-f19b-468a-8419-040f90e07d19-kube-api-access-df29f\") pod \"cluster-proxy-proxy-agent-5cb487cc5c-ctcl2\" (UID: \"aae0a6e3-f19b-468a-8419-040f90e07d19\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5cb487cc5c-ctcl2"
Jul 28 23:23:01.903491 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:23:01.903396    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5cb487cc5c-ctcl2"
Jul 28 23:23:01.908650 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:23:01.908612    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7b6747f5b8-blhkd"
Jul 28 23:23:02.039338 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:23:02.039302    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5cb487cc5c-ctcl2"]
Jul 28 23:23:02.042426 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:23:02.042400    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podaae0a6e3_f19b_468a_8419_040f90e07d19.slice/crio-b3b4e5183f9e01f46a8d4debd873d8f112d4fde12f494f94df0459a52ae1bf5f WatchSource:0}: Error finding container b3b4e5183f9e01f46a8d4debd873d8f112d4fde12f494f94df0459a52ae1bf5f: Status 404 returned error can't find the container with id b3b4e5183f9e01f46a8d4debd873d8f112d4fde12f494f94df0459a52ae1bf5f
Jul 28 23:23:02.054797 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:23:02.054773    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7b6747f5b8-blhkd"]
Jul 28 23:23:02.057259 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:23:02.057239    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcf9e10b1_7c21_41a9_bc28_1f4d63718a79.slice/crio-bf422b7d9f7f4569b14d479586212881df95b457bc2d3a98836d9640e238ba80 WatchSource:0}: Error finding container bf422b7d9f7f4569b14d479586212881df95b457bc2d3a98836d9640e238ba80: Status 404 returned error can't find the container with id bf422b7d9f7f4569b14d479586212881df95b457bc2d3a98836d9640e238ba80
Jul 28 23:23:02.199551 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:23:02.199467    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7b6747f5b8-blhkd" event={"ID":"cf9e10b1-7c21-41a9-bc28-1f4d63718a79","Type":"ContainerStarted","Data":"bf422b7d9f7f4569b14d479586212881df95b457bc2d3a98836d9640e238ba80"}
Jul 28 23:23:02.200484 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:23:02.200462    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5cb487cc5c-ctcl2" event={"ID":"aae0a6e3-f19b-468a-8419-040f90e07d19","Type":"ContainerStarted","Data":"b3b4e5183f9e01f46a8d4debd873d8f112d4fde12f494f94df0459a52ae1bf5f"}
Jul 28 23:23:04.111985 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:23:04.111952    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-v84z7"
Jul 28 23:23:05.216886 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:23:05.216841    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7b6747f5b8-blhkd" event={"ID":"cf9e10b1-7c21-41a9-bc28-1f4d63718a79","Type":"ContainerStarted","Data":"3799e9d28a629d0d9ebdf87462ea303517c0b549a29621a032863edb33eb1729"}
Jul 28 23:23:05.230567 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:23:05.230491    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7b6747f5b8-blhkd" podStartSLOduration=1.185424399 podStartE2EDuration="4.230475463s" totalImagesPullingTime="3.045051064s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 23:23:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 23:23:05.230452947 +0000 UTC m=+61.923127456" watchObservedRunningTime="2026-07-28 23:23:05.230475463 +0000 UTC m=+61.923149976"
Jul 28 23:23:06.182211 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:23:06.182175    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-8k4xj"
Jul 28 23:23:07.225555 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:23:07.225473    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5cb487cc5c-ctcl2" event={"ID":"aae0a6e3-f19b-468a-8419-040f90e07d19","Type":"ContainerStarted","Data":"25c400e0786cc5b8e00072b5ad9d18f7bc3277de82c00e75adcf61cbbecd62a0"}
Jul 28 23:23:07.225555 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:23:07.225517    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5cb487cc5c-ctcl2" event={"ID":"aae0a6e3-f19b-468a-8419-040f90e07d19","Type":"ContainerStarted","Data":"dec8b0743acb8bbc0c8976076fca08600ed6825b5d2336c2afc9936f917b11de"}
Jul 28 23:23:07.225555 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:23:07.225530    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5cb487cc5c-ctcl2" event={"ID":"aae0a6e3-f19b-468a-8419-040f90e07d19","Type":"ContainerStarted","Data":"e20c9ec7190ae3d51eb8820f2b4aec855c1bc4337ddafb7b212b50036aa7981f"}
Jul 28 23:23:07.242304 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:23:07.242254    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5cb487cc5c-ctcl2" podStartSLOduration=1.540147144 podStartE2EDuration="6.242238612s" totalImagesPullingTime="4.702091468s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 23:23:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 23:23:07.241193462 +0000 UTC m=+63.933867974" watchObservedRunningTime="2026-07-28 23:23:07.242238612 +0000 UTC m=+63.934913121"
Jul 28 23:23:08.580206 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:23:08.580157    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f0137726-3966-44f4-bcc7-c69cc72143be-original-pull-secret\") pod \"global-pull-secret-syncer-8pczd\" (UID: \"f0137726-3966-44f4-bcc7-c69cc72143be\") " pod="kube-system/global-pull-secret-syncer-8pczd"
Jul 28 23:23:08.580644 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:23:08.580250    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/df9e50e4-7650-40bb-83eb-becf4ba8c051-metrics-certs\") pod \"network-metrics-daemon-lpj5x\" (UID: \"df9e50e4-7650-40bb-83eb-becf4ba8c051\") " pod="openshift-multus/network-metrics-daemon-lpj5x"
Jul 28 23:23:08.582535 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:23:08.582506    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 28 23:23:08.582673 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:23:08.582507    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 28 23:23:08.593449 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:23:08.593423    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/df9e50e4-7650-40bb-83eb-becf4ba8c051-metrics-certs\") pod \"network-metrics-daemon-lpj5x\" (UID: \"df9e50e4-7650-40bb-83eb-becf4ba8c051\") " pod="openshift-multus/network-metrics-daemon-lpj5x"
Jul 28 23:23:08.593544 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:23:08.593468    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f0137726-3966-44f4-bcc7-c69cc72143be-original-pull-secret\") pod \"global-pull-secret-syncer-8pczd\" (UID: \"f0137726-3966-44f4-bcc7-c69cc72143be\") " pod="kube-system/global-pull-secret-syncer-8pczd"
Jul 28 23:23:08.681164 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:23:08.681119    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n2jnx\" (UniqueName: \"kubernetes.io/projected/ae951cd2-ca5f-4ac4-abaa-3293c8f1ea57-kube-api-access-n2jnx\") pod \"network-check-target-2svls\" (UID: \"ae951cd2-ca5f-4ac4-abaa-3293c8f1ea57\") " pod="openshift-network-diagnostics/network-check-target-2svls"
Jul 28 23:23:08.683534 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:23:08.683515    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 28 23:23:08.693115 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:23:08.693100    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 28 23:23:08.703815 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:23:08.703786    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n2jnx\" (UniqueName: \"kubernetes.io/projected/ae951cd2-ca5f-4ac4-abaa-3293c8f1ea57-kube-api-access-n2jnx\") pod \"network-check-target-2svls\" (UID: \"ae951cd2-ca5f-4ac4-abaa-3293c8f1ea57\") " pod="openshift-network-diagnostics/network-check-target-2svls"
Jul 28 23:23:08.772753 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:23:08.772729    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-bhxv7\""
Jul 28 23:23:08.777712 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:23:08.777696    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8pczd"
Jul 28 23:23:08.781515 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:23:08.781500    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2svls"
Jul 28 23:23:08.785179 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:23:08.785159    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-plrk7\""
Jul 28 23:23:08.793019 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:23:08.792996    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lpj5x"
Jul 28 23:23:08.916410 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:23:08.916382    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-2svls"]
Jul 28 23:23:08.919323 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:23:08.919292    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae951cd2_ca5f_4ac4_abaa_3293c8f1ea57.slice/crio-c4ead7f7380307395b65857a0cd3df2136688fd9d1312ada46798813e6f2d4ec WatchSource:0}: Error finding container c4ead7f7380307395b65857a0cd3df2136688fd9d1312ada46798813e6f2d4ec: Status 404 returned error can't find the container with id c4ead7f7380307395b65857a0cd3df2136688fd9d1312ada46798813e6f2d4ec
Jul 28 23:23:08.934591 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:23:08.934563    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-8pczd"]
Jul 28 23:23:08.937100 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:23:08.937070    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0137726_3966_44f4_bcc7_c69cc72143be.slice/crio-b4d25095529389dde7f24fbcafd352a94e4d061391e34ebd3a2e789cc295ee2e WatchSource:0}: Error finding container b4d25095529389dde7f24fbcafd352a94e4d061391e34ebd3a2e789cc295ee2e: Status 404 returned error can't find the container with id b4d25095529389dde7f24fbcafd352a94e4d061391e34ebd3a2e789cc295ee2e
Jul 28 23:23:08.950595 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:23:08.950574    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-lpj5x"]
Jul 28 23:23:08.953308 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:23:08.953285    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf9e50e4_7650_40bb_83eb_becf4ba8c051.slice/crio-fcfb32143caa900664369a99299d6ac790634ccb45ef8f3ae0eb1f6ac0bb9c11 WatchSource:0}: Error finding container fcfb32143caa900664369a99299d6ac790634ccb45ef8f3ae0eb1f6ac0bb9c11: Status 404 returned error can't find the container with id fcfb32143caa900664369a99299d6ac790634ccb45ef8f3ae0eb1f6ac0bb9c11
Jul 28 23:23:09.233986 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:23:09.233953    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-2svls" event={"ID":"ae951cd2-ca5f-4ac4-abaa-3293c8f1ea57","Type":"ContainerStarted","Data":"c4ead7f7380307395b65857a0cd3df2136688fd9d1312ada46798813e6f2d4ec"}
Jul 28 23:23:09.235680 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:23:09.235650    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-8pczd" event={"ID":"f0137726-3966-44f4-bcc7-c69cc72143be","Type":"ContainerStarted","Data":"b4d25095529389dde7f24fbcafd352a94e4d061391e34ebd3a2e789cc295ee2e"}
Jul 28 23:23:09.237068 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:23:09.237041    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-lpj5x" event={"ID":"df9e50e4-7650-40bb-83eb-becf4ba8c051","Type":"ContainerStarted","Data":"fcfb32143caa900664369a99299d6ac790634ccb45ef8f3ae0eb1f6ac0bb9c11"}
Jul 28 23:23:10.195461 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:23:10.195427    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2e08e8ce-573f-4068-8c3b-96315b3d60ce-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4p55m\" (UID: \"2e08e8ce-573f-4068-8c3b-96315b3d60ce\") " pod="openshift-insights/insights-runtime-extractor-4p55m"
Jul 28 23:23:10.198140 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:23:10.198108    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2e08e8ce-573f-4068-8c3b-96315b3d60ce-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4p55m\" (UID: \"2e08e8ce-573f-4068-8c3b-96315b3d60ce\") " pod="openshift-insights/insights-runtime-extractor-4p55m"
Jul 28 23:23:10.249749 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:23:10.249715    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-lpj5x" event={"ID":"df9e50e4-7650-40bb-83eb-becf4ba8c051","Type":"ContainerStarted","Data":"796c444a4eae9f7b0ceb2374250429cd03193d5710dd4b0bc09d3a529df99f63"}
Jul 28 23:23:10.411721 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:23:10.411633    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-xh4g8\""
Jul 28 23:23:10.419473 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:23:10.419443    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-4p55m"
Jul 28 23:23:10.573251 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:23:10.573217    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-4p55m"]
Jul 28 23:23:10.576601 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:23:10.576568    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e08e8ce_573f_4068_8c3b_96315b3d60ce.slice/crio-308daa5c39e471f4f5a3cd614bf45206f81b4783852c1d434268897fe93489ad WatchSource:0}: Error finding container 308daa5c39e471f4f5a3cd614bf45206f81b4783852c1d434268897fe93489ad: Status 404 returned error can't find the container with id 308daa5c39e471f4f5a3cd614bf45206f81b4783852c1d434268897fe93489ad
Jul 28 23:23:11.257049 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:23:11.257014    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-lpj5x" event={"ID":"df9e50e4-7650-40bb-83eb-becf4ba8c051","Type":"ContainerStarted","Data":"c022f206ec9caa164cac06b2505593b9dc1177b47705b7b8e32b5b2361be6bc3"}
Jul 28 23:23:11.258531 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:23:11.258463    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-4p55m" event={"ID":"2e08e8ce-573f-4068-8c3b-96315b3d60ce","Type":"ContainerStarted","Data":"308daa5c39e471f4f5a3cd614bf45206f81b4783852c1d434268897fe93489ad"}
Jul 28 23:23:11.272906 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:23:11.271566    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-lpj5x" podStartSLOduration=67.119185451 podStartE2EDuration="1m8.271546511s" totalImagesPullingTime="1.15236106s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 23:22:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 23:23:11.271383503 +0000 UTC m=+67.964058013" watchObservedRunningTime="2026-07-28 23:23:11.271546511 +0000 UTC m=+67.964221023"
Jul 28 23:23:17.283236 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:23:17.283139    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-8pczd" event={"ID":"f0137726-3966-44f4-bcc7-c69cc72143be","Type":"ContainerStarted","Data":"91ddd12e12ac9493d10398cc4bcd663c949e88acd6f7201e41985ae10d1d5986"}
Jul 28 23:23:17.284724 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:23:17.284701    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-2svls" event={"ID":"ae951cd2-ca5f-4ac4-abaa-3293c8f1ea57","Type":"ContainerStarted","Data":"eca946e67c03b6d7a596ae797fafc29c13d4e738996f9af305afb9fcb3d6131d"}
Jul 28 23:23:17.284854 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:23:17.284787    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-2svls"
Jul 28 23:23:17.286268 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:23:17.286246    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-4p55m" event={"ID":"2e08e8ce-573f-4068-8c3b-96315b3d60ce","Type":"ContainerStarted","Data":"7b071097a358b9d5b32f3aeddcd1313f330b3764206dce10ffc075a5e198cbc3"}
Jul 28 23:23:17.296542 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:23:17.296496    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-8pczd" podStartSLOduration=66.209296949 podStartE2EDuration="1m14.296481123s" totalImagesPullingTime="8.087184174s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 23:22:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 23:23:17.295882559 +0000 UTC m=+73.988557070" watchObservedRunningTime="2026-07-28 23:23:17.296481123 +0000 UTC m=+73.989155839"
Jul 28 23:23:17.310303 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:23:17.310255    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-2svls" podStartSLOduration=66.20637739 podStartE2EDuration="1m14.31024273s" totalImagesPullingTime="8.10386534s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 23:22:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 23:23:17.309712142 +0000 UTC m=+74.002386653" watchObservedRunningTime="2026-07-28 23:23:17.31024273 +0000 UTC m=+74.002917239"
Jul 28 23:23:21.305162 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:23:21.305124    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-4p55m" event={"ID":"2e08e8ce-573f-4068-8c3b-96315b3d60ce","Type":"ContainerStarted","Data":"120a144051e05d03028bf7a48346b2af9ee4aa920593f82690ce34af7535e081"}
Jul 28 23:23:21.798798 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:23:21.797993    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-ft86p"]
Jul 28 23:23:21.800432 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:23:21.800408    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-ft86p"
Jul 28 23:23:21.803436 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:23:21.803410    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 28 23:23:21.803723 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:23:21.803705    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-8ncnk\""
Jul 28 23:23:21.803826 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:23:21.803740    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 28 23:23:21.804761 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:23:21.804446    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 28 23:23:21.804761 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:23:21.804479    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 28 23:23:21.804761 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:23:21.804499    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 28 23:23:21.804761 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:23:21.804448    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 28 23:23:21.894723 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:23:21.894687    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/40858a66-ae35-4c67-850b-2ee7c1d4ace3-sys\") pod \"node-exporter-ft86p\" (UID: \"40858a66-ae35-4c67-850b-2ee7c1d4ace3\") " pod="openshift-monitoring/node-exporter-ft86p"
Jul 28 23:23:21.894899 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:23:21.894739    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/40858a66-ae35-4c67-850b-2ee7c1d4ace3-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-ft86p\" (UID: \"40858a66-ae35-4c67-850b-2ee7c1d4ace3\") " pod="openshift-monitoring/node-exporter-ft86p"
Jul 28 23:23:21.894899 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:23:21.894778    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rd6jk\" (UniqueName: \"kubernetes.io/projected/40858a66-ae35-4c67-850b-2ee7c1d4ace3-kube-api-access-rd6jk\") pod \"node-exporter-ft86p\" (UID: \"40858a66-ae35-4c67-850b-2ee7c1d4ace3\") " pod="openshift-monitoring/node-exporter-ft86p"
Jul 28 23:23:21.894899 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:23:21.894802    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/40858a66-ae35-4c67-850b-2ee7c1d4ace3-root\") pod \"node-exporter-ft86p\" (UID: \"40858a66-ae35-4c67-850b-2ee7c1d4ace3\") " pod="openshift-monitoring/node-exporter-ft86p"
Jul 28 23:23:21.894899 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:23:21.894830    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/40858a66-ae35-4c67-850b-2ee7c1d4ace3-node-exporter-wtmp\") pod \"node-exporter-ft86p\" (UID: \"40858a66-ae35-4c67-850b-2ee7c1d4ace3\") " pod="openshift-monitoring/node-exporter-ft86p"
Jul 28 23:23:21.894899 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:23:21.894878    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/40858a66-ae35-4c67-850b-2ee7c1d4ace3-node-exporter-textfile\") pod \"node-exporter-ft86p\" (UID: \"40858a66-ae35-4c67-850b-2ee7c1d4ace3\") " pod="openshift-monitoring/node-exporter-ft86p"
Jul 28 23:23:21.895167 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:23:21.894916    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/40858a66-ae35-4c67-850b-2ee7c1d4ace3-metrics-client-ca\") pod \"node-exporter-ft86p\" (UID: \"40858a66-ae35-4c67-850b-2ee7c1d4ace3\") " pod="openshift-monitoring/node-exporter-ft86p"
Jul 28 23:23:21.895167 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:23:21.894943    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/40858a66-ae35-4c67-850b-2ee7c1d4ace3-node-exporter-tls\") pod \"node-exporter-ft86p\" (UID: \"40858a66-ae35-4c67-850b-2ee7c1d4ace3\") " pod="openshift-monitoring/node-exporter-ft86p"
Jul 28 23:23:21.895167 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:23:21.895011    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/40858a66-ae35-4c67-850b-2ee7c1d4ace3-node-exporter-accelerators-collector-config\") pod \"node-exporter-ft86p\" (UID: \"40858a66-ae35-4c67-850b-2ee7c1d4ace3\") " pod="openshift-monitoring/node-exporter-ft86p"
Jul 28 23:23:21.996275 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:23:21.996234    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rd6jk\" (UniqueName: \"kubernetes.io/projected/40858a66-ae35-4c67-850b-2ee7c1d4ace3-kube-api-access-rd6jk\") pod \"node-exporter-ft86p\" (UID: \"40858a66-ae35-4c67-850b-2ee7c1d4ace3\") " pod="openshift-monitoring/node-exporter-ft86p"
Jul 28 23:23:21.996470 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:23:21.996284    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/40858a66-ae35-4c67-850b-2ee7c1d4ace3-root\") pod \"node-exporter-ft86p\" (UID: \"40858a66-ae35-4c67-850b-2ee7c1d4ace3\") " pod="openshift-monitoring/node-exporter-ft86p"
Jul 28 23:23:21.996470 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:23:21.996314    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/40858a66-ae35-4c67-850b-2ee7c1d4ace3-node-exporter-wtmp\") pod \"node-exporter-ft86p\" (UID: \"40858a66-ae35-4c67-850b-2ee7c1d4ace3\") " pod="openshift-monitoring/node-exporter-ft86p"
Jul 28 23:23:21.996470 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:23:21.996344    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/40858a66-ae35-4c67-850b-2ee7c1d4ace3-node-exporter-textfile\") pod \"node-exporter-ft86p\" (UID: \"40858a66-ae35-4c67-850b-2ee7c1d4ace3\") " pod="openshift-monitoring/node-exporter-ft86p"
Jul 28 23:23:21.996470 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:23:21.996382    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/40858a66-ae35-4c67-850b-2ee7c1d4ace3-metrics-client-ca\") pod \"node-exporter-ft86p\" (UID: \"40858a66-ae35-4c67-850b-2ee7c1d4ace3\") " pod="openshift-monitoring/node-exporter-ft86p"
Jul 28 23:23:21.996470 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:23:21.996406    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/40858a66-ae35-4c67-850b-2ee7c1d4ace3-node-exporter-tls\") pod \"node-exporter-ft86p\" (UID: \"40858a66-ae35-4c67-850b-2ee7c1d4ace3\") " pod="openshift-monitoring/node-exporter-ft86p"
Jul 28 23:23:21.996470 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:23:21.996435    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/40858a66-ae35-4c67-850b-2ee7c1d4ace3-node-exporter-accelerators-collector-config\") pod \"node-exporter-ft86p\" (UID: \"40858a66-ae35-4c67-850b-2ee7c1d4ace3\") " pod="openshift-monitoring/node-exporter-ft86p"
Jul 28 23:23:21.996792 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:23:21.996473    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/40858a66-ae35-4c67-850b-2ee7c1d4ace3-sys\") pod \"node-exporter-ft86p\" (UID: \"40858a66-ae35-4c67-850b-2ee7c1d4ace3\") " pod="openshift-monitoring/node-exporter-ft86p"
Jul 28 23:23:21.996792 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:23:21.996509    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/40858a66-ae35-4c67-850b-2ee7c1d4ace3-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-ft86p\" (UID: \"40858a66-ae35-4c67-850b-2ee7c1d4ace3\") " pod="openshift-monitoring/node-exporter-ft86p"
Jul 28 23:23:21.997444 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:23:21.997059    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/40858a66-ae35-4c67-850b-2ee7c1d4ace3-node-exporter-wtmp\") pod \"node-exporter-ft86p\" (UID: \"40858a66-ae35-4c67-850b-2ee7c1d4ace3\") " pod="openshift-monitoring/node-exporter-ft86p"
Jul 28 23:23:21.997444 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:23:21.997125    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/40858a66-ae35-4c67-850b-2ee7c1d4ace3-root\") pod \"node-exporter-ft86p\" (UID: \"40858a66-ae35-4c67-850b-2ee7c1d4ace3\") " pod="openshift-monitoring/node-exporter-ft86p"
Jul 28 23:23:21.997444 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:23:21.997162    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/40858a66-ae35-4c67-850b-2ee7c1d4ace3-sys\") pod \"node-exporter-ft86p\" (UID: \"40858a66-ae35-4c67-850b-2ee7c1d4ace3\") " pod="openshift-monitoring/node-exporter-ft86p"
Jul 28 23:23:21.997444 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:23:21.997233    2572 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 28 23:23:21.997444 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:23:21.997267    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/40858a66-ae35-4c67-850b-2ee7c1d4ace3-metrics-client-ca\") pod \"node-exporter-ft86p\" (UID: \"40858a66-ae35-4c67-850b-2ee7c1d4ace3\") " pod="openshift-monitoring/node-exporter-ft86p"
Jul 28 23:23:21.997444 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:23:21.997276    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/40858a66-ae35-4c67-850b-2ee7c1d4ace3-node-exporter-tls podName:40858a66-ae35-4c67-850b-2ee7c1d4ace3 nodeName:}" failed. No retries permitted until 2026-07-28 23:23:22.497261727 +0000 UTC m=+79.189936214 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/40858a66-ae35-4c67-850b-2ee7c1d4ace3-node-exporter-tls") pod "node-exporter-ft86p" (UID: "40858a66-ae35-4c67-850b-2ee7c1d4ace3") : secret "node-exporter-tls" not found
Jul 28 23:23:21.997444 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:23:21.997443    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/40858a66-ae35-4c67-850b-2ee7c1d4ace3-node-exporter-accelerators-collector-config\") pod \"node-exporter-ft86p\" (UID: \"40858a66-ae35-4c67-850b-2ee7c1d4ace3\") " pod="openshift-monitoring/node-exporter-ft86p"
Jul 28 23:23:21.997910 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:23:21.997538    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/40858a66-ae35-4c67-850b-2ee7c1d4ace3-node-exporter-textfile\") pod \"node-exporter-ft86p\" (UID: \"40858a66-ae35-4c67-850b-2ee7c1d4ace3\") " pod="openshift-monitoring/node-exporter-ft86p"
Jul 28 23:23:21.999281 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:23:21.999257    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/40858a66-ae35-4c67-850b-2ee7c1d4ace3-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-ft86p\" (UID: \"40858a66-ae35-4c67-850b-2ee7c1d4ace3\") " pod="openshift-monitoring/node-exporter-ft86p"
Jul 28 23:23:22.005383 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:23:22.005335    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rd6jk\" (UniqueName: \"kubernetes.io/projected/40858a66-ae35-4c67-850b-2ee7c1d4ace3-kube-api-access-rd6jk\") pod \"node-exporter-ft86p\" (UID: \"40858a66-ae35-4c67-850b-2ee7c1d4ace3\") " pod="openshift-monitoring/node-exporter-ft86p"
Jul 28 23:23:22.501537 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:23:22.501498    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/40858a66-ae35-4c67-850b-2ee7c1d4ace3-node-exporter-tls\") pod \"node-exporter-ft86p\" (UID: \"40858a66-ae35-4c67-850b-2ee7c1d4ace3\") " pod="openshift-monitoring/node-exporter-ft86p"
Jul 28 23:23:22.504429 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:23:22.504403    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/40858a66-ae35-4c67-850b-2ee7c1d4ace3-node-exporter-tls\") pod \"node-exporter-ft86p\" (UID: \"40858a66-ae35-4c67-850b-2ee7c1d4ace3\") " pod="openshift-monitoring/node-exporter-ft86p"
Jul 28 23:23:22.712168 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:23:22.712134    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-ft86p"
Jul 28 23:23:24.026676 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:23:24.026639    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40858a66_ae35_4c67_850b_2ee7c1d4ace3.slice/crio-30a7f85cd3bd9c8b63d350635fc4536665ba658e39f3fe7a3cbc12d7242a3d1a WatchSource:0}: Error finding container 30a7f85cd3bd9c8b63d350635fc4536665ba658e39f3fe7a3cbc12d7242a3d1a: Status 404 returned error can't find the container with id 30a7f85cd3bd9c8b63d350635fc4536665ba658e39f3fe7a3cbc12d7242a3d1a
Jul 28 23:23:24.317809 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:23:24.317713    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ft86p" event={"ID":"40858a66-ae35-4c67-850b-2ee7c1d4ace3","Type":"ContainerStarted","Data":"30a7f85cd3bd9c8b63d350635fc4536665ba658e39f3fe7a3cbc12d7242a3d1a"}
Jul 28 23:23:24.319710 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:23:24.319679    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-4p55m" event={"ID":"2e08e8ce-573f-4068-8c3b-96315b3d60ce","Type":"ContainerStarted","Data":"ca9da24525f6c6e2adfca9c38bd1dc5813537b26907828fd7b187725405126b0"}
Jul 28 23:23:24.339099 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:23:24.339047    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-4p55m" podStartSLOduration=32.935110522 podStartE2EDuration="46.339025144s" totalImagesPullingTime="13.403914622s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 23:22:38 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 23:23:24.337259027 +0000 UTC m=+81.029933575" watchObservedRunningTime="2026-07-28 23:23:24.339025144 +0000 UTC m=+81.031699684"
Jul 28 23:23:25.324913 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:23:25.324874    2572 generic.go:356] "Generic (PLEG): container finished" podID="40858a66-ae35-4c67-850b-2ee7c1d4ace3" containerID="00fab58f698649134c9e10ea8edc90fc33f6ec5b32037ae9d3f2ceba9ead6aa5" exitCode=0
Jul 28 23:23:25.325318 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:23:25.324954    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ft86p" event={"ID":"40858a66-ae35-4c67-850b-2ee7c1d4ace3","Type":"ContainerDied","Data":"00fab58f698649134c9e10ea8edc90fc33f6ec5b32037ae9d3f2ceba9ead6aa5"}
Jul 28 23:23:26.330234 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:23:26.330192    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ft86p" event={"ID":"40858a66-ae35-4c67-850b-2ee7c1d4ace3","Type":"ContainerStarted","Data":"456e422ee728c089497509e90a48304bbb56dc8a9caf2053e4a1823c77a8b6ea"}
Jul 28 23:23:26.330660 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:23:26.330268    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ft86p" event={"ID":"40858a66-ae35-4c67-850b-2ee7c1d4ace3","Type":"ContainerStarted","Data":"277b3d89ff0883c89f7977a6379758cbd29c842dc3860a853362e67b5f9d07ef"}
Jul 28 23:23:26.352305 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:23:26.352248    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-ft86p" podStartSLOduration=4.415979042 podStartE2EDuration="5.352234205s" totalImagesPullingTime="870.216477ms" totalInitContainerRuntime="66.038686ms" isStatefulPod=true podCreationTimestamp="2026-07-28 23:23:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 23:23:26.352096301 +0000 UTC m=+83.044770810" watchObservedRunningTime="2026-07-28 23:23:26.352234205 +0000 UTC m=+83.044908713"
Jul 28 23:23:48.294027 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:23:48.293995    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-2svls"
Jul 28 23:26:40.489987 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:26:40.489912    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6rmhf"]
Jul 28 23:26:40.491820 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:26:40.491803    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6rmhf"
Jul 28 23:26:40.494190 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:26:40.494167    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 28 23:26:40.495212 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:26:40.495193    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 28 23:26:40.495275 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:26:40.495194    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-jg9h2\""
Jul 28 23:26:40.502005 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:26:40.501981    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6rmhf"]
Jul 28 23:26:40.543892 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:26:40.543849    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pjm2c\" (UniqueName: \"kubernetes.io/projected/2482cb9c-2e06-4546-b57f-8fe6a1569339-kube-api-access-pjm2c\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6rmhf\" (UID: \"2482cb9c-2e06-4546-b57f-8fe6a1569339\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6rmhf"
Jul 28 23:26:40.543892 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:26:40.543896    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2482cb9c-2e06-4546-b57f-8fe6a1569339-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6rmhf\" (UID: \"2482cb9c-2e06-4546-b57f-8fe6a1569339\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6rmhf"
Jul 28 23:26:40.544096 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:26:40.543930    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2482cb9c-2e06-4546-b57f-8fe6a1569339-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6rmhf\" (UID: \"2482cb9c-2e06-4546-b57f-8fe6a1569339\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6rmhf"
Jul 28 23:26:40.644286 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:26:40.644251    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pjm2c\" (UniqueName: \"kubernetes.io/projected/2482cb9c-2e06-4546-b57f-8fe6a1569339-kube-api-access-pjm2c\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6rmhf\" (UID: \"2482cb9c-2e06-4546-b57f-8fe6a1569339\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6rmhf"
Jul 28 23:26:40.644476 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:26:40.644293    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2482cb9c-2e06-4546-b57f-8fe6a1569339-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6rmhf\" (UID: \"2482cb9c-2e06-4546-b57f-8fe6a1569339\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6rmhf"
Jul 28 23:26:40.644476 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:26:40.644321    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2482cb9c-2e06-4546-b57f-8fe6a1569339-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6rmhf\" (UID: \"2482cb9c-2e06-4546-b57f-8fe6a1569339\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6rmhf"
Jul 28 23:26:40.644685 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:26:40.644669    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2482cb9c-2e06-4546-b57f-8fe6a1569339-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6rmhf\" (UID: \"2482cb9c-2e06-4546-b57f-8fe6a1569339\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6rmhf"
Jul 28 23:26:40.644767 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:26:40.644748    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2482cb9c-2e06-4546-b57f-8fe6a1569339-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6rmhf\" (UID: \"2482cb9c-2e06-4546-b57f-8fe6a1569339\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6rmhf"
Jul 28 23:26:40.658055 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:26:40.658025    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pjm2c\" (UniqueName: \"kubernetes.io/projected/2482cb9c-2e06-4546-b57f-8fe6a1569339-kube-api-access-pjm2c\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6rmhf\" (UID: \"2482cb9c-2e06-4546-b57f-8fe6a1569339\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6rmhf"
Jul 28 23:26:40.801323 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:26:40.801238    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6rmhf"
Jul 28 23:26:40.924153 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:26:40.924128    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6rmhf"]
Jul 28 23:26:40.926298 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:26:40.926272    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2482cb9c_2e06_4546_b57f_8fe6a1569339.slice/crio-4afda85c1d909fafa01daf36a986981188d7631209943a542b8b7811a862ba5f WatchSource:0}: Error finding container 4afda85c1d909fafa01daf36a986981188d7631209943a542b8b7811a862ba5f: Status 404 returned error can't find the container with id 4afda85c1d909fafa01daf36a986981188d7631209943a542b8b7811a862ba5f
Jul 28 23:26:40.946264 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:26:40.946236    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6rmhf" event={"ID":"2482cb9c-2e06-4546-b57f-8fe6a1569339","Type":"ContainerStarted","Data":"4afda85c1d909fafa01daf36a986981188d7631209943a542b8b7811a862ba5f"}
Jul 28 23:26:41.568305 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:26:41.568263    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-pbn64"]
Jul 28 23:26:41.570330 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:26:41.570305    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-pbn64"
Jul 28 23:26:41.572853 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:26:41.572829    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 28 23:26:41.572987 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:26:41.572830    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-b894l\""
Jul 28 23:26:41.574099 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:26:41.574081    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 28 23:26:41.580133 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:26:41.580102    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-pbn64"]
Jul 28 23:26:41.650339 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:26:41.650297    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-225m6\" (UniqueName: \"kubernetes.io/projected/095dfd46-3c40-4d1a-9b65-a87ba60a0291-kube-api-access-225m6\") pod \"cert-manager-cainjector-559cdd8c87-pbn64\" (UID: \"095dfd46-3c40-4d1a-9b65-a87ba60a0291\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-pbn64"
Jul 28 23:26:41.650528 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:26:41.650346    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/095dfd46-3c40-4d1a-9b65-a87ba60a0291-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-pbn64\" (UID: \"095dfd46-3c40-4d1a-9b65-a87ba60a0291\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-pbn64"
Jul 28 23:26:41.751215 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:26:41.751182    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-225m6\" (UniqueName: \"kubernetes.io/projected/095dfd46-3c40-4d1a-9b65-a87ba60a0291-kube-api-access-225m6\") pod \"cert-manager-cainjector-559cdd8c87-pbn64\" (UID: \"095dfd46-3c40-4d1a-9b65-a87ba60a0291\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-pbn64"
Jul 28 23:26:41.751369 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:26:41.751232    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/095dfd46-3c40-4d1a-9b65-a87ba60a0291-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-pbn64\" (UID: \"095dfd46-3c40-4d1a-9b65-a87ba60a0291\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-pbn64"
Jul 28 23:26:41.759483 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:26:41.759457    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/095dfd46-3c40-4d1a-9b65-a87ba60a0291-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-pbn64\" (UID: \"095dfd46-3c40-4d1a-9b65-a87ba60a0291\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-pbn64"
Jul 28 23:26:41.759666 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:26:41.759591    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-225m6\" (UniqueName: \"kubernetes.io/projected/095dfd46-3c40-4d1a-9b65-a87ba60a0291-kube-api-access-225m6\") pod \"cert-manager-cainjector-559cdd8c87-pbn64\" (UID: \"095dfd46-3c40-4d1a-9b65-a87ba60a0291\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-pbn64"
Jul 28 23:26:41.882356 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:26:41.882270    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-pbn64"
Jul 28 23:26:42.020827 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:26:42.020789    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-pbn64"]
Jul 28 23:26:42.024149 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:26:42.024117    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod095dfd46_3c40_4d1a_9b65_a87ba60a0291.slice/crio-7ca493286e5a7dbd009c5fbcd58c0dde4a935c646f6d566af869647c1b4be07e WatchSource:0}: Error finding container 7ca493286e5a7dbd009c5fbcd58c0dde4a935c646f6d566af869647c1b4be07e: Status 404 returned error can't find the container with id 7ca493286e5a7dbd009c5fbcd58c0dde4a935c646f6d566af869647c1b4be07e
Jul 28 23:26:42.957330 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:26:42.957284    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-pbn64" event={"ID":"095dfd46-3c40-4d1a-9b65-a87ba60a0291","Type":"ContainerStarted","Data":"7ca493286e5a7dbd009c5fbcd58c0dde4a935c646f6d566af869647c1b4be07e"}
Jul 28 23:26:45.972187 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:26:45.972143    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-pbn64" event={"ID":"095dfd46-3c40-4d1a-9b65-a87ba60a0291","Type":"ContainerStarted","Data":"7bdc420da3cca1a5bf79d6120bdffbb3c1027cf9da3fd7e74229487bdcc52e25"}
Jul 28 23:26:45.987292 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:26:45.987238    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-pbn64" podStartSLOduration=1.550236462 podStartE2EDuration="4.98721925s" totalImagesPullingTime="3.436982788s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 23:26:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 23:26:45.986476223 +0000 UTC m=+282.679150748" watchObservedRunningTime="2026-07-28 23:26:45.98721925 +0000 UTC m=+282.679893753"
Jul 28 23:26:47.981352 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:26:47.981320    2572 generic.go:356] "Generic (PLEG): container finished" podID="2482cb9c-2e06-4546-b57f-8fe6a1569339" containerID="4310ed90cedb1602d8ddeb86c5da71d2fd0e1608bf081b69f212fffc56cb17c3" exitCode=0
Jul 28 23:26:47.981735 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:26:47.981384    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6rmhf" event={"ID":"2482cb9c-2e06-4546-b57f-8fe6a1569339","Type":"ContainerDied","Data":"4310ed90cedb1602d8ddeb86c5da71d2fd0e1608bf081b69f212fffc56cb17c3"}
Jul 28 23:26:50.992640 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:26:50.992584    2572 generic.go:356] "Generic (PLEG): container finished" podID="2482cb9c-2e06-4546-b57f-8fe6a1569339" containerID="334748f986d5c5cd3cdeec90a8f3bcef5d2de9e3ce48dc0e231acc25868df7ca" exitCode=0
Jul 28 23:26:50.992640 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:26:50.992641    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6rmhf" event={"ID":"2482cb9c-2e06-4546-b57f-8fe6a1569339","Type":"ContainerDied","Data":"334748f986d5c5cd3cdeec90a8f3bcef5d2de9e3ce48dc0e231acc25868df7ca"}
Jul 28 23:26:58.019083 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:26:58.019039    2572 generic.go:356] "Generic (PLEG): container finished" podID="2482cb9c-2e06-4546-b57f-8fe6a1569339" containerID="bd31237ef3c93515deb69a237a926eba8368dfb49db16db16bd3d4de9258b28a" exitCode=0
Jul 28 23:26:58.019540 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:26:58.019129    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6rmhf" event={"ID":"2482cb9c-2e06-4546-b57f-8fe6a1569339","Type":"ContainerDied","Data":"bd31237ef3c93515deb69a237a926eba8368dfb49db16db16bd3d4de9258b28a"}
Jul 28 23:26:59.144358 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:26:59.144333    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6rmhf"
Jul 28 23:26:59.174036 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:26:59.174003    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2482cb9c-2e06-4546-b57f-8fe6a1569339-bundle\") pod \"2482cb9c-2e06-4546-b57f-8fe6a1569339\" (UID: \"2482cb9c-2e06-4546-b57f-8fe6a1569339\") "
Jul 28 23:26:59.174174 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:26:59.174048    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pjm2c\" (UniqueName: \"kubernetes.io/projected/2482cb9c-2e06-4546-b57f-8fe6a1569339-kube-api-access-pjm2c\") pod \"2482cb9c-2e06-4546-b57f-8fe6a1569339\" (UID: \"2482cb9c-2e06-4546-b57f-8fe6a1569339\") "
Jul 28 23:26:59.174174 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:26:59.174079    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2482cb9c-2e06-4546-b57f-8fe6a1569339-util\") pod \"2482cb9c-2e06-4546-b57f-8fe6a1569339\" (UID: \"2482cb9c-2e06-4546-b57f-8fe6a1569339\") "
Jul 28 23:26:59.174476 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:26:59.174453    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2482cb9c-2e06-4546-b57f-8fe6a1569339-bundle" (OuterVolumeSpecName: "bundle") pod "2482cb9c-2e06-4546-b57f-8fe6a1569339" (UID: "2482cb9c-2e06-4546-b57f-8fe6a1569339"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 23:26:59.176298 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:26:59.176276    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2482cb9c-2e06-4546-b57f-8fe6a1569339-kube-api-access-pjm2c" (OuterVolumeSpecName: "kube-api-access-pjm2c") pod "2482cb9c-2e06-4546-b57f-8fe6a1569339" (UID: "2482cb9c-2e06-4546-b57f-8fe6a1569339"). InnerVolumeSpecName "kube-api-access-pjm2c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 23:26:59.179148 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:26:59.179126    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2482cb9c-2e06-4546-b57f-8fe6a1569339-util" (OuterVolumeSpecName: "util") pod "2482cb9c-2e06-4546-b57f-8fe6a1569339" (UID: "2482cb9c-2e06-4546-b57f-8fe6a1569339"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 23:26:59.274887 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:26:59.274795    2572 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2482cb9c-2e06-4546-b57f-8fe6a1569339-bundle\") on node \"ip-10-0-131-112.ec2.internal\" DevicePath \"\""
Jul 28 23:26:59.274887 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:26:59.274836    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pjm2c\" (UniqueName: \"kubernetes.io/projected/2482cb9c-2e06-4546-b57f-8fe6a1569339-kube-api-access-pjm2c\") on node \"ip-10-0-131-112.ec2.internal\" DevicePath \"\""
Jul 28 23:26:59.274887 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:26:59.274847    2572 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2482cb9c-2e06-4546-b57f-8fe6a1569339-util\") on node \"ip-10-0-131-112.ec2.internal\" DevicePath \"\""
Jul 28 23:27:00.027299 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:27:00.027264    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6rmhf" event={"ID":"2482cb9c-2e06-4546-b57f-8fe6a1569339","Type":"ContainerDied","Data":"4afda85c1d909fafa01daf36a986981188d7631209943a542b8b7811a862ba5f"}
Jul 28 23:27:00.027460 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:27:00.027312    2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4afda85c1d909fafa01daf36a986981188d7631209943a542b8b7811a862ba5f"
Jul 28 23:27:00.027460 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:27:00.027279    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6rmhf"
Jul 28 23:27:03.686185 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:27:03.686145    2572 kubelet.go:1648] "Image garbage collection succeeded"
Jul 28 23:27:20.267158 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:27:20.267122    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhknvv"]
Jul 28 23:27:20.269594 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:27:20.267354    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2482cb9c-2e06-4546-b57f-8fe6a1569339" containerName="util"
Jul 28 23:27:20.269594 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:27:20.267365    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="2482cb9c-2e06-4546-b57f-8fe6a1569339" containerName="util"
Jul 28 23:27:20.269594 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:27:20.267373    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2482cb9c-2e06-4546-b57f-8fe6a1569339" containerName="extract"
Jul 28 23:27:20.269594 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:27:20.267379    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="2482cb9c-2e06-4546-b57f-8fe6a1569339" containerName="extract"
Jul 28 23:27:20.269594 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:27:20.267407    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2482cb9c-2e06-4546-b57f-8fe6a1569339" containerName="pull"
Jul 28 23:27:20.269594 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:27:20.267412    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="2482cb9c-2e06-4546-b57f-8fe6a1569339" containerName="pull"
Jul 28 23:27:20.269594 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:27:20.267462    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="2482cb9c-2e06-4546-b57f-8fe6a1569339" containerName="extract"
Jul 28 23:27:20.269909 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:27:20.269645    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhknvv"
Jul 28 23:27:20.271922 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:27:20.271897    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 28 23:27:20.272018 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:27:20.271998    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 28 23:27:20.272195 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:27:20.272176    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-jg9h2\""
Jul 28 23:27:20.276979 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:27:20.276957    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhknvv"]
Jul 28 23:27:20.314382 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:27:20.314356    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8f279bca-2523-43f2-9e4b-f3ad4b8dec95-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhknvv\" (UID: \"8f279bca-2523-43f2-9e4b-f3ad4b8dec95\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhknvv"
Jul 28 23:27:20.314521 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:27:20.314395    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8f279bca-2523-43f2-9e4b-f3ad4b8dec95-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhknvv\" (UID: \"8f279bca-2523-43f2-9e4b-f3ad4b8dec95\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhknvv"
Jul 28 23:27:20.314521 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:27:20.314420    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rp8bq\" (UniqueName: \"kubernetes.io/projected/8f279bca-2523-43f2-9e4b-f3ad4b8dec95-kube-api-access-rp8bq\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhknvv\" (UID: \"8f279bca-2523-43f2-9e4b-f3ad4b8dec95\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhknvv"
Jul 28 23:27:20.415720 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:27:20.415674    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8f279bca-2523-43f2-9e4b-f3ad4b8dec95-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhknvv\" (UID: \"8f279bca-2523-43f2-9e4b-f3ad4b8dec95\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhknvv"
Jul 28 23:27:20.415889 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:27:20.415739    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8f279bca-2523-43f2-9e4b-f3ad4b8dec95-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhknvv\" (UID: \"8f279bca-2523-43f2-9e4b-f3ad4b8dec95\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhknvv"
Jul 28 23:27:20.415889 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:27:20.415767    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rp8bq\" (UniqueName: \"kubernetes.io/projected/8f279bca-2523-43f2-9e4b-f3ad4b8dec95-kube-api-access-rp8bq\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhknvv\" (UID: \"8f279bca-2523-43f2-9e4b-f3ad4b8dec95\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhknvv"
Jul 28 23:27:20.416075 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:27:20.416054    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8f279bca-2523-43f2-9e4b-f3ad4b8dec95-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhknvv\" (UID: \"8f279bca-2523-43f2-9e4b-f3ad4b8dec95\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhknvv"
Jul 28 23:27:20.416115 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:27:20.416087    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8f279bca-2523-43f2-9e4b-f3ad4b8dec95-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhknvv\" (UID: \"8f279bca-2523-43f2-9e4b-f3ad4b8dec95\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhknvv"
Jul 28 23:27:20.423270 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:27:20.423241    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rp8bq\" (UniqueName: \"kubernetes.io/projected/8f279bca-2523-43f2-9e4b-f3ad4b8dec95-kube-api-access-rp8bq\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhknvv\" (UID: \"8f279bca-2523-43f2-9e4b-f3ad4b8dec95\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhknvv"
Jul 28 23:27:20.579551 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:27:20.579453    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhknvv"
Jul 28 23:27:20.699393 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:27:20.699362    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhknvv"]
Jul 28 23:27:20.702170 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:27:20.702145    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f279bca_2523_43f2_9e4b_f3ad4b8dec95.slice/crio-ffa77dfcd20b56018e7cf040deb141be7aaab4d7329516b21900e834efaf0764 WatchSource:0}: Error finding container ffa77dfcd20b56018e7cf040deb141be7aaab4d7329516b21900e834efaf0764: Status 404 returned error can't find the container with id ffa77dfcd20b56018e7cf040deb141be7aaab4d7329516b21900e834efaf0764
Jul 28 23:27:21.100909 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:27:21.100880    2572 generic.go:356] "Generic (PLEG): container finished" podID="8f279bca-2523-43f2-9e4b-f3ad4b8dec95" containerID="8e04a9402509843a05b1f6daf1ca44011255e4c4aeb20db30f95927d3fd0e822" exitCode=0
Jul 28 23:27:21.101068 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:27:21.100952    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhknvv" event={"ID":"8f279bca-2523-43f2-9e4b-f3ad4b8dec95","Type":"ContainerDied","Data":"8e04a9402509843a05b1f6daf1ca44011255e4c4aeb20db30f95927d3fd0e822"}
Jul 28 23:27:21.101068 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:27:21.101004    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhknvv" event={"ID":"8f279bca-2523-43f2-9e4b-f3ad4b8dec95","Type":"ContainerStarted","Data":"ffa77dfcd20b56018e7cf040deb141be7aaab4d7329516b21900e834efaf0764"}
Jul 28 23:27:21.101379 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:27:21.101365    2572 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 23:27:22.106661 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:27:22.106614    2572 generic.go:356] "Generic (PLEG): container finished" podID="8f279bca-2523-43f2-9e4b-f3ad4b8dec95" containerID="81d64305c7ee4333bcadd86eeb0f9b32d709cdf70e4a551fc87a362071e316be" exitCode=0
Jul 28 23:27:22.106661 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:27:22.106652    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhknvv" event={"ID":"8f279bca-2523-43f2-9e4b-f3ad4b8dec95","Type":"ContainerDied","Data":"81d64305c7ee4333bcadd86eeb0f9b32d709cdf70e4a551fc87a362071e316be"}
Jul 28 23:27:23.112269 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:27:23.112233    2572 generic.go:356] "Generic (PLEG): container finished" podID="8f279bca-2523-43f2-9e4b-f3ad4b8dec95" containerID="6879d3fbf20e957a3db3ed40fb7baa9c82047b14556a31b2e0fae2e3cdc364c3" exitCode=0
Jul 28 23:27:23.112658 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:27:23.112296    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhknvv" event={"ID":"8f279bca-2523-43f2-9e4b-f3ad4b8dec95","Type":"ContainerDied","Data":"6879d3fbf20e957a3db3ed40fb7baa9c82047b14556a31b2e0fae2e3cdc364c3"}
Jul 28 23:27:23.490295 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:27:23.490262    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-7858699d7d-p25qn"]
Jul 28 23:27:23.492113 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:27:23.492097    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7858699d7d-p25qn"
Jul 28 23:27:23.494545 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:27:23.494525    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 28 23:27:23.494723 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:27:23.494701    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 28 23:27:23.495460 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:27:23.495429    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 28 23:27:23.495554 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:27:23.495483    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-gtszb\""
Jul 28 23:27:23.495554 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:27:23.495529    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 28 23:27:23.495554 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:27:23.495532    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 28 23:27:23.505060 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:27:23.504385    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7858699d7d-p25qn"]
Jul 28 23:27:23.542454 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:27:23.542428    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/bb9153ff-89de-4865-8ebe-0c91921dda7c-metrics-cert\") pod \"lws-controller-manager-7858699d7d-p25qn\" (UID: \"bb9153ff-89de-4865-8ebe-0c91921dda7c\") " pod="openshift-lws-operator/lws-controller-manager-7858699d7d-p25qn"
Jul 28 23:27:23.542566 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:27:23.542460    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bb9153ff-89de-4865-8ebe-0c91921dda7c-cert\") pod \"lws-controller-manager-7858699d7d-p25qn\" (UID: \"bb9153ff-89de-4865-8ebe-0c91921dda7c\") " pod="openshift-lws-operator/lws-controller-manager-7858699d7d-p25qn"
Jul 28 23:27:23.542566 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:27:23.542479    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/bb9153ff-89de-4865-8ebe-0c91921dda7c-manager-config\") pod \"lws-controller-manager-7858699d7d-p25qn\" (UID: \"bb9153ff-89de-4865-8ebe-0c91921dda7c\") " pod="openshift-lws-operator/lws-controller-manager-7858699d7d-p25qn"
Jul 28 23:27:23.542566 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:27:23.542505    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gxztl\" (UniqueName: \"kubernetes.io/projected/bb9153ff-89de-4865-8ebe-0c91921dda7c-kube-api-access-gxztl\") pod \"lws-controller-manager-7858699d7d-p25qn\" (UID: \"bb9153ff-89de-4865-8ebe-0c91921dda7c\") " pod="openshift-lws-operator/lws-controller-manager-7858699d7d-p25qn"
Jul 28 23:27:23.643799 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:27:23.643765    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/bb9153ff-89de-4865-8ebe-0c91921dda7c-manager-config\") pod \"lws-controller-manager-7858699d7d-p25qn\" (UID: \"bb9153ff-89de-4865-8ebe-0c91921dda7c\") " pod="openshift-lws-operator/lws-controller-manager-7858699d7d-p25qn"
Jul 28 23:27:23.643942 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:27:23.643807    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gxztl\" (UniqueName: \"kubernetes.io/projected/bb9153ff-89de-4865-8ebe-0c91921dda7c-kube-api-access-gxztl\") pod \"lws-controller-manager-7858699d7d-p25qn\" (UID: \"bb9153ff-89de-4865-8ebe-0c91921dda7c\") " pod="openshift-lws-operator/lws-controller-manager-7858699d7d-p25qn"
Jul 28 23:27:23.643942 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:27:23.643849    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/bb9153ff-89de-4865-8ebe-0c91921dda7c-metrics-cert\") pod \"lws-controller-manager-7858699d7d-p25qn\" (UID: \"bb9153ff-89de-4865-8ebe-0c91921dda7c\") " pod="openshift-lws-operator/lws-controller-manager-7858699d7d-p25qn"
Jul 28 23:27:23.643942 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:27:23.643866    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bb9153ff-89de-4865-8ebe-0c91921dda7c-cert\") pod \"lws-controller-manager-7858699d7d-p25qn\" (UID: \"bb9153ff-89de-4865-8ebe-0c91921dda7c\") " pod="openshift-lws-operator/lws-controller-manager-7858699d7d-p25qn"
Jul 28 23:27:23.644460 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:27:23.644433    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/bb9153ff-89de-4865-8ebe-0c91921dda7c-manager-config\") pod \"lws-controller-manager-7858699d7d-p25qn\" (UID: \"bb9153ff-89de-4865-8ebe-0c91921dda7c\") " pod="openshift-lws-operator/lws-controller-manager-7858699d7d-p25qn"
Jul 28 23:27:23.646291 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:27:23.646270    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/bb9153ff-89de-4865-8ebe-0c91921dda7c-metrics-cert\") pod \"lws-controller-manager-7858699d7d-p25qn\" (UID: \"bb9153ff-89de-4865-8ebe-0c91921dda7c\") " pod="openshift-lws-operator/lws-controller-manager-7858699d7d-p25qn"
Jul 28 23:27:23.646368 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:27:23.646340    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bb9153ff-89de-4865-8ebe-0c91921dda7c-cert\") pod \"lws-controller-manager-7858699d7d-p25qn\" (UID: \"bb9153ff-89de-4865-8ebe-0c91921dda7c\") " pod="openshift-lws-operator/lws-controller-manager-7858699d7d-p25qn"
Jul 28 23:27:23.652928 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:27:23.652910    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gxztl\" (UniqueName: \"kubernetes.io/projected/bb9153ff-89de-4865-8ebe-0c91921dda7c-kube-api-access-gxztl\") pod \"lws-controller-manager-7858699d7d-p25qn\" (UID: \"bb9153ff-89de-4865-8ebe-0c91921dda7c\") " pod="openshift-lws-operator/lws-controller-manager-7858699d7d-p25qn"
Jul 28 23:27:23.801157 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:27:23.801067    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7858699d7d-p25qn"
Jul 28 23:27:23.926328 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:27:23.926295    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7858699d7d-p25qn"]
Jul 28 23:27:23.929668 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:27:23.929639    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb9153ff_89de_4865_8ebe_0c91921dda7c.slice/crio-24137925b291bdb9ab039126ff460a9a2389076c2d8516a82e546e4f13f0a994 WatchSource:0}: Error finding container 24137925b291bdb9ab039126ff460a9a2389076c2d8516a82e546e4f13f0a994: Status 404 returned error can't find the container with id 24137925b291bdb9ab039126ff460a9a2389076c2d8516a82e546e4f13f0a994
Jul 28 23:27:24.116903 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:27:24.116826    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7858699d7d-p25qn" event={"ID":"bb9153ff-89de-4865-8ebe-0c91921dda7c","Type":"ContainerStarted","Data":"24137925b291bdb9ab039126ff460a9a2389076c2d8516a82e546e4f13f0a994"}
Jul 28 23:27:24.231579 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:27:24.231555    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhknvv"
Jul 28 23:27:24.347586 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:27:24.347556    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rp8bq\" (UniqueName: \"kubernetes.io/projected/8f279bca-2523-43f2-9e4b-f3ad4b8dec95-kube-api-access-rp8bq\") pod \"8f279bca-2523-43f2-9e4b-f3ad4b8dec95\" (UID: \"8f279bca-2523-43f2-9e4b-f3ad4b8dec95\") "
Jul 28 23:27:24.347768 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:27:24.347610    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8f279bca-2523-43f2-9e4b-f3ad4b8dec95-bundle\") pod \"8f279bca-2523-43f2-9e4b-f3ad4b8dec95\" (UID: \"8f279bca-2523-43f2-9e4b-f3ad4b8dec95\") "
Jul 28 23:27:24.347768 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:27:24.347659    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8f279bca-2523-43f2-9e4b-f3ad4b8dec95-util\") pod \"8f279bca-2523-43f2-9e4b-f3ad4b8dec95\" (UID: \"8f279bca-2523-43f2-9e4b-f3ad4b8dec95\") "
Jul 28 23:27:24.348342 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:27:24.348315    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8f279bca-2523-43f2-9e4b-f3ad4b8dec95-bundle" (OuterVolumeSpecName: "bundle") pod "8f279bca-2523-43f2-9e4b-f3ad4b8dec95" (UID: "8f279bca-2523-43f2-9e4b-f3ad4b8dec95"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 23:27:24.349671 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:27:24.349643    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8f279bca-2523-43f2-9e4b-f3ad4b8dec95-kube-api-access-rp8bq" (OuterVolumeSpecName: "kube-api-access-rp8bq") pod "8f279bca-2523-43f2-9e4b-f3ad4b8dec95" (UID: "8f279bca-2523-43f2-9e4b-f3ad4b8dec95"). InnerVolumeSpecName "kube-api-access-rp8bq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 23:27:24.353923 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:27:24.353882    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8f279bca-2523-43f2-9e4b-f3ad4b8dec95-util" (OuterVolumeSpecName: "util") pod "8f279bca-2523-43f2-9e4b-f3ad4b8dec95" (UID: "8f279bca-2523-43f2-9e4b-f3ad4b8dec95"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 23:27:24.448241 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:27:24.448165    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rp8bq\" (UniqueName: \"kubernetes.io/projected/8f279bca-2523-43f2-9e4b-f3ad4b8dec95-kube-api-access-rp8bq\") on node \"ip-10-0-131-112.ec2.internal\" DevicePath \"\""
Jul 28 23:27:24.448241 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:27:24.448191    2572 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8f279bca-2523-43f2-9e4b-f3ad4b8dec95-bundle\") on node \"ip-10-0-131-112.ec2.internal\" DevicePath \"\""
Jul 28 23:27:24.448241 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:27:24.448200    2572 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8f279bca-2523-43f2-9e4b-f3ad4b8dec95-util\") on node \"ip-10-0-131-112.ec2.internal\" DevicePath \"\""
Jul 28 23:27:25.123459 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:27:25.123432    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhknvv" event={"ID":"8f279bca-2523-43f2-9e4b-f3ad4b8dec95","Type":"ContainerDied","Data":"ffa77dfcd20b56018e7cf040deb141be7aaab4d7329516b21900e834efaf0764"}
Jul 28 23:27:25.123951 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:27:25.123479    2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ffa77dfcd20b56018e7cf040deb141be7aaab4d7329516b21900e834efaf0764"
Jul 28 23:27:25.123951 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:27:25.123509    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhknvv"
Jul 28 23:27:26.128537 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:27:26.128501    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7858699d7d-p25qn" event={"ID":"bb9153ff-89de-4865-8ebe-0c91921dda7c","Type":"ContainerStarted","Data":"63e61e5389f40097c2976e53fa97d480d00324d36ab29dfed365d240b3343030"}
Jul 28 23:27:26.128912 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:27:26.128640    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-7858699d7d-p25qn"
Jul 28 23:27:26.143518 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:27:26.143471    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-7858699d7d-p25qn" podStartSLOduration=1.553762627 podStartE2EDuration="3.143459076s" totalImagesPullingTime="1.589696449s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 23:27:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 23:27:26.14235704 +0000 UTC m=+322.835031548" watchObservedRunningTime="2026-07-28 23:27:26.143459076 +0000 UTC m=+322.836133584"
Jul 28 23:27:32.185834 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:27:32.185799    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-dc9c5fb48-rldzn"]
Jul 28 23:27:32.186298 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:27:32.186057    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8f279bca-2523-43f2-9e4b-f3ad4b8dec95" containerName="extract"
Jul 28 23:27:32.186298 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:27:32.186069    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="8f279bca-2523-43f2-9e4b-f3ad4b8dec95" containerName="extract"
Jul 28 23:27:32.186298 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:27:32.186080    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8f279bca-2523-43f2-9e4b-f3ad4b8dec95" containerName="util"
Jul 28 23:27:32.186298 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:27:32.186086    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="8f279bca-2523-43f2-9e4b-f3ad4b8dec95" containerName="util"
Jul 28 23:27:32.186298 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:27:32.186093    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8f279bca-2523-43f2-9e4b-f3ad4b8dec95" containerName="pull"
Jul 28 23:27:32.186298 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:27:32.186100    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="8f279bca-2523-43f2-9e4b-f3ad4b8dec95" containerName="pull"
Jul 28 23:27:32.186298 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:27:32.186153    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="8f279bca-2523-43f2-9e4b-f3ad4b8dec95" containerName="extract"
Jul 28 23:27:32.187995 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:27:32.187979    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-dc9c5fb48-rldzn"
Jul 28 23:27:32.190290 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:27:32.190271    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 28 23:27:32.190402 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:27:32.190304    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 28 23:27:32.190457 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:27:32.190411    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 28 23:27:32.190457 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:27:32.190420    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-7fb7f\""
Jul 28 23:27:32.190539 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:27:32.190483    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 28 23:27:32.204178 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:27:32.204155    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-dc9c5fb48-rldzn"]
Jul 28 23:27:32.304505 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:27:32.304469    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/5957eb3d-3eb0-4d40-86d9-92c0e31b9021-apiservice-cert\") pod \"opendatahub-operator-controller-manager-dc9c5fb48-rldzn\" (UID: \"5957eb3d-3eb0-4d40-86d9-92c0e31b9021\") " pod="opendatahub/opendatahub-operator-controller-manager-dc9c5fb48-rldzn"
Jul 28 23:27:32.304505 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:27:32.304507    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/5957eb3d-3eb0-4d40-86d9-92c0e31b9021-manifests\") pod \"opendatahub-operator-controller-manager-dc9c5fb48-rldzn\" (UID: \"5957eb3d-3eb0-4d40-86d9-92c0e31b9021\") " pod="opendatahub/opendatahub-operator-controller-manager-dc9c5fb48-rldzn"
Jul 28 23:27:32.304746 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:27:32.304528    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5957eb3d-3eb0-4d40-86d9-92c0e31b9021-webhook-cert\") pod \"opendatahub-operator-controller-manager-dc9c5fb48-rldzn\" (UID: \"5957eb3d-3eb0-4d40-86d9-92c0e31b9021\") " pod="opendatahub/opendatahub-operator-controller-manager-dc9c5fb48-rldzn"
Jul 28 23:27:32.304746 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:27:32.304619    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lvsxh\" (UniqueName: \"kubernetes.io/projected/5957eb3d-3eb0-4d40-86d9-92c0e31b9021-kube-api-access-lvsxh\") pod \"opendatahub-operator-controller-manager-dc9c5fb48-rldzn\" (UID: \"5957eb3d-3eb0-4d40-86d9-92c0e31b9021\") " pod="opendatahub/opendatahub-operator-controller-manager-dc9c5fb48-rldzn"
Jul 28 23:27:32.304746 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:27:32.304668    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5957eb3d-3eb0-4d40-86d9-92c0e31b9021-tmp\") pod \"opendatahub-operator-controller-manager-dc9c5fb48-rldzn\" (UID: \"5957eb3d-3eb0-4d40-86d9-92c0e31b9021\") " pod="opendatahub/opendatahub-operator-controller-manager-dc9c5fb48-rldzn"
Jul 28 23:27:32.405970 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:27:32.405933    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lvsxh\" (UniqueName: \"kubernetes.io/projected/5957eb3d-3eb0-4d40-86d9-92c0e31b9021-kube-api-access-lvsxh\") pod \"opendatahub-operator-controller-manager-dc9c5fb48-rldzn\" (UID: \"5957eb3d-3eb0-4d40-86d9-92c0e31b9021\") " pod="opendatahub/opendatahub-operator-controller-manager-dc9c5fb48-rldzn"
Jul 28 23:27:32.405970 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:27:32.405971    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5957eb3d-3eb0-4d40-86d9-92c0e31b9021-tmp\") pod \"opendatahub-operator-controller-manager-dc9c5fb48-rldzn\" (UID: \"5957eb3d-3eb0-4d40-86d9-92c0e31b9021\") " pod="opendatahub/opendatahub-operator-controller-manager-dc9c5fb48-rldzn"
Jul 28 23:27:32.406188 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:27:32.405995    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/5957eb3d-3eb0-4d40-86d9-92c0e31b9021-apiservice-cert\") pod \"opendatahub-operator-controller-manager-dc9c5fb48-rldzn\" (UID: \"5957eb3d-3eb0-4d40-86d9-92c0e31b9021\") " pod="opendatahub/opendatahub-operator-controller-manager-dc9c5fb48-rldzn"
Jul 28 23:27:32.406188 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:27:32.406018    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/5957eb3d-3eb0-4d40-86d9-92c0e31b9021-manifests\") pod \"opendatahub-operator-controller-manager-dc9c5fb48-rldzn\" (UID: \"5957eb3d-3eb0-4d40-86d9-92c0e31b9021\") " pod="opendatahub/opendatahub-operator-controller-manager-dc9c5fb48-rldzn"
Jul 28 23:27:32.406188 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:27:32.406036    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5957eb3d-3eb0-4d40-86d9-92c0e31b9021-webhook-cert\") pod \"opendatahub-operator-controller-manager-dc9c5fb48-rldzn\" (UID: \"5957eb3d-3eb0-4d40-86d9-92c0e31b9021\") " pod="opendatahub/opendatahub-operator-controller-manager-dc9c5fb48-rldzn"
Jul 28 23:27:32.406447 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:27:32.406426    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/5957eb3d-3eb0-4d40-86d9-92c0e31b9021-manifests\") pod \"opendatahub-operator-controller-manager-dc9c5fb48-rldzn\" (UID: \"5957eb3d-3eb0-4d40-86d9-92c0e31b9021\") " pod="opendatahub/opendatahub-operator-controller-manager-dc9c5fb48-rldzn"
Jul 28 23:27:32.408460 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:27:32.408429    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5957eb3d-3eb0-4d40-86d9-92c0e31b9021-tmp\") pod \"opendatahub-operator-controller-manager-dc9c5fb48-rldzn\" (UID: \"5957eb3d-3eb0-4d40-86d9-92c0e31b9021\") " pod="opendatahub/opendatahub-operator-controller-manager-dc9c5fb48-rldzn"
Jul 28 23:27:32.408585 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:27:32.408569    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/5957eb3d-3eb0-4d40-86d9-92c0e31b9021-apiservice-cert\") pod \"opendatahub-operator-controller-manager-dc9c5fb48-rldzn\" (UID: \"5957eb3d-3eb0-4d40-86d9-92c0e31b9021\") " pod="opendatahub/opendatahub-operator-controller-manager-dc9c5fb48-rldzn"
Jul 28 23:27:32.408681 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:27:32.408662    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5957eb3d-3eb0-4d40-86d9-92c0e31b9021-webhook-cert\") pod \"opendatahub-operator-controller-manager-dc9c5fb48-rldzn\" (UID: \"5957eb3d-3eb0-4d40-86d9-92c0e31b9021\") " pod="opendatahub/opendatahub-operator-controller-manager-dc9c5fb48-rldzn"
Jul 28 23:27:32.417544 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:27:32.417511    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lvsxh\" (UniqueName: \"kubernetes.io/projected/5957eb3d-3eb0-4d40-86d9-92c0e31b9021-kube-api-access-lvsxh\") pod \"opendatahub-operator-controller-manager-dc9c5fb48-rldzn\" (UID: \"5957eb3d-3eb0-4d40-86d9-92c0e31b9021\") " pod="opendatahub/opendatahub-operator-controller-manager-dc9c5fb48-rldzn"
Jul 28 23:27:32.498460 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:27:32.498431    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-dc9c5fb48-rldzn"
Jul 28 23:27:32.630704 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:27:32.630679    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-dc9c5fb48-rldzn"]
Jul 28 23:27:32.633117 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:27:32.633092    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5957eb3d_3eb0_4d40_86d9_92c0e31b9021.slice/crio-7befdbae7253e58c48c64b15567346485aecdefc69e774c5df3d64f109c91413 WatchSource:0}: Error finding container 7befdbae7253e58c48c64b15567346485aecdefc69e774c5df3d64f109c91413: Status 404 returned error can't find the container with id 7befdbae7253e58c48c64b15567346485aecdefc69e774c5df3d64f109c91413
Jul 28 23:27:33.153177 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:27:33.153141    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-dc9c5fb48-rldzn" event={"ID":"5957eb3d-3eb0-4d40-86d9-92c0e31b9021","Type":"ContainerStarted","Data":"7befdbae7253e58c48c64b15567346485aecdefc69e774c5df3d64f109c91413"}
Jul 28 23:27:36.167100 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:27:36.167061    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-dc9c5fb48-rldzn" event={"ID":"5957eb3d-3eb0-4d40-86d9-92c0e31b9021","Type":"ContainerStarted","Data":"90350755ad9a562f703fc3705ba59baaef8deb7393d1fd295d080d9d5de5b08f"}
Jul 28 23:27:37.134228 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:27:37.134142    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-7858699d7d-p25qn"
Jul 28 23:27:37.172762 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:27:37.172723    2572 generic.go:356] "Generic (PLEG): container finished" podID="5957eb3d-3eb0-4d40-86d9-92c0e31b9021" containerID="90350755ad9a562f703fc3705ba59baaef8deb7393d1fd295d080d9d5de5b08f" exitCode=0
Jul 28 23:27:37.173085 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:27:37.172812    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-dc9c5fb48-rldzn" event={"ID":"5957eb3d-3eb0-4d40-86d9-92c0e31b9021","Type":"ContainerDied","Data":"90350755ad9a562f703fc3705ba59baaef8deb7393d1fd295d080d9d5de5b08f"}
Jul 28 23:27:38.178228 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:27:38.178190    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-dc9c5fb48-rldzn" event={"ID":"5957eb3d-3eb0-4d40-86d9-92c0e31b9021","Type":"ContainerStarted","Data":"1aea382cc9e60bb769ef171b3c57648b7d67e94b124d1f045136d8d24f013b25"}
Jul 28 23:27:38.178598 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:27:38.178419    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-dc9c5fb48-rldzn"
Jul 28 23:27:38.199693 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:27:38.199646    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-dc9c5fb48-rldzn" podStartSLOduration=1.867708955 podStartE2EDuration="6.199617709s" totalImagesPullingTime="3.328882865s" totalInitContainerRuntime="1.003025889s" isStatefulPod=false podCreationTimestamp="2026-07-28 23:27:32 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 23:27:38.197586177 +0000 UTC m=+334.890260686" watchObservedRunningTime="2026-07-28 23:27:38.199617709 +0000 UTC m=+334.892292217"
Jul 28 23:27:49.183951 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:27:49.183918    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-dc9c5fb48-rldzn"
Jul 28 23:27:58.755827 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:27:58.755792    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-sf2rk"]
Jul 28 23:27:58.758852 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:27:58.758832    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-sf2rk"
Jul 28 23:27:58.761444 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:27:58.761420    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 28 23:27:58.761564 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:27:58.761487    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 28 23:27:58.761564 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:27:58.761493    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 28 23:27:58.761726 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:27:58.761675    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 28 23:27:58.761792 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:27:58.761702    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-5qfmt\""
Jul 28 23:27:58.761792 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:27:58.761747    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 28 23:27:58.761792 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:27:58.761729    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 28 23:27:58.771772 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:27:58.771749    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-sf2rk"]
Jul 28 23:27:58.912939 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:27:58.912902    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2cbc6725-3392-4245-b66f-cefbadc7b5b4-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-sf2rk\" (UID: \"2cbc6725-3392-4245-b66f-cefbadc7b5b4\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-sf2rk"
Jul 28 23:27:58.912939 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:27:58.912942    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/2cbc6725-3392-4245-b66f-cefbadc7b5b4-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-sf2rk\" (UID: \"2cbc6725-3392-4245-b66f-cefbadc7b5b4\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-sf2rk"
Jul 28 23:27:58.913180 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:27:58.912969    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/2cbc6725-3392-4245-b66f-cefbadc7b5b4-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-sf2rk\" (UID: \"2cbc6725-3392-4245-b66f-cefbadc7b5b4\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-sf2rk"
Jul 28 23:27:58.913180 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:27:58.912988    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2cbc6725-3392-4245-b66f-cefbadc7b5b4-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-sf2rk\" (UID: \"2cbc6725-3392-4245-b66f-cefbadc7b5b4\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-sf2rk"
Jul 28 23:27:58.913180 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:27:58.913014    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/2cbc6725-3392-4245-b66f-cefbadc7b5b4-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-sf2rk\" (UID: \"2cbc6725-3392-4245-b66f-cefbadc7b5b4\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-sf2rk"
Jul 28 23:27:58.913180 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:27:58.913038    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wx5x5\" (UniqueName: \"kubernetes.io/projected/2cbc6725-3392-4245-b66f-cefbadc7b5b4-kube-api-access-wx5x5\") pod \"istiod-openshift-gateway-5677ccb49b-sf2rk\" (UID: \"2cbc6725-3392-4245-b66f-cefbadc7b5b4\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-sf2rk"
Jul 28 23:27:58.913180 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:27:58.913062    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/2cbc6725-3392-4245-b66f-cefbadc7b5b4-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-sf2rk\" (UID: \"2cbc6725-3392-4245-b66f-cefbadc7b5b4\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-sf2rk"
Jul 28 23:27:59.014205 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:27:59.014114    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/2cbc6725-3392-4245-b66f-cefbadc7b5b4-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-sf2rk\" (UID: \"2cbc6725-3392-4245-b66f-cefbadc7b5b4\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-sf2rk"
Jul 28 23:27:59.014205 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:27:59.014163    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wx5x5\" (UniqueName: \"kubernetes.io/projected/2cbc6725-3392-4245-b66f-cefbadc7b5b4-kube-api-access-wx5x5\") pod \"istiod-openshift-gateway-5677ccb49b-sf2rk\" (UID: \"2cbc6725-3392-4245-b66f-cefbadc7b5b4\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-sf2rk"
Jul 28 23:27:59.014205 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:27:59.014181    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/2cbc6725-3392-4245-b66f-cefbadc7b5b4-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-sf2rk\" (UID: \"2cbc6725-3392-4245-b66f-cefbadc7b5b4\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-sf2rk"
Jul 28 23:27:59.014205 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:27:59.014207    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2cbc6725-3392-4245-b66f-cefbadc7b5b4-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-sf2rk\" (UID: \"2cbc6725-3392-4245-b66f-cefbadc7b5b4\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-sf2rk"
Jul 28 23:27:59.014525 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:27:59.014230    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/2cbc6725-3392-4245-b66f-cefbadc7b5b4-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-sf2rk\" (UID: \"2cbc6725-3392-4245-b66f-cefbadc7b5b4\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-sf2rk"
Jul 28 23:27:59.014525 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:27:59.014273    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/2cbc6725-3392-4245-b66f-cefbadc7b5b4-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-sf2rk\" (UID: \"2cbc6725-3392-4245-b66f-cefbadc7b5b4\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-sf2rk"
Jul 28 23:27:59.014525 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:27:59.014291    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2cbc6725-3392-4245-b66f-cefbadc7b5b4-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-sf2rk\" (UID: \"2cbc6725-3392-4245-b66f-cefbadc7b5b4\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-sf2rk"
Jul 28 23:27:59.014925 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:27:59.014892    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/2cbc6725-3392-4245-b66f-cefbadc7b5b4-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-sf2rk\" (UID: \"2cbc6725-3392-4245-b66f-cefbadc7b5b4\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-sf2rk"
Jul 28 23:27:59.016543 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:27:59.016518    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/2cbc6725-3392-4245-b66f-cefbadc7b5b4-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-sf2rk\" (UID: \"2cbc6725-3392-4245-b66f-cefbadc7b5b4\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-sf2rk"
Jul 28 23:27:59.016729 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:27:59.016714    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/2cbc6725-3392-4245-b66f-cefbadc7b5b4-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-sf2rk\" (UID: \"2cbc6725-3392-4245-b66f-cefbadc7b5b4\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-sf2rk"
Jul 28 23:27:59.016830 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:27:59.016805    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2cbc6725-3392-4245-b66f-cefbadc7b5b4-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-sf2rk\" (UID: \"2cbc6725-3392-4245-b66f-cefbadc7b5b4\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-sf2rk"
Jul 28 23:27:59.016900 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:27:59.016883    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/2cbc6725-3392-4245-b66f-cefbadc7b5b4-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-sf2rk\" (UID: \"2cbc6725-3392-4245-b66f-cefbadc7b5b4\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-sf2rk"
Jul 28 23:27:59.031004 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:27:59.030978    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2cbc6725-3392-4245-b66f-cefbadc7b5b4-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-sf2rk\" (UID: \"2cbc6725-3392-4245-b66f-cefbadc7b5b4\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-sf2rk"
Jul 28 23:27:59.031490 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:27:59.031473    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wx5x5\" (UniqueName: \"kubernetes.io/projected/2cbc6725-3392-4245-b66f-cefbadc7b5b4-kube-api-access-wx5x5\") pod \"istiod-openshift-gateway-5677ccb49b-sf2rk\" (UID: \"2cbc6725-3392-4245-b66f-cefbadc7b5b4\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-sf2rk"
Jul 28 23:27:59.068455 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:27:59.068425    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-sf2rk"
Jul 28 23:27:59.202241 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:27:59.202207    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-sf2rk"]
Jul 28 23:27:59.207020 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:27:59.206985    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2cbc6725_3392_4245_b66f_cefbadc7b5b4.slice/crio-fafc60c712b8250468a4a63181a953008e499ec315e7b64fc76ec77908ce531d WatchSource:0}: Error finding container fafc60c712b8250468a4a63181a953008e499ec315e7b64fc76ec77908ce531d: Status 404 returned error can't find the container with id fafc60c712b8250468a4a63181a953008e499ec315e7b64fc76ec77908ce531d
Jul 28 23:27:59.260327 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:27:59.260183    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-sf2rk" event={"ID":"2cbc6725-3392-4245-b66f-cefbadc7b5b4","Type":"ContainerStarted","Data":"fafc60c712b8250468a4a63181a953008e499ec315e7b64fc76ec77908ce531d"}
Jul 28 23:28:01.958477 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:28:01.958435    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 23:28:01.958731 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:28:01.958524    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 23:28:02.276611 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:28:02.276575    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-sf2rk" event={"ID":"2cbc6725-3392-4245-b66f-cefbadc7b5b4","Type":"ContainerStarted","Data":"bbf941d19a58e8e7e8e01f5b6f2cef8ce572e3f88b4af90098fa6c965a02dd04"}
Jul 28 23:28:02.276802 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:28:02.276677    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-sf2rk"
Jul 28 23:28:02.306894 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:28:02.306810    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-sf2rk" podStartSLOduration=1.558230379 podStartE2EDuration="4.306787322s" totalImagesPullingTime="2.748556943s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 23:27:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 23:28:02.305578115 +0000 UTC m=+358.998252638" watchObservedRunningTime="2026-07-28 23:28:02.306787322 +0000 UTC m=+358.999461834"
Jul 28 23:28:03.282377 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:28:03.282342    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-sf2rk"
Jul 28 23:29:48.128168 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:29:48.128131    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-9bnmh"]
Jul 28 23:29:48.131244 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:29:48.131228    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-9bnmh"
Jul 28 23:29:48.134434 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:29:48.134407    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 28 23:29:48.134720 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:29:48.134698    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 28 23:29:48.134831 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:29:48.134726    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-z6vrk\""
Jul 28 23:29:48.142803 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:29:48.142778    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-9bnmh"]
Jul 28 23:29:48.224023 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:29:48.223984    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-9bnmh"]
Jul 28 23:29:48.224269 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:29:48.224246    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[extensions-socket-volume kube-api-access-72fz7], unattached volumes=[], failed to process volumes=[extensions-socket-volume kube-api-access-72fz7]: context canceled" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-9bnmh" podUID="0ab25461-a1b1-412b-a893-4df7d2f14917"
Jul 28 23:29:48.237914 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:29:48.237878    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-72fz7\" (UniqueName: \"kubernetes.io/projected/0ab25461-a1b1-412b-a893-4df7d2f14917-kube-api-access-72fz7\") pod \"kuadrant-operator-controller-manager-55467d9f7-9bnmh\" (UID: \"0ab25461-a1b1-412b-a893-4df7d2f14917\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-9bnmh"
Jul 28 23:29:48.238042 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:29:48.237938    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0ab25461-a1b1-412b-a893-4df7d2f14917-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-9bnmh\" (UID: \"0ab25461-a1b1-412b-a893-4df7d2f14917\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-9bnmh"
Jul 28 23:29:48.269275 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:29:48.269242    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-9bnmh"]
Jul 28 23:29:48.279357 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:29:48.279330    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-9bnmh"]
Jul 28 23:29:48.317945 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:29:48.317913    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8bgf8"]
Jul 28 23:29:48.321244 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:29:48.321221    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8bgf8"
Jul 28 23:29:48.336680 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:29:48.336652    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8bgf8"]
Jul 28 23:29:48.339222 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:29:48.339197    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7jr6m\" (UniqueName: \"kubernetes.io/projected/f16fb56c-8325-42c3-a7fe-1b82890c6ae2-kube-api-access-7jr6m\") pod \"kuadrant-operator-controller-manager-8684c9ff49-8bgf8\" (UID: \"f16fb56c-8325-42c3-a7fe-1b82890c6ae2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8bgf8"
Jul 28 23:29:48.339354 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:29:48.339244    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f16fb56c-8325-42c3-a7fe-1b82890c6ae2-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-8bgf8\" (UID: \"f16fb56c-8325-42c3-a7fe-1b82890c6ae2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8bgf8"
Jul 28 23:29:48.339354 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:29:48.339313    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-72fz7\" (UniqueName: \"kubernetes.io/projected/0ab25461-a1b1-412b-a893-4df7d2f14917-kube-api-access-72fz7\") pod \"kuadrant-operator-controller-manager-55467d9f7-9bnmh\" (UID: \"0ab25461-a1b1-412b-a893-4df7d2f14917\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-9bnmh"
Jul 28 23:29:48.339465 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:29:48.339373    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0ab25461-a1b1-412b-a893-4df7d2f14917-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-9bnmh\" (UID: \"0ab25461-a1b1-412b-a893-4df7d2f14917\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-9bnmh"
Jul 28 23:29:48.339753 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:29:48.339733    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0ab25461-a1b1-412b-a893-4df7d2f14917-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-9bnmh\" (UID: \"0ab25461-a1b1-412b-a893-4df7d2f14917\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-9bnmh"
Jul 28 23:29:48.344424 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:29:48.344404    2572 projected.go:196] Error preparing data for projected volume kube-api-access-72fz7 for pod kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-9bnmh: failed to fetch token: pod "kuadrant-operator-controller-manager-55467d9f7-9bnmh" not found
Jul 28 23:29:48.344531 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:29:48.344481    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0ab25461-a1b1-412b-a893-4df7d2f14917-kube-api-access-72fz7 podName:0ab25461-a1b1-412b-a893-4df7d2f14917 nodeName:}" failed. No retries permitted until 2026-07-28 23:29:48.844459218 +0000 UTC m=+465.537133718 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-72fz7" (UniqueName: "kubernetes.io/projected/0ab25461-a1b1-412b-a893-4df7d2f14917-kube-api-access-72fz7") pod "kuadrant-operator-controller-manager-55467d9f7-9bnmh" (UID: "0ab25461-a1b1-412b-a893-4df7d2f14917") : failed to fetch token: pod "kuadrant-operator-controller-manager-55467d9f7-9bnmh" not found
Jul 28 23:29:48.440151 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:29:48.440065    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7jr6m\" (UniqueName: \"kubernetes.io/projected/f16fb56c-8325-42c3-a7fe-1b82890c6ae2-kube-api-access-7jr6m\") pod \"kuadrant-operator-controller-manager-8684c9ff49-8bgf8\" (UID: \"f16fb56c-8325-42c3-a7fe-1b82890c6ae2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8bgf8"
Jul 28 23:29:48.440151 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:29:48.440112    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f16fb56c-8325-42c3-a7fe-1b82890c6ae2-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-8bgf8\" (UID: \"f16fb56c-8325-42c3-a7fe-1b82890c6ae2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8bgf8"
Jul 28 23:29:48.440536 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:29:48.440517    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f16fb56c-8325-42c3-a7fe-1b82890c6ae2-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-8bgf8\" (UID: \"f16fb56c-8325-42c3-a7fe-1b82890c6ae2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8bgf8"
Jul 28 23:29:48.455181 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:29:48.455154    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7jr6m\" (UniqueName: \"kubernetes.io/projected/f16fb56c-8325-42c3-a7fe-1b82890c6ae2-kube-api-access-7jr6m\") pod \"kuadrant-operator-controller-manager-8684c9ff49-8bgf8\" (UID: \"f16fb56c-8325-42c3-a7fe-1b82890c6ae2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8bgf8"
Jul 28 23:29:48.631686 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:29:48.631642    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8bgf8"
Jul 28 23:29:48.679157 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:29:48.676165    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-9bnmh"
Jul 28 23:29:48.679415 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:29:48.679312    2572 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-9bnmh\" is forbidden: User \"system:node:ip-10-0-131-112.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-112.ec2.internal' and this object" podUID="0ab25461-a1b1-412b-a893-4df7d2f14917" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-9bnmh"
Jul 28 23:29:48.688576 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:29:48.688550    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-9bnmh"
Jul 28 23:29:48.690791 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:29:48.690729    2572 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-9bnmh\" is forbidden: User \"system:node:ip-10-0-131-112.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-112.ec2.internal' and this object" podUID="0ab25461-a1b1-412b-a893-4df7d2f14917" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-9bnmh"
Jul 28 23:29:48.742692 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:29:48.742661    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0ab25461-a1b1-412b-a893-4df7d2f14917-extensions-socket-volume\") pod \"0ab25461-a1b1-412b-a893-4df7d2f14917\" (UID: \"0ab25461-a1b1-412b-a893-4df7d2f14917\") "
Jul 28 23:29:48.742841 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:29:48.742813    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-72fz7\" (UniqueName: \"kubernetes.io/projected/0ab25461-a1b1-412b-a893-4df7d2f14917-kube-api-access-72fz7\") on node \"ip-10-0-131-112.ec2.internal\" DevicePath \"\""
Jul 28 23:29:48.742959 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:29:48.742932    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ab25461-a1b1-412b-a893-4df7d2f14917-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "0ab25461-a1b1-412b-a893-4df7d2f14917" (UID: "0ab25461-a1b1-412b-a893-4df7d2f14917"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 23:29:48.764609 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:29:48.764585    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8bgf8"]
Jul 28 23:29:48.767446 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:29:48.767421    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf16fb56c_8325_42c3_a7fe_1b82890c6ae2.slice/crio-65efbd4579ea21446c81839168070825dc619cf10ca47e1822dec08a76c129cc WatchSource:0}: Error finding container 65efbd4579ea21446c81839168070825dc619cf10ca47e1822dec08a76c129cc: Status 404 returned error can't find the container with id 65efbd4579ea21446c81839168070825dc619cf10ca47e1822dec08a76c129cc
Jul 28 23:29:48.843670 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:29:48.843616    2572 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0ab25461-a1b1-412b-a893-4df7d2f14917-extensions-socket-volume\") on node \"ip-10-0-131-112.ec2.internal\" DevicePath \"\""
Jul 28 23:29:49.682442 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:29:49.682409    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-9bnmh"
Jul 28 23:29:49.682939 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:29:49.682438    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8bgf8" event={"ID":"f16fb56c-8325-42c3-a7fe-1b82890c6ae2","Type":"ContainerStarted","Data":"65efbd4579ea21446c81839168070825dc619cf10ca47e1822dec08a76c129cc"}
Jul 28 23:29:49.684579 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:29:49.684537    2572 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-9bnmh\" is forbidden: User \"system:node:ip-10-0-131-112.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-112.ec2.internal' and this object" podUID="0ab25461-a1b1-412b-a893-4df7d2f14917" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-9bnmh"
Jul 28 23:29:49.693043 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:29:49.693008    2572 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-9bnmh\" is forbidden: User \"system:node:ip-10-0-131-112.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-112.ec2.internal' and this object" podUID="0ab25461-a1b1-412b-a893-4df7d2f14917" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-9bnmh"
Jul 28 23:29:49.866727 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:29:49.866694    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0ab25461-a1b1-412b-a893-4df7d2f14917" path="/var/lib/kubelet/pods/0ab25461-a1b1-412b-a893-4df7d2f14917/volumes"
Jul 28 23:29:52.697423 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:29:52.697389    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8bgf8" event={"ID":"f16fb56c-8325-42c3-a7fe-1b82890c6ae2","Type":"ContainerStarted","Data":"6096ecc6258de2df66dd990f91e5488c7ac022dffff8031772505959d8561520"}
Jul 28 23:29:52.697774 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:29:52.697559    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8bgf8"
Jul 28 23:29:52.717132 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:29:52.717080    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8bgf8" podStartSLOduration=0.875145017 podStartE2EDuration="4.717063902s" totalImagesPullingTime="3.841918885s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 23:29:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 23:29:52.713785728 +0000 UTC m=+469.406460236" watchObservedRunningTime="2026-07-28 23:29:52.717063902 +0000 UTC m=+469.409738410"
Jul 28 23:30:00.665847 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:30:00.665813    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8bgf8"]
Jul 28 23:30:00.666242 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:30:00.665999    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8bgf8" podUID="f16fb56c-8325-42c3-a7fe-1b82890c6ae2" containerName="manager" containerID="cri-o://6096ecc6258de2df66dd990f91e5488c7ac022dffff8031772505959d8561520" gracePeriod=10
Jul 28 23:30:00.667761 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:30:00.667735    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8bgf8"
Jul 28 23:30:00.919514 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:30:00.919454    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8bgf8"
Jul 28 23:30:01.040512 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:30:01.040465    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f16fb56c-8325-42c3-a7fe-1b82890c6ae2-extensions-socket-volume\") pod \"f16fb56c-8325-42c3-a7fe-1b82890c6ae2\" (UID: \"f16fb56c-8325-42c3-a7fe-1b82890c6ae2\") "
Jul 28 23:30:01.040724 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:30:01.040543    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7jr6m\" (UniqueName: \"kubernetes.io/projected/f16fb56c-8325-42c3-a7fe-1b82890c6ae2-kube-api-access-7jr6m\") pod \"f16fb56c-8325-42c3-a7fe-1b82890c6ae2\" (UID: \"f16fb56c-8325-42c3-a7fe-1b82890c6ae2\") "
Jul 28 23:30:01.041111 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:30:01.041085    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f16fb56c-8325-42c3-a7fe-1b82890c6ae2-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "f16fb56c-8325-42c3-a7fe-1b82890c6ae2" (UID: "f16fb56c-8325-42c3-a7fe-1b82890c6ae2"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 23:30:01.042832 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:30:01.042814    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f16fb56c-8325-42c3-a7fe-1b82890c6ae2-kube-api-access-7jr6m" (OuterVolumeSpecName: "kube-api-access-7jr6m") pod "f16fb56c-8325-42c3-a7fe-1b82890c6ae2" (UID: "f16fb56c-8325-42c3-a7fe-1b82890c6ae2"). InnerVolumeSpecName "kube-api-access-7jr6m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 23:30:01.141510 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:30:01.141473    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7jr6m\" (UniqueName: \"kubernetes.io/projected/f16fb56c-8325-42c3-a7fe-1b82890c6ae2-kube-api-access-7jr6m\") on node \"ip-10-0-131-112.ec2.internal\" DevicePath \"\""
Jul 28 23:30:01.141510 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:30:01.141503    2572 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f16fb56c-8325-42c3-a7fe-1b82890c6ae2-extensions-socket-volume\") on node \"ip-10-0-131-112.ec2.internal\" DevicePath \"\""
Jul 28 23:30:01.737022 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:30:01.736984    2572 generic.go:356] "Generic (PLEG): container finished" podID="f16fb56c-8325-42c3-a7fe-1b82890c6ae2" containerID="6096ecc6258de2df66dd990f91e5488c7ac022dffff8031772505959d8561520" exitCode=0
Jul 28 23:30:01.737416 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:30:01.737040    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8bgf8"
Jul 28 23:30:01.737416 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:30:01.737070    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8bgf8" event={"ID":"f16fb56c-8325-42c3-a7fe-1b82890c6ae2","Type":"ContainerDied","Data":"6096ecc6258de2df66dd990f91e5488c7ac022dffff8031772505959d8561520"}
Jul 28 23:30:01.737416 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:30:01.737141    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8bgf8" event={"ID":"f16fb56c-8325-42c3-a7fe-1b82890c6ae2","Type":"ContainerDied","Data":"65efbd4579ea21446c81839168070825dc619cf10ca47e1822dec08a76c129cc"}
Jul 28 23:30:01.737416 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:30:01.737163    2572 scope.go:117] "RemoveContainer" containerID="6096ecc6258de2df66dd990f91e5488c7ac022dffff8031772505959d8561520"
Jul 28 23:30:01.746357 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:30:01.746340    2572 scope.go:117] "RemoveContainer" containerID="6096ecc6258de2df66dd990f91e5488c7ac022dffff8031772505959d8561520"
Jul 28 23:30:01.746653 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:30:01.746615    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6096ecc6258de2df66dd990f91e5488c7ac022dffff8031772505959d8561520\": container with ID starting with 6096ecc6258de2df66dd990f91e5488c7ac022dffff8031772505959d8561520 not found: ID does not exist" containerID="6096ecc6258de2df66dd990f91e5488c7ac022dffff8031772505959d8561520"
Jul 28 23:30:01.746703 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:30:01.746665    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6096ecc6258de2df66dd990f91e5488c7ac022dffff8031772505959d8561520"} err="failed to get container status \"6096ecc6258de2df66dd990f91e5488c7ac022dffff8031772505959d8561520\": rpc error: code = NotFound desc = could not find container \"6096ecc6258de2df66dd990f91e5488c7ac022dffff8031772505959d8561520\": container with ID starting with 6096ecc6258de2df66dd990f91e5488c7ac022dffff8031772505959d8561520 not found: ID does not exist"
Jul 28 23:30:01.765431 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:30:01.765404    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8bgf8"]
Jul 28 23:30:01.770381 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:30:01.770355    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8bgf8"]
Jul 28 23:30:01.864749 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:30:01.864714    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f16fb56c-8325-42c3-a7fe-1b82890c6ae2" path="/var/lib/kubelet/pods/f16fb56c-8325-42c3-a7fe-1b82890c6ae2/volumes"
Jul 28 23:30:17.079979 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:30:17.079947    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-wvxnh"]
Jul 28 23:30:17.080491 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:30:17.080195    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f16fb56c-8325-42c3-a7fe-1b82890c6ae2" containerName="manager"
Jul 28 23:30:17.080491 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:30:17.080205    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="f16fb56c-8325-42c3-a7fe-1b82890c6ae2" containerName="manager"
Jul 28 23:30:17.080491 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:30:17.080259    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="f16fb56c-8325-42c3-a7fe-1b82890c6ae2" containerName="manager"
Jul 28 23:30:17.084453 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:30:17.084429    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-wvxnh"
Jul 28 23:30:17.086808 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:30:17.086786    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-kjvp8\""
Jul 28 23:30:17.095103 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:30:17.095081    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-wvxnh"]
Jul 28 23:30:17.170656 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:30:17.170602    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f59a2eeb-6c07-48ed-88e5-35f2bd35db3c-workload-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-wvxnh\" (UID: \"f59a2eeb-6c07-48ed-88e5-35f2bd35db3c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-wvxnh"
Jul 28 23:30:17.170656 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:30:17.170658    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gmfk6\" (UniqueName: \"kubernetes.io/projected/f59a2eeb-6c07-48ed-88e5-35f2bd35db3c-kube-api-access-gmfk6\") pod \"maas-default-gateway-openshift-default-5ccb968c66-wvxnh\" (UID: \"f59a2eeb-6c07-48ed-88e5-35f2bd35db3c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-wvxnh"
Jul 28 23:30:17.170849 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:30:17.170732    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f59a2eeb-6c07-48ed-88e5-35f2bd35db3c-istio-data\") pod \"maas-default-gateway-openshift-default-5ccb968c66-wvxnh\" (UID: \"f59a2eeb-6c07-48ed-88e5-35f2bd35db3c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-wvxnh"
Jul 28 23:30:17.170849 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:30:17.170756    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f59a2eeb-6c07-48ed-88e5-35f2bd35db3c-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5ccb968c66-wvxnh\" (UID: \"f59a2eeb-6c07-48ed-88e5-35f2bd35db3c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-wvxnh"
Jul 28 23:30:17.170849 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:30:17.170781    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f59a2eeb-6c07-48ed-88e5-35f2bd35db3c-workload-certs\") pod \"maas-default-gateway-openshift-default-5ccb968c66-wvxnh\" (UID: \"f59a2eeb-6c07-48ed-88e5-35f2bd35db3c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-wvxnh"
Jul 28 23:30:17.170849 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:30:17.170799    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f59a2eeb-6c07-48ed-88e5-35f2bd35db3c-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5ccb968c66-wvxnh\" (UID: \"f59a2eeb-6c07-48ed-88e5-35f2bd35db3c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-wvxnh"
Jul 28 23:30:17.170849 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:30:17.170817    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f59a2eeb-6c07-48ed-88e5-35f2bd35db3c-istio-token\") pod \"maas-default-gateway-openshift-default-5ccb968c66-wvxnh\" (UID: \"f59a2eeb-6c07-48ed-88e5-35f2bd35db3c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-wvxnh"
Jul 28 23:30:17.171009 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:30:17.170864    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f59a2eeb-6c07-48ed-88e5-35f2bd35db3c-credential-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-wvxnh\" (UID: \"f59a2eeb-6c07-48ed-88e5-35f2bd35db3c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-wvxnh"
Jul 28 23:30:17.171009 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:30:17.170902    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f59a2eeb-6c07-48ed-88e5-35f2bd35db3c-istio-envoy\") pod \"maas-default-gateway-openshift-default-5ccb968c66-wvxnh\" (UID: \"f59a2eeb-6c07-48ed-88e5-35f2bd35db3c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-wvxnh"
Jul 28 23:30:17.271972 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:30:17.271928    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f59a2eeb-6c07-48ed-88e5-35f2bd35db3c-istio-envoy\") pod \"maas-default-gateway-openshift-default-5ccb968c66-wvxnh\" (UID: \"f59a2eeb-6c07-48ed-88e5-35f2bd35db3c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-wvxnh"
Jul 28 23:30:17.271972 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:30:17.271975    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f59a2eeb-6c07-48ed-88e5-35f2bd35db3c-workload-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-wvxnh\" (UID: \"f59a2eeb-6c07-48ed-88e5-35f2bd35db3c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-wvxnh"
Jul 28 23:30:17.272228 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:30:17.271993    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gmfk6\" (UniqueName: \"kubernetes.io/projected/f59a2eeb-6c07-48ed-88e5-35f2bd35db3c-kube-api-access-gmfk6\") pod \"maas-default-gateway-openshift-default-5ccb968c66-wvxnh\" (UID: \"f59a2eeb-6c07-48ed-88e5-35f2bd35db3c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-wvxnh"
Jul 28 23:30:17.272228 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:30:17.272052    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f59a2eeb-6c07-48ed-88e5-35f2bd35db3c-istio-data\") pod \"maas-default-gateway-openshift-default-5ccb968c66-wvxnh\" (UID: \"f59a2eeb-6c07-48ed-88e5-35f2bd35db3c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-wvxnh"
Jul 28 23:30:17.272228 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:30:17.272077    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f59a2eeb-6c07-48ed-88e5-35f2bd35db3c-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5ccb968c66-wvxnh\" (UID: \"f59a2eeb-6c07-48ed-88e5-35f2bd35db3c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-wvxnh"
Jul 28 23:30:17.272228 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:30:17.272104    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f59a2eeb-6c07-48ed-88e5-35f2bd35db3c-workload-certs\") pod \"maas-default-gateway-openshift-default-5ccb968c66-wvxnh\" (UID: \"f59a2eeb-6c07-48ed-88e5-35f2bd35db3c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-wvxnh"
Jul 28 23:30:17.272228 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:30:17.272125    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f59a2eeb-6c07-48ed-88e5-35f2bd35db3c-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5ccb968c66-wvxnh\" (UID: \"f59a2eeb-6c07-48ed-88e5-35f2bd35db3c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-wvxnh"
Jul 28 23:30:17.272479 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:30:17.272457    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f59a2eeb-6c07-48ed-88e5-35f2bd35db3c-istio-token\") pod \"maas-default-gateway-openshift-default-5ccb968c66-wvxnh\" (UID: \"f59a2eeb-6c07-48ed-88e5-35f2bd35db3c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-wvxnh"
Jul 28 23:30:17.272533 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:30:17.272489    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f59a2eeb-6c07-48ed-88e5-35f2bd35db3c-workload-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-wvxnh\" (UID: \"f59a2eeb-6c07-48ed-88e5-35f2bd35db3c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-wvxnh"
Jul 28 23:30:17.272533 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:30:17.272504    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f59a2eeb-6c07-48ed-88e5-35f2bd35db3c-istio-data\") pod \"maas-default-gateway-openshift-default-5ccb968c66-wvxnh\" (UID: \"f59a2eeb-6c07-48ed-88e5-35f2bd35db3c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-wvxnh"
Jul 28 23:30:17.272659 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:30:17.272532    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f59a2eeb-6c07-48ed-88e5-35f2bd35db3c-credential-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-wvxnh\" (UID: \"f59a2eeb-6c07-48ed-88e5-35f2bd35db3c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-wvxnh"
Jul 28 23:30:17.272852 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:30:17.272824    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f59a2eeb-6c07-48ed-88e5-35f2bd35db3c-workload-certs\") pod \"maas-default-gateway-openshift-default-5ccb968c66-wvxnh\" (UID: \"f59a2eeb-6c07-48ed-88e5-35f2bd35db3c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-wvxnh"
Jul 28 23:30:17.272967 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:30:17.272917    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f59a2eeb-6c07-48ed-88e5-35f2bd35db3c-credential-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-wvxnh\" (UID: \"f59a2eeb-6c07-48ed-88e5-35f2bd35db3c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-wvxnh"
Jul 28 23:30:17.273223 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:30:17.273202    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f59a2eeb-6c07-48ed-88e5-35f2bd35db3c-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5ccb968c66-wvxnh\" (UID: \"f59a2eeb-6c07-48ed-88e5-35f2bd35db3c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-wvxnh"
Jul 28 23:30:17.274378 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:30:17.274350    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f59a2eeb-6c07-48ed-88e5-35f2bd35db3c-istio-envoy\") pod \"maas-default-gateway-openshift-default-5ccb968c66-wvxnh\" (UID: \"f59a2eeb-6c07-48ed-88e5-35f2bd35db3c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-wvxnh"
Jul 28 23:30:17.274579 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:30:17.274560    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f59a2eeb-6c07-48ed-88e5-35f2bd35db3c-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5ccb968c66-wvxnh\" (UID: \"f59a2eeb-6c07-48ed-88e5-35f2bd35db3c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-wvxnh"
Jul 28 23:30:17.279395 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:30:17.279371    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gmfk6\" (UniqueName: \"kubernetes.io/projected/f59a2eeb-6c07-48ed-88e5-35f2bd35db3c-kube-api-access-gmfk6\") pod \"maas-default-gateway-openshift-default-5ccb968c66-wvxnh\" (UID: \"f59a2eeb-6c07-48ed-88e5-35f2bd35db3c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-wvxnh"
Jul 28 23:30:17.279487 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:30:17.279404    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f59a2eeb-6c07-48ed-88e5-35f2bd35db3c-istio-token\") pod \"maas-default-gateway-openshift-default-5ccb968c66-wvxnh\" (UID: \"f59a2eeb-6c07-48ed-88e5-35f2bd35db3c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-wvxnh"
Jul 28 23:30:17.395728 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:30:17.395616    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-wvxnh"
Jul 28 23:30:17.527881 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:30:17.527836    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-wvxnh"]
Jul 28 23:30:17.530229 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:30:17.530199    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf59a2eeb_6c07_48ed_88e5_35f2bd35db3c.slice/crio-b0a6cb099106f33504b51d18b13110cfcd79d40f803ce5c10ba476285eac3225 WatchSource:0}: Error finding container b0a6cb099106f33504b51d18b13110cfcd79d40f803ce5c10ba476285eac3225: Status 404 returned error can't find the container with id b0a6cb099106f33504b51d18b13110cfcd79d40f803ce5c10ba476285eac3225
Jul 28 23:30:17.797750 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:30:17.797713    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-wvxnh" event={"ID":"f59a2eeb-6c07-48ed-88e5-35f2bd35db3c","Type":"ContainerStarted","Data":"b0a6cb099106f33504b51d18b13110cfcd79d40f803ce5c10ba476285eac3225"}
Jul 28 23:30:20.396118 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:30:20.396070    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 23:30:20.396411 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:30:20.396164    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 23:30:20.396411 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:30:20.396194    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 23:30:20.812307 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:30:20.812275    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-wvxnh" event={"ID":"f59a2eeb-6c07-48ed-88e5-35f2bd35db3c","Type":"ContainerStarted","Data":"a5add8ca7d582d494417e978891d77907fa4422fa6db1debbe61be33fb28224f"}
Jul 28 23:30:20.831370 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:30:20.831320    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-wvxnh" podStartSLOduration=0.968236355 podStartE2EDuration="3.831303769s" totalImagesPullingTime="2.863067414s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 23:30:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 23:30:20.829897643 +0000 UTC m=+497.522572152" watchObservedRunningTime="2026-07-28 23:30:20.831303769 +0000 UTC m=+497.523978278"
Jul 28 23:30:21.395910 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:30:21.395871    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-wvxnh"
Jul 28 23:30:21.400475 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:30:21.400448    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-wvxnh"
Jul 28 23:30:21.815858 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:30:21.815828    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-wvxnh"
Jul 28 23:30:21.816843 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:30:21.816826    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-wvxnh"
Jul 28 23:30:58.216639 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:30:58.216587    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5db89c9c54-tvfs7"]
Jul 28 23:30:58.219461 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:30:58.219437    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5db89c9c54-tvfs7"
Jul 28 23:30:58.221803 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:30:58.221771    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 28 23:30:58.221803 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:30:58.221798    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 28 23:30:58.222004 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:30:58.221798    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-j2h85\""
Jul 28 23:30:58.228548 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:30:58.228521    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5db89c9c54-tvfs7"]
Jul 28 23:30:58.301950 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:30:58.301920    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b624p\" (UniqueName: \"kubernetes.io/projected/ac2a54b5-c479-45a0-bc93-c249641f85ae-kube-api-access-b624p\") pod \"maas-controller-5db89c9c54-tvfs7\" (UID: \"ac2a54b5-c479-45a0-bc93-c249641f85ae\") " pod="opendatahub/maas-controller-5db89c9c54-tvfs7"
Jul 28 23:30:58.302129 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:30:58.301956    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ac2a54b5-c479-45a0-bc93-c249641f85ae-webhook-cert\") pod \"maas-controller-5db89c9c54-tvfs7\" (UID: \"ac2a54b5-c479-45a0-bc93-c249641f85ae\") " pod="opendatahub/maas-controller-5db89c9c54-tvfs7"
Jul 28 23:30:58.373036 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:30:58.373000    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-99946b6f7-vdg87"]
Jul 28 23:30:58.375969 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:30:58.375945    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-99946b6f7-vdg87"
Jul 28 23:30:58.386479 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:30:58.386451    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-99946b6f7-vdg87"]
Jul 28 23:30:58.402412 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:30:58.402384    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b624p\" (UniqueName: \"kubernetes.io/projected/ac2a54b5-c479-45a0-bc93-c249641f85ae-kube-api-access-b624p\") pod \"maas-controller-5db89c9c54-tvfs7\" (UID: \"ac2a54b5-c479-45a0-bc93-c249641f85ae\") " pod="opendatahub/maas-controller-5db89c9c54-tvfs7"
Jul 28 23:30:58.402555 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:30:58.402431    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ac2a54b5-c479-45a0-bc93-c249641f85ae-webhook-cert\") pod \"maas-controller-5db89c9c54-tvfs7\" (UID: \"ac2a54b5-c479-45a0-bc93-c249641f85ae\") " pod="opendatahub/maas-controller-5db89c9c54-tvfs7"
Jul 28 23:30:58.402555 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:30:58.402470    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/134f5eda-3789-4a72-a158-a43f6425cc9a-webhook-cert\") pod \"maas-controller-99946b6f7-vdg87\" (UID: \"134f5eda-3789-4a72-a158-a43f6425cc9a\") " pod="opendatahub/maas-controller-99946b6f7-vdg87"
Jul 28 23:30:58.402555 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:30:58.402494    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n48sc\" (UniqueName: \"kubernetes.io/projected/134f5eda-3789-4a72-a158-a43f6425cc9a-kube-api-access-n48sc\") pod \"maas-controller-99946b6f7-vdg87\" (UID: \"134f5eda-3789-4a72-a158-a43f6425cc9a\") " pod="opendatahub/maas-controller-99946b6f7-vdg87"
Jul 28 23:30:58.402741 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:30:58.402610    2572 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found
Jul 28 23:30:58.402741 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:30:58.402687    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ac2a54b5-c479-45a0-bc93-c249641f85ae-webhook-cert podName:ac2a54b5-c479-45a0-bc93-c249641f85ae nodeName:}" failed. No retries permitted until 2026-07-28 23:30:58.902670588 +0000 UTC m=+535.595345074 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/ac2a54b5-c479-45a0-bc93-c249641f85ae-webhook-cert") pod "maas-controller-5db89c9c54-tvfs7" (UID: "ac2a54b5-c479-45a0-bc93-c249641f85ae") : secret "maas-controller-webhook-cert" not found
Jul 28 23:30:58.410511 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:30:58.410491    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b624p\" (UniqueName: \"kubernetes.io/projected/ac2a54b5-c479-45a0-bc93-c249641f85ae-kube-api-access-b624p\") pod \"maas-controller-5db89c9c54-tvfs7\" (UID: \"ac2a54b5-c479-45a0-bc93-c249641f85ae\") " pod="opendatahub/maas-controller-5db89c9c54-tvfs7"
Jul 28 23:30:58.503715 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:30:58.503684    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/134f5eda-3789-4a72-a158-a43f6425cc9a-webhook-cert\") pod \"maas-controller-99946b6f7-vdg87\" (UID: \"134f5eda-3789-4a72-a158-a43f6425cc9a\") " pod="opendatahub/maas-controller-99946b6f7-vdg87"
Jul 28 23:30:58.503909 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:30:58.503725    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n48sc\" (UniqueName: \"kubernetes.io/projected/134f5eda-3789-4a72-a158-a43f6425cc9a-kube-api-access-n48sc\") pod \"maas-controller-99946b6f7-vdg87\" (UID: \"134f5eda-3789-4a72-a158-a43f6425cc9a\") " pod="opendatahub/maas-controller-99946b6f7-vdg87"
Jul 28 23:30:58.503909 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:30:58.503846    2572 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found
Jul 28 23:30:58.504030 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:30:58.503922    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/134f5eda-3789-4a72-a158-a43f6425cc9a-webhook-cert podName:134f5eda-3789-4a72-a158-a43f6425cc9a nodeName:}" failed. No retries permitted until 2026-07-28 23:30:59.003899825 +0000 UTC m=+535.696574311 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/134f5eda-3789-4a72-a158-a43f6425cc9a-webhook-cert") pod "maas-controller-99946b6f7-vdg87" (UID: "134f5eda-3789-4a72-a158-a43f6425cc9a") : secret "maas-controller-webhook-cert" not found
Jul 28 23:30:58.513720 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:30:58.513697    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n48sc\" (UniqueName: \"kubernetes.io/projected/134f5eda-3789-4a72-a158-a43f6425cc9a-kube-api-access-n48sc\") pod \"maas-controller-99946b6f7-vdg87\" (UID: \"134f5eda-3789-4a72-a158-a43f6425cc9a\") " pod="opendatahub/maas-controller-99946b6f7-vdg87"
Jul 28 23:30:58.907370 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:30:58.907268    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ac2a54b5-c479-45a0-bc93-c249641f85ae-webhook-cert\") pod \"maas-controller-5db89c9c54-tvfs7\" (UID: \"ac2a54b5-c479-45a0-bc93-c249641f85ae\") " pod="opendatahub/maas-controller-5db89c9c54-tvfs7"
Jul 28 23:30:58.909732 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:30:58.909704    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ac2a54b5-c479-45a0-bc93-c249641f85ae-webhook-cert\") pod \"maas-controller-5db89c9c54-tvfs7\" (UID: \"ac2a54b5-c479-45a0-bc93-c249641f85ae\") " pod="opendatahub/maas-controller-5db89c9c54-tvfs7"
Jul 28 23:30:59.008488 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:30:59.008451    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/134f5eda-3789-4a72-a158-a43f6425cc9a-webhook-cert\") pod \"maas-controller-99946b6f7-vdg87\" (UID: \"134f5eda-3789-4a72-a158-a43f6425cc9a\") " pod="opendatahub/maas-controller-99946b6f7-vdg87"
Jul 28 23:30:59.010945 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:30:59.010922    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/134f5eda-3789-4a72-a158-a43f6425cc9a-webhook-cert\") pod \"maas-controller-99946b6f7-vdg87\" (UID: \"134f5eda-3789-4a72-a158-a43f6425cc9a\") " pod="opendatahub/maas-controller-99946b6f7-vdg87"
Jul 28 23:30:59.131422 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:30:59.131380    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5db89c9c54-tvfs7"
Jul 28 23:30:59.256380 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:30:59.256355    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5db89c9c54-tvfs7"]
Jul 28 23:30:59.259185 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:30:59.259156    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac2a54b5_c479_45a0_bc93_c249641f85ae.slice/crio-1453da7e462665f74fac4e47beb850fef86942309ec4d7f2ab64246e3c5a91ec WatchSource:0}: Error finding container 1453da7e462665f74fac4e47beb850fef86942309ec4d7f2ab64246e3c5a91ec: Status 404 returned error can't find the container with id 1453da7e462665f74fac4e47beb850fef86942309ec4d7f2ab64246e3c5a91ec
Jul 28 23:30:59.286714 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:30:59.286675    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-99946b6f7-vdg87"
Jul 28 23:30:59.415448 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:30:59.415421    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-99946b6f7-vdg87"]
Jul 28 23:30:59.417312 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:30:59.417284    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod134f5eda_3789_4a72_a158_a43f6425cc9a.slice/crio-30188916cc92bc3df1cdbf12286459744169d0c2b11de5ab31281d9532f9bef9 WatchSource:0}: Error finding container 30188916cc92bc3df1cdbf12286459744169d0c2b11de5ab31281d9532f9bef9: Status 404 returned error can't find the container with id 30188916cc92bc3df1cdbf12286459744169d0c2b11de5ab31281d9532f9bef9
Jul 28 23:30:59.973222 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:30:59.973098    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5db89c9c54-tvfs7" event={"ID":"ac2a54b5-c479-45a0-bc93-c249641f85ae","Type":"ContainerStarted","Data":"1453da7e462665f74fac4e47beb850fef86942309ec4d7f2ab64246e3c5a91ec"}
Jul 28 23:30:59.975692 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:30:59.975660    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-99946b6f7-vdg87" event={"ID":"134f5eda-3789-4a72-a158-a43f6425cc9a","Type":"ContainerStarted","Data":"30188916cc92bc3df1cdbf12286459744169d0c2b11de5ab31281d9532f9bef9"}
Jul 28 23:31:01.987226 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:31:01.987192    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5db89c9c54-tvfs7" event={"ID":"ac2a54b5-c479-45a0-bc93-c249641f85ae","Type":"ContainerStarted","Data":"94bcc28396b3c9f098554841520f39be0e84e18306c6ce6fc5ff9a43d9fe831c"}
Jul 28 23:31:01.987686 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:31:01.987261    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5db89c9c54-tvfs7"
Jul 28 23:31:01.988729 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:31:01.988707    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-99946b6f7-vdg87" event={"ID":"134f5eda-3789-4a72-a158-a43f6425cc9a","Type":"ContainerStarted","Data":"c573200086928acdb6e9246e792b4b6aa055d7a24aadd108924239237d2e9a1a"}
Jul 28 23:31:01.988809 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:31:01.988782    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-99946b6f7-vdg87"
Jul 28 23:31:02.006254 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:31:02.006204    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-5db89c9c54-tvfs7" podStartSLOduration=1.543330588 podStartE2EDuration="4.006187246s" totalImagesPullingTime="2.462856658s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 23:30:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 23:31:02.004927134 +0000 UTC m=+538.697601639" watchObservedRunningTime="2026-07-28 23:31:02.006187246 +0000 UTC m=+538.698861754"
Jul 28 23:31:02.021838 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:31:02.021777    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-99946b6f7-vdg87" podStartSLOduration=1.707984441 podStartE2EDuration="4.021759467s" totalImagesPullingTime="2.313775026s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 23:30:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 23:31:02.020527676 +0000 UTC m=+538.713202189" watchObservedRunningTime="2026-07-28 23:31:02.021759467 +0000 UTC m=+538.714433975"
Jul 28 23:31:12.995415 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:31:12.995379    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-99946b6f7-vdg87"
Jul 28 23:31:12.995872 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:31:12.995449    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5db89c9c54-tvfs7"
Jul 28 23:31:13.057962 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:31:13.057917    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5db89c9c54-tvfs7"]
Jul 28 23:31:13.058190 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:31:13.058135    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-5db89c9c54-tvfs7" podUID="ac2a54b5-c479-45a0-bc93-c249641f85ae" containerName="manager" containerID="cri-o://94bcc28396b3c9f098554841520f39be0e84e18306c6ce6fc5ff9a43d9fe831c" gracePeriod=30
Jul 28 23:31:13.301085 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:31:13.301059    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5db89c9c54-tvfs7"
Jul 28 23:31:13.433928 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:31:13.433895    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ac2a54b5-c479-45a0-bc93-c249641f85ae-webhook-cert\") pod \"ac2a54b5-c479-45a0-bc93-c249641f85ae\" (UID: \"ac2a54b5-c479-45a0-bc93-c249641f85ae\") "
Jul 28 23:31:13.434094 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:31:13.433979    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b624p\" (UniqueName: \"kubernetes.io/projected/ac2a54b5-c479-45a0-bc93-c249641f85ae-kube-api-access-b624p\") pod \"ac2a54b5-c479-45a0-bc93-c249641f85ae\" (UID: \"ac2a54b5-c479-45a0-bc93-c249641f85ae\") "
Jul 28 23:31:13.436132 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:31:13.436096    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ac2a54b5-c479-45a0-bc93-c249641f85ae-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "ac2a54b5-c479-45a0-bc93-c249641f85ae" (UID: "ac2a54b5-c479-45a0-bc93-c249641f85ae"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 23:31:13.436247 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:31:13.436142    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ac2a54b5-c479-45a0-bc93-c249641f85ae-kube-api-access-b624p" (OuterVolumeSpecName: "kube-api-access-b624p") pod "ac2a54b5-c479-45a0-bc93-c249641f85ae" (UID: "ac2a54b5-c479-45a0-bc93-c249641f85ae"). InnerVolumeSpecName "kube-api-access-b624p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 23:31:13.534839 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:31:13.534747    2572 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ac2a54b5-c479-45a0-bc93-c249641f85ae-webhook-cert\") on node \"ip-10-0-131-112.ec2.internal\" DevicePath \"\""
Jul 28 23:31:13.534839 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:31:13.534789    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b624p\" (UniqueName: \"kubernetes.io/projected/ac2a54b5-c479-45a0-bc93-c249641f85ae-kube-api-access-b624p\") on node \"ip-10-0-131-112.ec2.internal\" DevicePath \"\""
Jul 28 23:31:14.039871 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:31:14.039836    2572 generic.go:356] "Generic (PLEG): container finished" podID="ac2a54b5-c479-45a0-bc93-c249641f85ae" containerID="94bcc28396b3c9f098554841520f39be0e84e18306c6ce6fc5ff9a43d9fe831c" exitCode=0
Jul 28 23:31:14.040327 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:31:14.039909    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5db89c9c54-tvfs7"
Jul 28 23:31:14.040327 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:31:14.039932    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5db89c9c54-tvfs7" event={"ID":"ac2a54b5-c479-45a0-bc93-c249641f85ae","Type":"ContainerDied","Data":"94bcc28396b3c9f098554841520f39be0e84e18306c6ce6fc5ff9a43d9fe831c"}
Jul 28 23:31:14.040327 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:31:14.039988    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5db89c9c54-tvfs7" event={"ID":"ac2a54b5-c479-45a0-bc93-c249641f85ae","Type":"ContainerDied","Data":"1453da7e462665f74fac4e47beb850fef86942309ec4d7f2ab64246e3c5a91ec"}
Jul 28 23:31:14.040327 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:31:14.040019    2572 scope.go:117] "RemoveContainer" containerID="94bcc28396b3c9f098554841520f39be0e84e18306c6ce6fc5ff9a43d9fe831c"
Jul 28 23:31:14.050038 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:31:14.050017    2572 scope.go:117] "RemoveContainer" containerID="94bcc28396b3c9f098554841520f39be0e84e18306c6ce6fc5ff9a43d9fe831c"
Jul 28 23:31:14.050296 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:31:14.050280    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"94bcc28396b3c9f098554841520f39be0e84e18306c6ce6fc5ff9a43d9fe831c\": container with ID starting with 94bcc28396b3c9f098554841520f39be0e84e18306c6ce6fc5ff9a43d9fe831c not found: ID does not exist" containerID="94bcc28396b3c9f098554841520f39be0e84e18306c6ce6fc5ff9a43d9fe831c"
Jul 28 23:31:14.050336 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:31:14.050312    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"94bcc28396b3c9f098554841520f39be0e84e18306c6ce6fc5ff9a43d9fe831c"} err="failed to get container status \"94bcc28396b3c9f098554841520f39be0e84e18306c6ce6fc5ff9a43d9fe831c\": rpc error: code = NotFound desc = could not find container \"94bcc28396b3c9f098554841520f39be0e84e18306c6ce6fc5ff9a43d9fe831c\": container with ID starting with 94bcc28396b3c9f098554841520f39be0e84e18306c6ce6fc5ff9a43d9fe831c not found: ID does not exist"
Jul 28 23:31:14.056825 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:31:14.056801    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5db89c9c54-tvfs7"]
Jul 28 23:31:14.058065 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:31:14.058046    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-5db89c9c54-tvfs7"]
Jul 28 23:31:15.865172 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:31:15.865141    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ac2a54b5-c479-45a0-bc93-c249641f85ae" path="/var/lib/kubelet/pods/ac2a54b5-c479-45a0-bc93-c249641f85ae/volumes"
Jul 28 23:31:34.168671 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:31:34.168616    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-d69cbc5b-6r7tb"]
Jul 28 23:31:34.169140 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:31:34.168918    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ac2a54b5-c479-45a0-bc93-c249641f85ae" containerName="manager"
Jul 28 23:31:34.169140 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:31:34.168929    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="ac2a54b5-c479-45a0-bc93-c249641f85ae" containerName="manager"
Jul 28 23:31:34.169140 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:31:34.168976    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="ac2a54b5-c479-45a0-bc93-c249641f85ae" containerName="manager"
Jul 28 23:31:34.172004 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:31:34.171981    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-d69cbc5b-6r7tb"
Jul 28 23:31:34.174407 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:31:34.174379    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-p9ml2\""
Jul 28 23:31:34.174545 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:31:34.174386    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Jul 28 23:31:34.174545 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:31:34.174444    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 28 23:31:34.175350 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:31:34.175334    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 28 23:31:34.182090 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:31:34.182069    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-d69cbc5b-6r7tb"]
Jul 28 23:31:34.302763 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:31:34.302730    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/233d13f2-5403-433a-9c4a-09fb722e747d-maas-api-tls\") pod \"maas-api-d69cbc5b-6r7tb\" (UID: \"233d13f2-5403-433a-9c4a-09fb722e747d\") " pod="odh-ai-gateway-infra/maas-api-d69cbc5b-6r7tb"
Jul 28 23:31:34.302763 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:31:34.302772    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xz2md\" (UniqueName: \"kubernetes.io/projected/233d13f2-5403-433a-9c4a-09fb722e747d-kube-api-access-xz2md\") pod \"maas-api-d69cbc5b-6r7tb\" (UID: \"233d13f2-5403-433a-9c4a-09fb722e747d\") " pod="odh-ai-gateway-infra/maas-api-d69cbc5b-6r7tb"
Jul 28 23:31:34.302979 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:31:34.302866    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/233d13f2-5403-433a-9c4a-09fb722e747d-openshift-service-ca\") pod \"maas-api-d69cbc5b-6r7tb\" (UID: \"233d13f2-5403-433a-9c4a-09fb722e747d\") " pod="odh-ai-gateway-infra/maas-api-d69cbc5b-6r7tb"
Jul 28 23:31:34.404164 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:31:34.404133    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/233d13f2-5403-433a-9c4a-09fb722e747d-openshift-service-ca\") pod \"maas-api-d69cbc5b-6r7tb\" (UID: \"233d13f2-5403-433a-9c4a-09fb722e747d\") " pod="odh-ai-gateway-infra/maas-api-d69cbc5b-6r7tb"
Jul 28 23:31:34.404360 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:31:34.404317    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/233d13f2-5403-433a-9c4a-09fb722e747d-maas-api-tls\") pod \"maas-api-d69cbc5b-6r7tb\" (UID: \"233d13f2-5403-433a-9c4a-09fb722e747d\") " pod="odh-ai-gateway-infra/maas-api-d69cbc5b-6r7tb"
Jul 28 23:31:34.404426 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:31:34.404356    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xz2md\" (UniqueName: \"kubernetes.io/projected/233d13f2-5403-433a-9c4a-09fb722e747d-kube-api-access-xz2md\") pod \"maas-api-d69cbc5b-6r7tb\" (UID: \"233d13f2-5403-433a-9c4a-09fb722e747d\") " pod="odh-ai-gateway-infra/maas-api-d69cbc5b-6r7tb"
Jul 28 23:31:34.404472 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:31:34.404448    2572 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-serving-cert: secret "maas-api-serving-cert" not found
Jul 28 23:31:34.404533 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:31:34.404521    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/233d13f2-5403-433a-9c4a-09fb722e747d-maas-api-tls podName:233d13f2-5403-433a-9c4a-09fb722e747d nodeName:}" failed. No retries permitted until 2026-07-28 23:31:34.904500747 +0000 UTC m=+571.597175235 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/233d13f2-5403-433a-9c4a-09fb722e747d-maas-api-tls") pod "maas-api-d69cbc5b-6r7tb" (UID: "233d13f2-5403-433a-9c4a-09fb722e747d") : secret "maas-api-serving-cert" not found
Jul 28 23:31:34.404837 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:31:34.404816    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/233d13f2-5403-433a-9c4a-09fb722e747d-openshift-service-ca\") pod \"maas-api-d69cbc5b-6r7tb\" (UID: \"233d13f2-5403-433a-9c4a-09fb722e747d\") " pod="odh-ai-gateway-infra/maas-api-d69cbc5b-6r7tb"
Jul 28 23:31:34.412300 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:31:34.412275    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xz2md\" (UniqueName: \"kubernetes.io/projected/233d13f2-5403-433a-9c4a-09fb722e747d-kube-api-access-xz2md\") pod \"maas-api-d69cbc5b-6r7tb\" (UID: \"233d13f2-5403-433a-9c4a-09fb722e747d\") " pod="odh-ai-gateway-infra/maas-api-d69cbc5b-6r7tb"
Jul 28 23:31:34.909556 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:31:34.909513    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/233d13f2-5403-433a-9c4a-09fb722e747d-maas-api-tls\") pod \"maas-api-d69cbc5b-6r7tb\" (UID: \"233d13f2-5403-433a-9c4a-09fb722e747d\") " pod="odh-ai-gateway-infra/maas-api-d69cbc5b-6r7tb"
Jul 28 23:31:34.912031 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:31:34.911999    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/233d13f2-5403-433a-9c4a-09fb722e747d-maas-api-tls\") pod \"maas-api-d69cbc5b-6r7tb\" (UID: \"233d13f2-5403-433a-9c4a-09fb722e747d\") " pod="odh-ai-gateway-infra/maas-api-d69cbc5b-6r7tb"
Jul 28 23:31:35.083472 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:31:35.083438    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-d69cbc5b-6r7tb"
Jul 28 23:31:35.215470 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:31:35.215441    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-d69cbc5b-6r7tb"]
Jul 28 23:31:35.217342 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:31:35.217317    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod233d13f2_5403_433a_9c4a_09fb722e747d.slice/crio-7eb22f7f991177aa638cd73012440612535db16f6cf1a66c64d7f67a5ac471be WatchSource:0}: Error finding container 7eb22f7f991177aa638cd73012440612535db16f6cf1a66c64d7f67a5ac471be: Status 404 returned error can't find the container with id 7eb22f7f991177aa638cd73012440612535db16f6cf1a66c64d7f67a5ac471be
Jul 28 23:31:36.131330 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:31:36.131287    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-d69cbc5b-6r7tb" event={"ID":"233d13f2-5403-433a-9c4a-09fb722e747d","Type":"ContainerStarted","Data":"7eb22f7f991177aa638cd73012440612535db16f6cf1a66c64d7f67a5ac471be"}
Jul 28 23:31:37.140257 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:31:37.139363    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-d69cbc5b-6r7tb" event={"ID":"233d13f2-5403-433a-9c4a-09fb722e747d","Type":"ContainerStarted","Data":"5abad6299bc1a4b104482c687962a25234a8686797e9641941745d2e5056bdf7"}
Jul 28 23:31:37.140257 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:31:37.140191    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-d69cbc5b-6r7tb"
Jul 28 23:31:37.157365 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:31:37.157311    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-d69cbc5b-6r7tb" podStartSLOduration=1.426115929 podStartE2EDuration="3.157294512s" totalImagesPullingTime="1.731178583s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 23:31:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 23:31:37.155174173 +0000 UTC m=+573.847848681" watchObservedRunningTime="2026-07-28 23:31:37.157294512 +0000 UTC m=+573.849969020"
Jul 28 23:31:44.158695 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:31:44.158663    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-d69cbc5b-6r7tb"
Jul 28 23:31:44.176449 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:31:44.176416    2572 generic.go:356] "Generic (PLEG): container finished" podID="f59a2eeb-6c07-48ed-88e5-35f2bd35db3c" containerID="a5add8ca7d582d494417e978891d77907fa4422fa6db1debbe61be33fb28224f" exitCode=0
Jul 28 23:31:44.176603 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:31:44.176483    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-wvxnh" event={"ID":"f59a2eeb-6c07-48ed-88e5-35f2bd35db3c","Type":"ContainerDied","Data":"a5add8ca7d582d494417e978891d77907fa4422fa6db1debbe61be33fb28224f"}
Jul 28 23:31:44.176845 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:31:44.176831    2572 scope.go:117] "RemoveContainer" containerID="a5add8ca7d582d494417e978891d77907fa4422fa6db1debbe61be33fb28224f"
Jul 28 23:31:44.177775 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:31:44.177746    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 23:31:44.177840 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:31:44.177807    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 23:31:44.177876 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:31:44.177848    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 23:31:45.183028 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:31:45.182992    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-wvxnh" event={"ID":"f59a2eeb-6c07-48ed-88e5-35f2bd35db3c","Type":"ContainerStarted","Data":"e0203ecd37ee2a4d6ebfa34da143914070413bda502a3d4b68ba4b7fe9bb4f04"}
Jul 28 23:31:45.396472 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:31:45.396427    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-wvxnh"
Jul 28 23:31:45.396880 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:31:45.396852    2572 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5ccb968c66-wvxnh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.21:15021/healthz/ready\": dial tcp 10.134.0.21:15021: connect: connection refused" start-of-body=
Jul 28 23:31:45.396934 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:31:45.396911    2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-wvxnh" podUID="f59a2eeb-6c07-48ed-88e5-35f2bd35db3c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.21:15021/healthz/ready\": dial tcp 10.134.0.21:15021: connect: connection refused"
Jul 28 23:31:46.396047 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:31:46.396011    2572 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5ccb968c66-wvxnh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.21:15021/healthz/ready\": dial tcp 10.134.0.21:15021: connect: connection refused" start-of-body=
Jul 28 23:31:46.396468 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:31:46.396084    2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-wvxnh" podUID="f59a2eeb-6c07-48ed-88e5-35f2bd35db3c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.21:15021/healthz/ready\": dial tcp 10.134.0.21:15021: connect: connection refused"
Jul 28 23:31:47.396405 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:31:47.396363    2572 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5ccb968c66-wvxnh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.21:15021/healthz/ready\": dial tcp 10.134.0.21:15021: connect: connection refused" start-of-body=
Jul 28 23:31:47.396821 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:31:47.396435    2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-wvxnh" podUID="f59a2eeb-6c07-48ed-88e5-35f2bd35db3c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.21:15021/healthz/ready\": dial tcp 10.134.0.21:15021: connect: connection refused"
Jul 28 23:31:48.396359 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:31:48.396318    2572 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5ccb968c66-wvxnh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.21:15021/healthz/ready\": dial tcp 10.134.0.21:15021: connect: connection refused" start-of-body=
Jul 28 23:31:48.396555 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:31:48.396383    2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-wvxnh" podUID="f59a2eeb-6c07-48ed-88e5-35f2bd35db3c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.21:15021/healthz/ready\": dial tcp 10.134.0.21:15021: connect: connection refused"
Jul 28 23:31:49.396929 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:31:49.396887    2572 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5ccb968c66-wvxnh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.21:15021/healthz/ready\": dial tcp 10.134.0.21:15021: connect: connection refused" start-of-body=
Jul 28 23:31:49.397410 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:31:49.396951    2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-wvxnh" podUID="f59a2eeb-6c07-48ed-88e5-35f2bd35db3c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.21:15021/healthz/ready\": dial tcp 10.134.0.21:15021: connect: connection refused"
Jul 28 23:31:50.400816 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:31:50.400783    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-wvxnh"
Jul 28 23:31:50.401262 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:31:50.401070    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-wvxnh"
Jul 28 23:31:50.401747 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:31:50.401730    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-wvxnh"
Jul 28 23:31:53.909770 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:31:53.909738    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6zd86"]
Jul 28 23:31:53.912382 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:31:53.912363    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6zd86"
Jul 28 23:31:53.914775 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:31:53.914754    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Jul 28 23:31:53.914889 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:31:53.914837    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 28 23:31:53.915613 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:31:53.915598    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 28 23:31:53.915707 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:31:53.915650    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-dlh7r\""
Jul 28 23:31:53.922404 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:31:53.922381    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6zd86"]
Jul 28 23:31:53.976789 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:31:53.976750    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/36a2cdec-3fba-4930-9fb7-afc7ceec1e0f-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-6zd86\" (UID: \"36a2cdec-3fba-4930-9fb7-afc7ceec1e0f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6zd86"
Jul 28 23:31:53.976789 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:31:53.976788    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/36a2cdec-3fba-4930-9fb7-afc7ceec1e0f-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-6zd86\" (UID: \"36a2cdec-3fba-4930-9fb7-afc7ceec1e0f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6zd86"
Jul 28 23:31:53.976990 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:31:53.976810    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/36a2cdec-3fba-4930-9fb7-afc7ceec1e0f-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-6zd86\" (UID: \"36a2cdec-3fba-4930-9fb7-afc7ceec1e0f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6zd86"
Jul 28 23:31:53.976990 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:31:53.976882    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8r845\" (UniqueName: \"kubernetes.io/projected/36a2cdec-3fba-4930-9fb7-afc7ceec1e0f-kube-api-access-8r845\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-6zd86\" (UID: \"36a2cdec-3fba-4930-9fb7-afc7ceec1e0f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6zd86"
Jul 28 23:31:53.976990 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:31:53.976926    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/36a2cdec-3fba-4930-9fb7-afc7ceec1e0f-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-6zd86\" (UID: \"36a2cdec-3fba-4930-9fb7-afc7ceec1e0f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6zd86"
Jul 28 23:31:53.977090 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:31:53.977015    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/36a2cdec-3fba-4930-9fb7-afc7ceec1e0f-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-6zd86\" (UID: \"36a2cdec-3fba-4930-9fb7-afc7ceec1e0f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6zd86"
Jul 28 23:31:54.078181 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:31:54.078142    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/36a2cdec-3fba-4930-9fb7-afc7ceec1e0f-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-6zd86\" (UID: \"36a2cdec-3fba-4930-9fb7-afc7ceec1e0f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6zd86"
Jul 28 23:31:54.078181 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:31:54.078188    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/36a2cdec-3fba-4930-9fb7-afc7ceec1e0f-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-6zd86\" (UID: \"36a2cdec-3fba-4930-9fb7-afc7ceec1e0f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6zd86"
Jul 28 23:31:54.078422 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:31:54.078216    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/36a2cdec-3fba-4930-9fb7-afc7ceec1e0f-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-6zd86\" (UID: \"36a2cdec-3fba-4930-9fb7-afc7ceec1e0f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6zd86"
Jul 28 23:31:54.078422 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:31:54.078239    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/36a2cdec-3fba-4930-9fb7-afc7ceec1e0f-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-6zd86\" (UID: \"36a2cdec-3fba-4930-9fb7-afc7ceec1e0f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6zd86"
Jul 28 23:31:54.078422 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:31:54.078271    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8r845\" (UniqueName: \"kubernetes.io/projected/36a2cdec-3fba-4930-9fb7-afc7ceec1e0f-kube-api-access-8r845\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-6zd86\" (UID: \"36a2cdec-3fba-4930-9fb7-afc7ceec1e0f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6zd86"
Jul 28 23:31:54.078422 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:31:54.078312    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/36a2cdec-3fba-4930-9fb7-afc7ceec1e0f-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-6zd86\" (UID: \"36a2cdec-3fba-4930-9fb7-afc7ceec1e0f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6zd86"
Jul 28 23:31:54.078679 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:31:54.078654    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/36a2cdec-3fba-4930-9fb7-afc7ceec1e0f-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-6zd86\" (UID: \"36a2cdec-3fba-4930-9fb7-afc7ceec1e0f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6zd86"
Jul 28 23:31:54.078755 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:31:54.078705    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/36a2cdec-3fba-4930-9fb7-afc7ceec1e0f-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-6zd86\" (UID: \"36a2cdec-3fba-4930-9fb7-afc7ceec1e0f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6zd86"
Jul 28 23:31:54.078793 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:31:54.078748    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/36a2cdec-3fba-4930-9fb7-afc7ceec1e0f-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-6zd86\" (UID: \"36a2cdec-3fba-4930-9fb7-afc7ceec1e0f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6zd86"
Jul 28 23:31:54.080717 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:31:54.080697    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/36a2cdec-3fba-4930-9fb7-afc7ceec1e0f-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-6zd86\" (UID: \"36a2cdec-3fba-4930-9fb7-afc7ceec1e0f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6zd86"
Jul 28 23:31:54.080963 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:31:54.080946    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/36a2cdec-3fba-4930-9fb7-afc7ceec1e0f-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-6zd86\" (UID: \"36a2cdec-3fba-4930-9fb7-afc7ceec1e0f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6zd86"
Jul 28 23:31:54.085266 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:31:54.085241    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8r845\" (UniqueName: \"kubernetes.io/projected/36a2cdec-3fba-4930-9fb7-afc7ceec1e0f-kube-api-access-8r845\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-6zd86\" (UID: \"36a2cdec-3fba-4930-9fb7-afc7ceec1e0f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6zd86"
Jul 28 23:31:54.223860 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:31:54.223776    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6zd86"
Jul 28 23:31:54.358124 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:31:54.358097    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6zd86"]
Jul 28 23:31:54.359959 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:31:54.359929    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36a2cdec_3fba_4930_9fb7_afc7ceec1e0f.slice/crio-ed0f258eafdd1b65145a491c0d0e8ea0c6ed7ac288c43e722e3a04716d2a08b4 WatchSource:0}: Error finding container ed0f258eafdd1b65145a491c0d0e8ea0c6ed7ac288c43e722e3a04716d2a08b4: Status 404 returned error can't find the container with id ed0f258eafdd1b65145a491c0d0e8ea0c6ed7ac288c43e722e3a04716d2a08b4
Jul 28 23:31:55.227807 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:31:55.226646    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6zd86" event={"ID":"36a2cdec-3fba-4930-9fb7-afc7ceec1e0f","Type":"ContainerStarted","Data":"ed0f258eafdd1b65145a491c0d0e8ea0c6ed7ac288c43e722e3a04716d2a08b4"}
Jul 28 23:31:56.232251 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:31:56.232211    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6zd86" event={"ID":"36a2cdec-3fba-4930-9fb7-afc7ceec1e0f","Type":"ContainerStarted","Data":"f803806d79b088b076275fa7abebecafd601a151266fc715b52cde9cbe9d2b2f"}
Jul 28 23:31:56.247220 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:31:56.247158    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6zd86" podStartSLOduration=2.108788413 podStartE2EDuration="3.24713638s" totalImagesPullingTime="1.138347967s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 23:31:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 23:31:56.246372065 +0000 UTC m=+592.939046574" watchObservedRunningTime="2026-07-28 23:31:56.24713638 +0000 UTC m=+592.939810890"
Jul 28 23:32:04.224286 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:32:04.224239    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6zd86"
Jul 28 23:32:04.224778 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:32:04.224321    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6zd86"
Jul 28 23:32:04.236880 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:32:04.236851    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6zd86"
Jul 28 23:32:04.277243 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:32:04.277219    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6zd86"
Jul 28 23:32:50.799339 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:32:50.799301    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbjw4x"]
Jul 28 23:32:50.802805 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:32:50.802784    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbjw4x"
Jul 28 23:32:50.805081 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:32:50.805058    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Jul 28 23:32:50.812247 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:32:50.812226    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbjw4x"]
Jul 28 23:32:50.854210 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:32:50.854173    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/706f10a5-2ca0-4135-8f41-9ec3e86f05df-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbjw4x\" (UID: \"706f10a5-2ca0-4135-8f41-9ec3e86f05df\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbjw4x"
Jul 28 23:32:50.854424 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:32:50.854217    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/706f10a5-2ca0-4135-8f41-9ec3e86f05df-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbjw4x\" (UID: \"706f10a5-2ca0-4135-8f41-9ec3e86f05df\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbjw4x"
Jul 28 23:32:50.854424 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:32:50.854293    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lf2zr\" (UniqueName: \"kubernetes.io/projected/706f10a5-2ca0-4135-8f41-9ec3e86f05df-kube-api-access-lf2zr\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbjw4x\" (UID: \"706f10a5-2ca0-4135-8f41-9ec3e86f05df\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbjw4x"
Jul 28 23:32:50.854424 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:32:50.854326    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/706f10a5-2ca0-4135-8f41-9ec3e86f05df-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbjw4x\" (UID: \"706f10a5-2ca0-4135-8f41-9ec3e86f05df\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbjw4x"
Jul 28 23:32:50.854424 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:32:50.854354    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/706f10a5-2ca0-4135-8f41-9ec3e86f05df-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbjw4x\" (UID: \"706f10a5-2ca0-4135-8f41-9ec3e86f05df\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbjw4x"
Jul 28 23:32:50.854574 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:32:50.854474    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/706f10a5-2ca0-4135-8f41-9ec3e86f05df-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbjw4x\" (UID: \"706f10a5-2ca0-4135-8f41-9ec3e86f05df\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbjw4x"
Jul 28 23:32:50.955715 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:32:50.955682    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/706f10a5-2ca0-4135-8f41-9ec3e86f05df-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbjw4x\" (UID: \"706f10a5-2ca0-4135-8f41-9ec3e86f05df\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbjw4x"
Jul 28 23:32:50.955900 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:32:50.955726    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/706f10a5-2ca0-4135-8f41-9ec3e86f05df-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbjw4x\" (UID: \"706f10a5-2ca0-4135-8f41-9ec3e86f05df\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbjw4x"
Jul 28 23:32:50.955900 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:32:50.955748    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/706f10a5-2ca0-4135-8f41-9ec3e86f05df-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbjw4x\" (UID: \"706f10a5-2ca0-4135-8f41-9ec3e86f05df\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbjw4x"
Jul 28 23:32:50.955900 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:32:50.955877    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lf2zr\" (UniqueName: \"kubernetes.io/projected/706f10a5-2ca0-4135-8f41-9ec3e86f05df-kube-api-access-lf2zr\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbjw4x\" (UID: \"706f10a5-2ca0-4135-8f41-9ec3e86f05df\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbjw4x"
Jul 28 23:32:50.956066 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:32:50.955923    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/706f10a5-2ca0-4135-8f41-9ec3e86f05df-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbjw4x\" (UID: \"706f10a5-2ca0-4135-8f41-9ec3e86f05df\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbjw4x"
Jul 28 23:32:50.956066 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:32:50.955969    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/706f10a5-2ca0-4135-8f41-9ec3e86f05df-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbjw4x\" (UID: \"706f10a5-2ca0-4135-8f41-9ec3e86f05df\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbjw4x"
Jul 28 23:32:50.956168 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:32:50.956087    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/706f10a5-2ca0-4135-8f41-9ec3e86f05df-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbjw4x\" (UID: \"706f10a5-2ca0-4135-8f41-9ec3e86f05df\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbjw4x"
Jul 28 23:32:50.956298 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:32:50.956277    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/706f10a5-2ca0-4135-8f41-9ec3e86f05df-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbjw4x\" (UID: \"706f10a5-2ca0-4135-8f41-9ec3e86f05df\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbjw4x"
Jul 28 23:32:50.956358 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:32:50.956315    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/706f10a5-2ca0-4135-8f41-9ec3e86f05df-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbjw4x\" (UID: \"706f10a5-2ca0-4135-8f41-9ec3e86f05df\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbjw4x"
Jul 28 23:32:50.958034 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:32:50.958013    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/706f10a5-2ca0-4135-8f41-9ec3e86f05df-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbjw4x\" (UID: \"706f10a5-2ca0-4135-8f41-9ec3e86f05df\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbjw4x"
Jul 28 23:32:50.958299 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:32:50.958282    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/706f10a5-2ca0-4135-8f41-9ec3e86f05df-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbjw4x\" (UID: \"706f10a5-2ca0-4135-8f41-9ec3e86f05df\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbjw4x"
Jul 28 23:32:50.963285 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:32:50.963266    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lf2zr\" (UniqueName: \"kubernetes.io/projected/706f10a5-2ca0-4135-8f41-9ec3e86f05df-kube-api-access-lf2zr\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbjw4x\" (UID: \"706f10a5-2ca0-4135-8f41-9ec3e86f05df\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbjw4x"
Jul 28 23:32:51.113502 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:32:51.113427    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbjw4x"
Jul 28 23:32:51.237346 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:32:51.237246    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbjw4x"]
Jul 28 23:32:51.239894 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:32:51.239854    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod706f10a5_2ca0_4135_8f41_9ec3e86f05df.slice/crio-077a635e9125b4ffb5d819bef58f9b7d257b69d33d56880ee7a5f5dd187f12a3 WatchSource:0}: Error finding container 077a635e9125b4ffb5d819bef58f9b7d257b69d33d56880ee7a5f5dd187f12a3: Status 404 returned error can't find the container with id 077a635e9125b4ffb5d819bef58f9b7d257b69d33d56880ee7a5f5dd187f12a3
Jul 28 23:32:51.241172 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:32:51.241153    2572 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 23:32:51.457524 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:32:51.457460    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbjw4x" event={"ID":"706f10a5-2ca0-4135-8f41-9ec3e86f05df","Type":"ContainerStarted","Data":"077a635e9125b4ffb5d819bef58f9b7d257b69d33d56880ee7a5f5dd187f12a3"}
Jul 28 23:32:52.465650 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:32:52.465604    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbjw4x" event={"ID":"706f10a5-2ca0-4135-8f41-9ec3e86f05df","Type":"ContainerStarted","Data":"55420572552fa993e902fcdf5e5b09673ae33782b1ef68bc3038deb93b9ffee5"}
Jul 28 23:32:52.483374 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:32:52.483322    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbjw4x" podStartSLOduration=2.27130306 podStartE2EDuration="2.483305108s" totalImagesPullingTime="212.002048ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 23:32:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 23:32:52.481405561 +0000 UTC m=+649.174080069" watchObservedRunningTime="2026-07-28 23:32:52.483305108 +0000 UTC m=+649.175979618"
Jul 28 23:33:01.114221 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:33:01.114188    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbjw4x"
Jul 28 23:33:01.114670 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:33:01.114253    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbjw4x"
Jul 28 23:33:01.126547 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:33:01.126526    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbjw4x"
Jul 28 23:33:01.513159 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:33:01.513127    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbjw4x"
Jul 28 23:33:11.615129 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:33:11.615097    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-k268w"]
Jul 28 23:33:11.618344 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:33:11.618324    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-k268w"
Jul 28 23:33:11.620707 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:33:11.620689    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Jul 28 23:33:11.628444 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:33:11.628420    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-k268w"]
Jul 28 23:33:11.731141 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:33:11.731101    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ebfea51e-ea81-4923-afe3-986b98a60be4-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-k268w\" (UID: \"ebfea51e-ea81-4923-afe3-986b98a60be4\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-k268w"
Jul 28 23:33:11.731141 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:33:11.731148    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ebfea51e-ea81-4923-afe3-986b98a60be4-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-k268w\" (UID: \"ebfea51e-ea81-4923-afe3-986b98a60be4\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-k268w"
Jul 28 23:33:11.731339 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:33:11.731181    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ebfea51e-ea81-4923-afe3-986b98a60be4-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-k268w\" (UID: \"ebfea51e-ea81-4923-afe3-986b98a60be4\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-k268w"
Jul 28 23:33:11.731339 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:33:11.731209    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ebfea51e-ea81-4923-afe3-986b98a60be4-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-k268w\" (UID: \"ebfea51e-ea81-4923-afe3-986b98a60be4\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-k268w"
Jul 28 23:33:11.731339 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:33:11.731231    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wzjxm\" (UniqueName: \"kubernetes.io/projected/ebfea51e-ea81-4923-afe3-986b98a60be4-kube-api-access-wzjxm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-k268w\" (UID: \"ebfea51e-ea81-4923-afe3-986b98a60be4\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-k268w"
Jul 28 23:33:11.731339 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:33:11.731292    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ebfea51e-ea81-4923-afe3-986b98a60be4-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-k268w\" (UID: \"ebfea51e-ea81-4923-afe3-986b98a60be4\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-k268w"
Jul 28 23:33:11.832414 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:33:11.832374    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wzjxm\" (UniqueName: \"kubernetes.io/projected/ebfea51e-ea81-4923-afe3-986b98a60be4-kube-api-access-wzjxm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-k268w\" (UID: \"ebfea51e-ea81-4923-afe3-986b98a60be4\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-k268w"
Jul 28 23:33:11.832414 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:33:11.832416    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ebfea51e-ea81-4923-afe3-986b98a60be4-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-k268w\" (UID: \"ebfea51e-ea81-4923-afe3-986b98a60be4\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-k268w"
Jul 28 23:33:11.832690 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:33:11.832468    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ebfea51e-ea81-4923-afe3-986b98a60be4-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-k268w\" (UID: \"ebfea51e-ea81-4923-afe3-986b98a60be4\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-k268w"
Jul 28 23:33:11.832690 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:33:11.832491    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ebfea51e-ea81-4923-afe3-986b98a60be4-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-k268w\" (UID: \"ebfea51e-ea81-4923-afe3-986b98a60be4\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-k268w"
Jul 28 23:33:11.832690 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:33:11.832524    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ebfea51e-ea81-4923-afe3-986b98a60be4-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-k268w\" (UID: \"ebfea51e-ea81-4923-afe3-986b98a60be4\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-k268w"
Jul 28 23:33:11.832690 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:33:11.832549    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ebfea51e-ea81-4923-afe3-986b98a60be4-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-k268w\" (UID: \"ebfea51e-ea81-4923-afe3-986b98a60be4\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-k268w"
Jul 28 23:33:11.832977 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:33:11.832952    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ebfea51e-ea81-4923-afe3-986b98a60be4-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-k268w\" (UID: \"ebfea51e-ea81-4923-afe3-986b98a60be4\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-k268w"
Jul 28 23:33:11.832977 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:33:11.832967    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ebfea51e-ea81-4923-afe3-986b98a60be4-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-k268w\" (UID: \"ebfea51e-ea81-4923-afe3-986b98a60be4\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-k268w"
Jul 28 23:33:11.833167 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:33:11.833100    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ebfea51e-ea81-4923-afe3-986b98a60be4-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-k268w\" (UID: \"ebfea51e-ea81-4923-afe3-986b98a60be4\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-k268w"
Jul 28 23:33:11.834931 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:33:11.834912    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ebfea51e-ea81-4923-afe3-986b98a60be4-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-k268w\" (UID: \"ebfea51e-ea81-4923-afe3-986b98a60be4\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-k268w"
Jul 28 23:33:11.835419 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:33:11.835398    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ebfea51e-ea81-4923-afe3-986b98a60be4-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-k268w\" (UID: \"ebfea51e-ea81-4923-afe3-986b98a60be4\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-k268w"
Jul 28 23:33:11.839999 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:33:11.839976    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wzjxm\" (UniqueName: \"kubernetes.io/projected/ebfea51e-ea81-4923-afe3-986b98a60be4-kube-api-access-wzjxm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-k268w\" (UID: \"ebfea51e-ea81-4923-afe3-986b98a60be4\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-k268w"
Jul 28 23:33:11.931999 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:33:11.931909    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-k268w"
Jul 28 23:33:12.062784 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:33:12.062729    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-k268w"]
Jul 28 23:33:12.065287 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:33:12.065260    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podebfea51e_ea81_4923_afe3_986b98a60be4.slice/crio-0c747316f1b716d38f36b9e862f2ae887f430cc3adf2884b03ff4b9b6919c6ed WatchSource:0}: Error finding container 0c747316f1b716d38f36b9e862f2ae887f430cc3adf2884b03ff4b9b6919c6ed: Status 404 returned error can't find the container with id 0c747316f1b716d38f36b9e862f2ae887f430cc3adf2884b03ff4b9b6919c6ed
Jul 28 23:33:12.547416 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:33:12.547318    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-k268w" event={"ID":"ebfea51e-ea81-4923-afe3-986b98a60be4","Type":"ContainerStarted","Data":"cebef533b3b2c1c01c53a15bc20fef3bd4db294dba54cd8ba39fa96eae267f0e"}
Jul 28 23:33:12.547416 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:33:12.547402    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-k268w" event={"ID":"ebfea51e-ea81-4923-afe3-986b98a60be4","Type":"ContainerStarted","Data":"0c747316f1b716d38f36b9e862f2ae887f430cc3adf2884b03ff4b9b6919c6ed"}
Jul 28 23:33:12.563908 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:33:12.563847    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-k268w" podStartSLOduration=1.332403136 podStartE2EDuration="1.563829886s" totalImagesPullingTime="231.42675ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 23:33:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 23:33:12.562350936 +0000 UTC m=+669.255025446" watchObservedRunningTime="2026-07-28 23:33:12.563829886 +0000 UTC m=+669.256504394"
Jul 28 23:33:21.932737 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:33:21.932698    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-k268w"
Jul 28 23:33:21.933193 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:33:21.932759    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-k268w"
Jul 28 23:33:21.944853 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:33:21.944823    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-k268w"
Jul 28 23:33:22.600505 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:33:22.600473    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-k268w"
Jul 28 23:35:13.361550 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:35:13.361510    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-99946b6f7-vdg87"]
Jul 28 23:35:13.362204 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:35:13.361729    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-99946b6f7-vdg87" podUID="134f5eda-3789-4a72-a158-a43f6425cc9a" containerName="manager" containerID="cri-o://c573200086928acdb6e9246e792b4b6aa055d7a24aadd108924239237d2e9a1a" gracePeriod=30
Jul 28 23:35:13.620512 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:35:13.620451    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-99946b6f7-vdg87"
Jul 28 23:35:13.754199 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:35:13.754168    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/134f5eda-3789-4a72-a158-a43f6425cc9a-webhook-cert\") pod \"134f5eda-3789-4a72-a158-a43f6425cc9a\" (UID: \"134f5eda-3789-4a72-a158-a43f6425cc9a\") "
Jul 28 23:35:13.754383 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:35:13.754217    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n48sc\" (UniqueName: \"kubernetes.io/projected/134f5eda-3789-4a72-a158-a43f6425cc9a-kube-api-access-n48sc\") pod \"134f5eda-3789-4a72-a158-a43f6425cc9a\" (UID: \"134f5eda-3789-4a72-a158-a43f6425cc9a\") "
Jul 28 23:35:13.756312 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:35:13.756284    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/134f5eda-3789-4a72-a158-a43f6425cc9a-kube-api-access-n48sc" (OuterVolumeSpecName: "kube-api-access-n48sc") pod "134f5eda-3789-4a72-a158-a43f6425cc9a" (UID: "134f5eda-3789-4a72-a158-a43f6425cc9a"). InnerVolumeSpecName "kube-api-access-n48sc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 23:35:13.756444 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:35:13.756281    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/134f5eda-3789-4a72-a158-a43f6425cc9a-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "134f5eda-3789-4a72-a158-a43f6425cc9a" (UID: "134f5eda-3789-4a72-a158-a43f6425cc9a"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 23:35:13.854772 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:35:13.854732    2572 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/134f5eda-3789-4a72-a158-a43f6425cc9a-webhook-cert\") on node \"ip-10-0-131-112.ec2.internal\" DevicePath \"\""
Jul 28 23:35:13.854772 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:35:13.854767    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n48sc\" (UniqueName: \"kubernetes.io/projected/134f5eda-3789-4a72-a158-a43f6425cc9a-kube-api-access-n48sc\") on node \"ip-10-0-131-112.ec2.internal\" DevicePath \"\""
Jul 28 23:35:14.057060 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:35:14.057023    2572 generic.go:356] "Generic (PLEG): container finished" podID="134f5eda-3789-4a72-a158-a43f6425cc9a" containerID="c573200086928acdb6e9246e792b4b6aa055d7a24aadd108924239237d2e9a1a" exitCode=0
Jul 28 23:35:14.057208 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:35:14.057096    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-99946b6f7-vdg87"
Jul 28 23:35:14.057208 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:35:14.057118    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-99946b6f7-vdg87" event={"ID":"134f5eda-3789-4a72-a158-a43f6425cc9a","Type":"ContainerDied","Data":"c573200086928acdb6e9246e792b4b6aa055d7a24aadd108924239237d2e9a1a"}
Jul 28 23:35:14.057208 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:35:14.057189    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-99946b6f7-vdg87" event={"ID":"134f5eda-3789-4a72-a158-a43f6425cc9a","Type":"ContainerDied","Data":"30188916cc92bc3df1cdbf12286459744169d0c2b11de5ab31281d9532f9bef9"}
Jul 28 23:35:14.057332 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:35:14.057215    2572 scope.go:117] "RemoveContainer" containerID="c573200086928acdb6e9246e792b4b6aa055d7a24aadd108924239237d2e9a1a"
Jul 28 23:35:14.065451 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:35:14.065429    2572 scope.go:117] "RemoveContainer" containerID="c573200086928acdb6e9246e792b4b6aa055d7a24aadd108924239237d2e9a1a"
Jul 28 23:35:14.065763 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:35:14.065735    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c573200086928acdb6e9246e792b4b6aa055d7a24aadd108924239237d2e9a1a\": container with ID starting with c573200086928acdb6e9246e792b4b6aa055d7a24aadd108924239237d2e9a1a not found: ID does not exist" containerID="c573200086928acdb6e9246e792b4b6aa055d7a24aadd108924239237d2e9a1a"
Jul 28 23:35:14.065825 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:35:14.065772    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c573200086928acdb6e9246e792b4b6aa055d7a24aadd108924239237d2e9a1a"} err="failed to get container status \"c573200086928acdb6e9246e792b4b6aa055d7a24aadd108924239237d2e9a1a\": rpc error: code = NotFound desc = could not find container \"c573200086928acdb6e9246e792b4b6aa055d7a24aadd108924239237d2e9a1a\": container with ID starting with c573200086928acdb6e9246e792b4b6aa055d7a24aadd108924239237d2e9a1a not found: ID does not exist"
Jul 28 23:35:14.073763 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:35:14.073736    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-99946b6f7-vdg87"]
Jul 28 23:35:14.076760 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:35:14.076736    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-99946b6f7-vdg87"]
Jul 28 23:35:15.864673 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:35:15.864644    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="134f5eda-3789-4a72-a158-a43f6425cc9a" path="/var/lib/kubelet/pods/134f5eda-3789-4a72-a158-a43f6425cc9a/volumes"
Jul 28 23:36:14.840079 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:36:14.840047    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-99946b6f7-2tk8v"]
Jul 28 23:36:14.840679 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:36:14.840336    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="134f5eda-3789-4a72-a158-a43f6425cc9a" containerName="manager"
Jul 28 23:36:14.840679 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:36:14.840346    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="134f5eda-3789-4a72-a158-a43f6425cc9a" containerName="manager"
Jul 28 23:36:14.840679 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:36:14.840395    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="134f5eda-3789-4a72-a158-a43f6425cc9a" containerName="manager"
Jul 28 23:36:14.845935 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:36:14.845914    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-99946b6f7-2tk8v"
Jul 28 23:36:14.849649 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:36:14.849608    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 28 23:36:14.849768 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:36:14.849662    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-j2h85\""
Jul 28 23:36:14.849768 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:36:14.849664    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 28 23:36:14.854340 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:36:14.854315    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-99946b6f7-2tk8v"]
Jul 28 23:36:14.982845 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:36:14.982804    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/dac322d2-3c62-4488-8c71-e1827d9d8566-webhook-cert\") pod \"maas-controller-99946b6f7-2tk8v\" (UID: \"dac322d2-3c62-4488-8c71-e1827d9d8566\") " pod="opendatahub/maas-controller-99946b6f7-2tk8v"
Jul 28 23:36:14.982845 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:36:14.982851    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xvdqm\" (UniqueName: \"kubernetes.io/projected/dac322d2-3c62-4488-8c71-e1827d9d8566-kube-api-access-xvdqm\") pod \"maas-controller-99946b6f7-2tk8v\" (UID: \"dac322d2-3c62-4488-8c71-e1827d9d8566\") " pod="opendatahub/maas-controller-99946b6f7-2tk8v"
Jul 28 23:36:15.083632 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:36:15.083592    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/dac322d2-3c62-4488-8c71-e1827d9d8566-webhook-cert\") pod \"maas-controller-99946b6f7-2tk8v\" (UID: \"dac322d2-3c62-4488-8c71-e1827d9d8566\") " pod="opendatahub/maas-controller-99946b6f7-2tk8v"
Jul 28 23:36:15.083766 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:36:15.083675    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xvdqm\" (UniqueName: \"kubernetes.io/projected/dac322d2-3c62-4488-8c71-e1827d9d8566-kube-api-access-xvdqm\") pod \"maas-controller-99946b6f7-2tk8v\" (UID: \"dac322d2-3c62-4488-8c71-e1827d9d8566\") " pod="opendatahub/maas-controller-99946b6f7-2tk8v"
Jul 28 23:36:15.086185 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:36:15.086154    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/dac322d2-3c62-4488-8c71-e1827d9d8566-webhook-cert\") pod \"maas-controller-99946b6f7-2tk8v\" (UID: \"dac322d2-3c62-4488-8c71-e1827d9d8566\") " pod="opendatahub/maas-controller-99946b6f7-2tk8v"
Jul 28 23:36:15.090989 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:36:15.090925    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xvdqm\" (UniqueName: \"kubernetes.io/projected/dac322d2-3c62-4488-8c71-e1827d9d8566-kube-api-access-xvdqm\") pod \"maas-controller-99946b6f7-2tk8v\" (UID: \"dac322d2-3c62-4488-8c71-e1827d9d8566\") " pod="opendatahub/maas-controller-99946b6f7-2tk8v"
Jul 28 23:36:15.158104 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:36:15.158056    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-99946b6f7-2tk8v"
Jul 28 23:36:15.289544 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:36:15.289518    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-99946b6f7-2tk8v"]
Jul 28 23:36:15.291711 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:36:15.291683    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddac322d2_3c62_4488_8c71_e1827d9d8566.slice/crio-26ab38878ff54e64e43cb0a2ac956b91006565d974d5184d042b95de7f753713 WatchSource:0}: Error finding container 26ab38878ff54e64e43cb0a2ac956b91006565d974d5184d042b95de7f753713: Status 404 returned error can't find the container with id 26ab38878ff54e64e43cb0a2ac956b91006565d974d5184d042b95de7f753713
Jul 28 23:36:15.314565 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:36:15.314534    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-99946b6f7-2tk8v" event={"ID":"dac322d2-3c62-4488-8c71-e1827d9d8566","Type":"ContainerStarted","Data":"26ab38878ff54e64e43cb0a2ac956b91006565d974d5184d042b95de7f753713"}
Jul 28 23:36:16.320551 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:36:16.320512    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-99946b6f7-2tk8v" event={"ID":"dac322d2-3c62-4488-8c71-e1827d9d8566","Type":"ContainerStarted","Data":"977ac58dc24edd22d3574f837b8b4af61c7e661045228d86e9ed8f1c542c5f0a"}
Jul 28 23:36:16.320990 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:36:16.320648    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-99946b6f7-2tk8v"
Jul 28 23:36:16.336812 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:36:16.336760    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-99946b6f7-2tk8v" podStartSLOduration=1.9411798839999999 podStartE2EDuration="2.336745414s" totalImagesPullingTime="395.56553ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 23:36:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 23:36:16.335283358 +0000 UTC m=+853.027957868" watchObservedRunningTime="2026-07-28 23:36:16.336745414 +0000 UTC m=+853.029419922"
Jul 28 23:36:27.327379 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:36:27.327350    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-99946b6f7-2tk8v"
Jul 28 23:36:31.940026 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:36:31.939990    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-99946b6f7-2tk8v"]
Jul 28 23:36:31.940420 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:36:31.940187    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-99946b6f7-2tk8v" podUID="dac322d2-3c62-4488-8c71-e1827d9d8566" containerName="manager" containerID="cri-o://977ac58dc24edd22d3574f837b8b4af61c7e661045228d86e9ed8f1c542c5f0a" gracePeriod=30
Jul 28 23:36:32.188342 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:36:32.188319    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-99946b6f7-2tk8v"
Jul 28 23:36:32.230017 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:36:32.229986    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/dac322d2-3c62-4488-8c71-e1827d9d8566-webhook-cert\") pod \"dac322d2-3c62-4488-8c71-e1827d9d8566\" (UID: \"dac322d2-3c62-4488-8c71-e1827d9d8566\") "
Jul 28 23:36:32.230192 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:36:32.230118    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xvdqm\" (UniqueName: \"kubernetes.io/projected/dac322d2-3c62-4488-8c71-e1827d9d8566-kube-api-access-xvdqm\") pod \"dac322d2-3c62-4488-8c71-e1827d9d8566\" (UID: \"dac322d2-3c62-4488-8c71-e1827d9d8566\") "
Jul 28 23:36:32.232084 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:36:32.232057    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dac322d2-3c62-4488-8c71-e1827d9d8566-kube-api-access-xvdqm" (OuterVolumeSpecName: "kube-api-access-xvdqm") pod "dac322d2-3c62-4488-8c71-e1827d9d8566" (UID: "dac322d2-3c62-4488-8c71-e1827d9d8566"). InnerVolumeSpecName "kube-api-access-xvdqm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 23:36:32.232206 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:36:32.232097    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dac322d2-3c62-4488-8c71-e1827d9d8566-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "dac322d2-3c62-4488-8c71-e1827d9d8566" (UID: "dac322d2-3c62-4488-8c71-e1827d9d8566"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 23:36:32.331728 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:36:32.331691    2572 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/dac322d2-3c62-4488-8c71-e1827d9d8566-webhook-cert\") on node \"ip-10-0-131-112.ec2.internal\" DevicePath \"\""
Jul 28 23:36:32.331728 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:36:32.331721    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xvdqm\" (UniqueName: \"kubernetes.io/projected/dac322d2-3c62-4488-8c71-e1827d9d8566-kube-api-access-xvdqm\") on node \"ip-10-0-131-112.ec2.internal\" DevicePath \"\""
Jul 28 23:36:32.387553 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:36:32.387513    2572 generic.go:356] "Generic (PLEG): container finished" podID="dac322d2-3c62-4488-8c71-e1827d9d8566" containerID="977ac58dc24edd22d3574f837b8b4af61c7e661045228d86e9ed8f1c542c5f0a" exitCode=0
Jul 28 23:36:32.387553 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:36:32.387557    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-99946b6f7-2tk8v" event={"ID":"dac322d2-3c62-4488-8c71-e1827d9d8566","Type":"ContainerDied","Data":"977ac58dc24edd22d3574f837b8b4af61c7e661045228d86e9ed8f1c542c5f0a"}
Jul 28 23:36:32.387828 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:36:32.387565    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-99946b6f7-2tk8v"
Jul 28 23:36:32.387828 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:36:32.387611    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-99946b6f7-2tk8v" event={"ID":"dac322d2-3c62-4488-8c71-e1827d9d8566","Type":"ContainerDied","Data":"26ab38878ff54e64e43cb0a2ac956b91006565d974d5184d042b95de7f753713"}
Jul 28 23:36:32.387828 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:36:32.387674    2572 scope.go:117] "RemoveContainer" containerID="977ac58dc24edd22d3574f837b8b4af61c7e661045228d86e9ed8f1c542c5f0a"
Jul 28 23:36:32.396355 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:36:32.396337    2572 scope.go:117] "RemoveContainer" containerID="977ac58dc24edd22d3574f837b8b4af61c7e661045228d86e9ed8f1c542c5f0a"
Jul 28 23:36:32.396675 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:36:32.396651    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"977ac58dc24edd22d3574f837b8b4af61c7e661045228d86e9ed8f1c542c5f0a\": container with ID starting with 977ac58dc24edd22d3574f837b8b4af61c7e661045228d86e9ed8f1c542c5f0a not found: ID does not exist" containerID="977ac58dc24edd22d3574f837b8b4af61c7e661045228d86e9ed8f1c542c5f0a"
Jul 28 23:36:32.396722 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:36:32.396686    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"977ac58dc24edd22d3574f837b8b4af61c7e661045228d86e9ed8f1c542c5f0a"} err="failed to get container status \"977ac58dc24edd22d3574f837b8b4af61c7e661045228d86e9ed8f1c542c5f0a\": rpc error: code = NotFound desc = could not find container \"977ac58dc24edd22d3574f837b8b4af61c7e661045228d86e9ed8f1c542c5f0a\": container with ID starting with 977ac58dc24edd22d3574f837b8b4af61c7e661045228d86e9ed8f1c542c5f0a not found: ID does not exist"
Jul 28 23:36:32.408909 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:36:32.408871    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-99946b6f7-2tk8v"]
Jul 28 23:36:32.411781 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:36:32.411757    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-99946b6f7-2tk8v"]
Jul 28 23:36:33.173759 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:36:33.173724    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-99946b6f7-x5hp9"]
Jul 28 23:36:33.174125 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:36:33.174046    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dac322d2-3c62-4488-8c71-e1827d9d8566" containerName="manager"
Jul 28 23:36:33.174125 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:36:33.174057    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="dac322d2-3c62-4488-8c71-e1827d9d8566" containerName="manager"
Jul 28 23:36:33.174125 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:36:33.174121    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="dac322d2-3c62-4488-8c71-e1827d9d8566" containerName="manager"
Jul 28 23:36:33.178181 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:36:33.178160    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-99946b6f7-x5hp9"
Jul 28 23:36:33.181406 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:36:33.181386    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 28 23:36:33.181530 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:36:33.181386    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 28 23:36:33.181530 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:36:33.181386    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-j2h85\""
Jul 28 23:36:33.186530 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:36:33.186506    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-99946b6f7-x5hp9"]
Jul 28 23:36:33.241459 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:36:33.241422    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e1c66ad8-11e8-4447-b28f-decc108cfaf5-webhook-cert\") pod \"maas-controller-99946b6f7-x5hp9\" (UID: \"e1c66ad8-11e8-4447-b28f-decc108cfaf5\") " pod="opendatahub/maas-controller-99946b6f7-x5hp9"
Jul 28 23:36:33.241676 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:36:33.241482    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xg2xm\" (UniqueName: \"kubernetes.io/projected/e1c66ad8-11e8-4447-b28f-decc108cfaf5-kube-api-access-xg2xm\") pod \"maas-controller-99946b6f7-x5hp9\" (UID: \"e1c66ad8-11e8-4447-b28f-decc108cfaf5\") " pod="opendatahub/maas-controller-99946b6f7-x5hp9"
Jul 28 23:36:33.342824 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:36:33.342784    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xg2xm\" (UniqueName: \"kubernetes.io/projected/e1c66ad8-11e8-4447-b28f-decc108cfaf5-kube-api-access-xg2xm\") pod \"maas-controller-99946b6f7-x5hp9\" (UID: \"e1c66ad8-11e8-4447-b28f-decc108cfaf5\") " pod="opendatahub/maas-controller-99946b6f7-x5hp9"
Jul 28 23:36:33.342995 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:36:33.342862    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e1c66ad8-11e8-4447-b28f-decc108cfaf5-webhook-cert\") pod \"maas-controller-99946b6f7-x5hp9\" (UID: \"e1c66ad8-11e8-4447-b28f-decc108cfaf5\") " pod="opendatahub/maas-controller-99946b6f7-x5hp9"
Jul 28 23:36:33.345275 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:36:33.345243    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e1c66ad8-11e8-4447-b28f-decc108cfaf5-webhook-cert\") pod \"maas-controller-99946b6f7-x5hp9\" (UID: \"e1c66ad8-11e8-4447-b28f-decc108cfaf5\") " pod="opendatahub/maas-controller-99946b6f7-x5hp9"
Jul 28 23:36:33.350123 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:36:33.350095    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xg2xm\" (UniqueName: \"kubernetes.io/projected/e1c66ad8-11e8-4447-b28f-decc108cfaf5-kube-api-access-xg2xm\") pod \"maas-controller-99946b6f7-x5hp9\" (UID: \"e1c66ad8-11e8-4447-b28f-decc108cfaf5\") " pod="opendatahub/maas-controller-99946b6f7-x5hp9"
Jul 28 23:36:33.489593 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:36:33.489565    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-99946b6f7-x5hp9"
Jul 28 23:36:33.627390 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:36:33.627364    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-99946b6f7-x5hp9"]
Jul 28 23:36:33.629946 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:36:33.629916    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1c66ad8_11e8_4447_b28f_decc108cfaf5.slice/crio-8b8b89e69b6e62f5e498b0e85d93d3da0e0f96dfff97e1107b3f9edbb09597c7 WatchSource:0}: Error finding container 8b8b89e69b6e62f5e498b0e85d93d3da0e0f96dfff97e1107b3f9edbb09597c7: Status 404 returned error can't find the container with id 8b8b89e69b6e62f5e498b0e85d93d3da0e0f96dfff97e1107b3f9edbb09597c7
Jul 28 23:36:33.866161 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:36:33.866081    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dac322d2-3c62-4488-8c71-e1827d9d8566" path="/var/lib/kubelet/pods/dac322d2-3c62-4488-8c71-e1827d9d8566/volumes"
Jul 28 23:36:34.398224 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:36:34.398187    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-99946b6f7-x5hp9" event={"ID":"e1c66ad8-11e8-4447-b28f-decc108cfaf5","Type":"ContainerStarted","Data":"7fd8da927589d98ca56d23db2fbe77a099ba054b557ae91aec5de10d904a6f69"}
Jul 28 23:36:34.398728 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:36:34.398243    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-99946b6f7-x5hp9" event={"ID":"e1c66ad8-11e8-4447-b28f-decc108cfaf5","Type":"ContainerStarted","Data":"8b8b89e69b6e62f5e498b0e85d93d3da0e0f96dfff97e1107b3f9edbb09597c7"}
Jul 28 23:36:34.398728 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:36:34.398322    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-99946b6f7-x5hp9"
Jul 28 23:36:34.414160 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:36:34.414102    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-99946b6f7-x5hp9" podStartSLOduration=1.016791124 podStartE2EDuration="1.414081731s" totalImagesPullingTime="397.290607ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 23:36:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 23:36:34.413520562 +0000 UTC m=+871.106195071" watchObservedRunningTime="2026-07-28 23:36:34.414081731 +0000 UTC m=+871.106756242"
Jul 28 23:36:45.405562 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:36:45.405529    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-99946b6f7-x5hp9"
Jul 28 23:51:01.802977 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:01.802944    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-cccd540e-openshift-default-7dc5c98ff4-qc7dc"]
Jul 28 23:51:01.806546 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:01.806529    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-cccd540e-openshift-default-7dc5c98ff4-qc7dc"
Jul 28 23:51:01.808848 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:01.808817    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-cccd540e-openshift-default-dockercfg-5wddc\""
Jul 28 23:51:01.829792 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:01.829762    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-cccd540e-openshift-default-7dc5c98ff4-qc7dc"]
Jul 28 23:51:01.926494 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:01.926458    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e5c4fac5-120e-460a-a55c-3af87f0304bb-istio-podinfo\") pod \"e2e-ait-cccd540e-openshift-default-7dc5c98ff4-qc7dc\" (UID: \"e5c4fac5-120e-460a-a55c-3af87f0304bb\") " pod="openshift-ingress/e2e-ait-cccd540e-openshift-default-7dc5c98ff4-qc7dc"
Jul 28 23:51:01.926678 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:01.926502    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e5c4fac5-120e-460a-a55c-3af87f0304bb-istiod-ca-cert\") pod \"e2e-ait-cccd540e-openshift-default-7dc5c98ff4-qc7dc\" (UID: \"e5c4fac5-120e-460a-a55c-3af87f0304bb\") " pod="openshift-ingress/e2e-ait-cccd540e-openshift-default-7dc5c98ff4-qc7dc"
Jul 28 23:51:01.926678 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:01.926536    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e5c4fac5-120e-460a-a55c-3af87f0304bb-credential-socket\") pod \"e2e-ait-cccd540e-openshift-default-7dc5c98ff4-qc7dc\" (UID: \"e5c4fac5-120e-460a-a55c-3af87f0304bb\") " pod="openshift-ingress/e2e-ait-cccd540e-openshift-default-7dc5c98ff4-qc7dc"
Jul 28 23:51:01.926678 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:01.926567    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d9mn2\" (UniqueName: \"kubernetes.io/projected/e5c4fac5-120e-460a-a55c-3af87f0304bb-kube-api-access-d9mn2\") pod \"e2e-ait-cccd540e-openshift-default-7dc5c98ff4-qc7dc\" (UID: \"e5c4fac5-120e-460a-a55c-3af87f0304bb\") " pod="openshift-ingress/e2e-ait-cccd540e-openshift-default-7dc5c98ff4-qc7dc"
Jul 28 23:51:01.926678 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:01.926598    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e5c4fac5-120e-460a-a55c-3af87f0304bb-istio-envoy\") pod \"e2e-ait-cccd540e-openshift-default-7dc5c98ff4-qc7dc\" (UID: \"e5c4fac5-120e-460a-a55c-3af87f0304bb\") " pod="openshift-ingress/e2e-ait-cccd540e-openshift-default-7dc5c98ff4-qc7dc"
Jul 28 23:51:01.926678 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:01.926660    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e5c4fac5-120e-460a-a55c-3af87f0304bb-workload-socket\") pod \"e2e-ait-cccd540e-openshift-default-7dc5c98ff4-qc7dc\" (UID: \"e5c4fac5-120e-460a-a55c-3af87f0304bb\") " pod="openshift-ingress/e2e-ait-cccd540e-openshift-default-7dc5c98ff4-qc7dc"
Jul 28 23:51:01.926854 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:01.926695    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e5c4fac5-120e-460a-a55c-3af87f0304bb-istio-token\") pod \"e2e-ait-cccd540e-openshift-default-7dc5c98ff4-qc7dc\" (UID: \"e5c4fac5-120e-460a-a55c-3af87f0304bb\") " pod="openshift-ingress/e2e-ait-cccd540e-openshift-default-7dc5c98ff4-qc7dc"
Jul 28 23:51:01.926854 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:01.926735    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e5c4fac5-120e-460a-a55c-3af87f0304bb-istio-data\") pod \"e2e-ait-cccd540e-openshift-default-7dc5c98ff4-qc7dc\" (UID: \"e5c4fac5-120e-460a-a55c-3af87f0304bb\") " pod="openshift-ingress/e2e-ait-cccd540e-openshift-default-7dc5c98ff4-qc7dc"
Jul 28 23:51:01.926854 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:01.926782    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e5c4fac5-120e-460a-a55c-3af87f0304bb-workload-certs\") pod \"e2e-ait-cccd540e-openshift-default-7dc5c98ff4-qc7dc\" (UID: \"e5c4fac5-120e-460a-a55c-3af87f0304bb\") " pod="openshift-ingress/e2e-ait-cccd540e-openshift-default-7dc5c98ff4-qc7dc"
Jul 28 23:51:02.027553 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:02.027516    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e5c4fac5-120e-460a-a55c-3af87f0304bb-istio-podinfo\") pod \"e2e-ait-cccd540e-openshift-default-7dc5c98ff4-qc7dc\" (UID: \"e5c4fac5-120e-460a-a55c-3af87f0304bb\") " pod="openshift-ingress/e2e-ait-cccd540e-openshift-default-7dc5c98ff4-qc7dc"
Jul 28 23:51:02.027553 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:02.027557    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e5c4fac5-120e-460a-a55c-3af87f0304bb-istiod-ca-cert\") pod \"e2e-ait-cccd540e-openshift-default-7dc5c98ff4-qc7dc\" (UID: \"e5c4fac5-120e-460a-a55c-3af87f0304bb\") " pod="openshift-ingress/e2e-ait-cccd540e-openshift-default-7dc5c98ff4-qc7dc"
Jul 28 23:51:02.027835 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:02.027586    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e5c4fac5-120e-460a-a55c-3af87f0304bb-credential-socket\") pod \"e2e-ait-cccd540e-openshift-default-7dc5c98ff4-qc7dc\" (UID: \"e5c4fac5-120e-460a-a55c-3af87f0304bb\") " pod="openshift-ingress/e2e-ait-cccd540e-openshift-default-7dc5c98ff4-qc7dc"
Jul 28 23:51:02.027835 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:02.027610    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d9mn2\" (UniqueName: \"kubernetes.io/projected/e5c4fac5-120e-460a-a55c-3af87f0304bb-kube-api-access-d9mn2\") pod \"e2e-ait-cccd540e-openshift-default-7dc5c98ff4-qc7dc\" (UID: \"e5c4fac5-120e-460a-a55c-3af87f0304bb\") " pod="openshift-ingress/e2e-ait-cccd540e-openshift-default-7dc5c98ff4-qc7dc"
Jul 28 23:51:02.027835 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:02.027691    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e5c4fac5-120e-460a-a55c-3af87f0304bb-istio-envoy\") pod \"e2e-ait-cccd540e-openshift-default-7dc5c98ff4-qc7dc\" (UID: \"e5c4fac5-120e-460a-a55c-3af87f0304bb\") " pod="openshift-ingress/e2e-ait-cccd540e-openshift-default-7dc5c98ff4-qc7dc"
Jul 28 23:51:02.027835 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:02.027747    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e5c4fac5-120e-460a-a55c-3af87f0304bb-workload-socket\") pod \"e2e-ait-cccd540e-openshift-default-7dc5c98ff4-qc7dc\" (UID: \"e5c4fac5-120e-460a-a55c-3af87f0304bb\") " pod="openshift-ingress/e2e-ait-cccd540e-openshift-default-7dc5c98ff4-qc7dc"
Jul 28 23:51:02.027835 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:02.027768    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e5c4fac5-120e-460a-a55c-3af87f0304bb-istio-token\") pod \"e2e-ait-cccd540e-openshift-default-7dc5c98ff4-qc7dc\" (UID: \"e5c4fac5-120e-460a-a55c-3af87f0304bb\") " pod="openshift-ingress/e2e-ait-cccd540e-openshift-default-7dc5c98ff4-qc7dc"
Jul 28 23:51:02.027835 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:02.027800    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e5c4fac5-120e-460a-a55c-3af87f0304bb-istio-data\") pod \"e2e-ait-cccd540e-openshift-default-7dc5c98ff4-qc7dc\" (UID: \"e5c4fac5-120e-460a-a55c-3af87f0304bb\") " pod="openshift-ingress/e2e-ait-cccd540e-openshift-default-7dc5c98ff4-qc7dc"
Jul 28 23:51:02.028138 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:02.027890    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e5c4fac5-120e-460a-a55c-3af87f0304bb-workload-certs\") pod \"e2e-ait-cccd540e-openshift-default-7dc5c98ff4-qc7dc\" (UID: \"e5c4fac5-120e-460a-a55c-3af87f0304bb\") " pod="openshift-ingress/e2e-ait-cccd540e-openshift-default-7dc5c98ff4-qc7dc"
Jul 28 23:51:02.028227 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:02.028196    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e5c4fac5-120e-460a-a55c-3af87f0304bb-credential-socket\") pod \"e2e-ait-cccd540e-openshift-default-7dc5c98ff4-qc7dc\" (UID: \"e5c4fac5-120e-460a-a55c-3af87f0304bb\") " pod="openshift-ingress/e2e-ait-cccd540e-openshift-default-7dc5c98ff4-qc7dc"
Jul 28 23:51:02.028372 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:02.028352    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e5c4fac5-120e-460a-a55c-3af87f0304bb-istiod-ca-cert\") pod \"e2e-ait-cccd540e-openshift-default-7dc5c98ff4-qc7dc\" (UID: \"e5c4fac5-120e-460a-a55c-3af87f0304bb\") " pod="openshift-ingress/e2e-ait-cccd540e-openshift-default-7dc5c98ff4-qc7dc"
Jul 28 23:51:02.028425 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:02.028378    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e5c4fac5-120e-460a-a55c-3af87f0304bb-workload-socket\") pod \"e2e-ait-cccd540e-openshift-default-7dc5c98ff4-qc7dc\" (UID: \"e5c4fac5-120e-460a-a55c-3af87f0304bb\") " pod="openshift-ingress/e2e-ait-cccd540e-openshift-default-7dc5c98ff4-qc7dc"
Jul 28 23:51:02.028425 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:02.028407    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e5c4fac5-120e-460a-a55c-3af87f0304bb-workload-certs\") pod \"e2e-ait-cccd540e-openshift-default-7dc5c98ff4-qc7dc\" (UID: \"e5c4fac5-120e-460a-a55c-3af87f0304bb\") " pod="openshift-ingress/e2e-ait-cccd540e-openshift-default-7dc5c98ff4-qc7dc"
Jul 28 23:51:02.028609 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:02.028591    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e5c4fac5-120e-460a-a55c-3af87f0304bb-istio-data\") pod \"e2e-ait-cccd540e-openshift-default-7dc5c98ff4-qc7dc\" (UID: \"e5c4fac5-120e-460a-a55c-3af87f0304bb\") " pod="openshift-ingress/e2e-ait-cccd540e-openshift-default-7dc5c98ff4-qc7dc"
Jul 28 23:51:02.030353 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:02.030330    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e5c4fac5-120e-460a-a55c-3af87f0304bb-istio-envoy\") pod \"e2e-ait-cccd540e-openshift-default-7dc5c98ff4-qc7dc\" (UID: \"e5c4fac5-120e-460a-a55c-3af87f0304bb\") " pod="openshift-ingress/e2e-ait-cccd540e-openshift-default-7dc5c98ff4-qc7dc"
Jul 28 23:51:02.030489 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:02.030470    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e5c4fac5-120e-460a-a55c-3af87f0304bb-istio-podinfo\") pod \"e2e-ait-cccd540e-openshift-default-7dc5c98ff4-qc7dc\" (UID: \"e5c4fac5-120e-460a-a55c-3af87f0304bb\") " pod="openshift-ingress/e2e-ait-cccd540e-openshift-default-7dc5c98ff4-qc7dc"
Jul 28 23:51:02.034945 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:02.034919    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e5c4fac5-120e-460a-a55c-3af87f0304bb-istio-token\") pod \"e2e-ait-cccd540e-openshift-default-7dc5c98ff4-qc7dc\" (UID: \"e5c4fac5-120e-460a-a55c-3af87f0304bb\") " pod="openshift-ingress/e2e-ait-cccd540e-openshift-default-7dc5c98ff4-qc7dc"
Jul 28 23:51:02.035065 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:02.034998    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d9mn2\" (UniqueName: \"kubernetes.io/projected/e5c4fac5-120e-460a-a55c-3af87f0304bb-kube-api-access-d9mn2\") pod \"e2e-ait-cccd540e-openshift-default-7dc5c98ff4-qc7dc\" (UID: \"e5c4fac5-120e-460a-a55c-3af87f0304bb\") " pod="openshift-ingress/e2e-ait-cccd540e-openshift-default-7dc5c98ff4-qc7dc"
Jul 28 23:51:02.117599 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:02.117517    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-cccd540e-openshift-default-7dc5c98ff4-qc7dc"
Jul 28 23:51:02.246667 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:02.246640    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-cccd540e-openshift-default-7dc5c98ff4-qc7dc"]
Jul 28 23:51:02.249350 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:51:02.249322    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5c4fac5_120e_460a_a55c_3af87f0304bb.slice/crio-7e5c31a4a63f643fb2dab3c028023777999b73a152b5a610ad90135ac6a3e6e6 WatchSource:0}: Error finding container 7e5c31a4a63f643fb2dab3c028023777999b73a152b5a610ad90135ac6a3e6e6: Status 404 returned error can't find the container with id 7e5c31a4a63f643fb2dab3c028023777999b73a152b5a610ad90135ac6a3e6e6
Jul 28 23:51:02.251899 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:02.251859    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 23:51:02.251993 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:02.251944    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 23:51:02.251993 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:02.251983    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 23:51:03.052874 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:03.052837    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-cccd540e-openshift-default-7dc5c98ff4-qc7dc" event={"ID":"e5c4fac5-120e-460a-a55c-3af87f0304bb","Type":"ContainerStarted","Data":"70755797a2818943a0e006cfa5400eab5c3ca37cc76d27018e23e92cda68ad8c"}
Jul 28 23:51:03.053349 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:03.052939    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-cccd540e-openshift-default-7dc5c98ff4-qc7dc" event={"ID":"e5c4fac5-120e-460a-a55c-3af87f0304bb","Type":"ContainerStarted","Data":"7e5c31a4a63f643fb2dab3c028023777999b73a152b5a610ad90135ac6a3e6e6"}
Jul 28 23:51:03.073597 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:03.073532    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-cccd540e-openshift-default-7dc5c98ff4-qc7dc" podStartSLOduration=2.073513716 podStartE2EDuration="2.073513716s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 23:51:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 23:51:03.071332806 +0000 UTC m=+1739.764007315" watchObservedRunningTime="2026-07-28 23:51:03.073513716 +0000 UTC m=+1739.766188225"
Jul 28 23:51:03.118663 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:03.118601    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-cccd540e-openshift-default-7dc5c98ff4-qc7dc"
Jul 28 23:51:03.123315 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:03.123288    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-cccd540e-openshift-default-7dc5c98ff4-qc7dc"
Jul 28 23:51:04.057404 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:04.057374    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-cccd540e-openshift-default-7dc5c98ff4-qc7dc"
Jul 28 23:51:04.058288 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:04.058268    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-cccd540e-openshift-default-7dc5c98ff4-qc7dc"
Jul 28 23:51:04.490857 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:04.490825    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-cccd540e-87788ccd8-57vc9"]
Jul 28 23:51:04.494260 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:04.494240    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-cccd540e-87788ccd8-57vc9"
Jul 28 23:51:04.496422 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:04.496404    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-cccd540e-serving-cert\""
Jul 28 23:51:04.504583 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:04.504558    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-cccd540e-87788ccd8-57vc9"]
Jul 28 23:51:04.654171 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:04.654132    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6mpj4\" (UniqueName: \"kubernetes.io/projected/ebeec626-f9e6-4a03-829d-17c29ed4b286-kube-api-access-6mpj4\") pod \"maas-api-e2e-ait-cccd540e-87788ccd8-57vc9\" (UID: \"ebeec626-f9e6-4a03-829d-17c29ed4b286\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-cccd540e-87788ccd8-57vc9"
Jul 28 23:51:04.654382 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:04.654180    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ebeec626-f9e6-4a03-829d-17c29ed4b286-openshift-service-ca\") pod \"maas-api-e2e-ait-cccd540e-87788ccd8-57vc9\" (UID: \"ebeec626-f9e6-4a03-829d-17c29ed4b286\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-cccd540e-87788ccd8-57vc9"
Jul 28 23:51:04.654382 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:04.654253    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ebeec626-f9e6-4a03-829d-17c29ed4b286-maas-api-tls\") pod \"maas-api-e2e-ait-cccd540e-87788ccd8-57vc9\" (UID: \"ebeec626-f9e6-4a03-829d-17c29ed4b286\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-cccd540e-87788ccd8-57vc9"
Jul 28 23:51:04.755687 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:04.755582    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6mpj4\" (UniqueName: \"kubernetes.io/projected/ebeec626-f9e6-4a03-829d-17c29ed4b286-kube-api-access-6mpj4\") pod \"maas-api-e2e-ait-cccd540e-87788ccd8-57vc9\" (UID: \"ebeec626-f9e6-4a03-829d-17c29ed4b286\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-cccd540e-87788ccd8-57vc9"
Jul 28 23:51:04.755687 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:04.755653    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ebeec626-f9e6-4a03-829d-17c29ed4b286-openshift-service-ca\") pod \"maas-api-e2e-ait-cccd540e-87788ccd8-57vc9\" (UID: \"ebeec626-f9e6-4a03-829d-17c29ed4b286\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-cccd540e-87788ccd8-57vc9"
Jul 28 23:51:04.755687 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:04.755673    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ebeec626-f9e6-4a03-829d-17c29ed4b286-maas-api-tls\") pod \"maas-api-e2e-ait-cccd540e-87788ccd8-57vc9\" (UID: \"ebeec626-f9e6-4a03-829d-17c29ed4b286\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-cccd540e-87788ccd8-57vc9"
Jul 28 23:51:04.755899 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:51:04.755790    2572 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-cccd540e-serving-cert: secret "maas-api-e2e-ait-cccd540e-serving-cert" not found
Jul 28 23:51:04.755899 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:51:04.755847    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ebeec626-f9e6-4a03-829d-17c29ed4b286-maas-api-tls podName:ebeec626-f9e6-4a03-829d-17c29ed4b286 nodeName:}" failed. No retries permitted until 2026-07-28 23:51:05.255828969 +0000 UTC m=+1741.948503458 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/ebeec626-f9e6-4a03-829d-17c29ed4b286-maas-api-tls") pod "maas-api-e2e-ait-cccd540e-87788ccd8-57vc9" (UID: "ebeec626-f9e6-4a03-829d-17c29ed4b286") : secret "maas-api-e2e-ait-cccd540e-serving-cert" not found
Jul 28 23:51:04.756281 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:04.756260    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ebeec626-f9e6-4a03-829d-17c29ed4b286-openshift-service-ca\") pod \"maas-api-e2e-ait-cccd540e-87788ccd8-57vc9\" (UID: \"ebeec626-f9e6-4a03-829d-17c29ed4b286\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-cccd540e-87788ccd8-57vc9"
Jul 28 23:51:04.763998 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:04.763969    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6mpj4\" (UniqueName: \"kubernetes.io/projected/ebeec626-f9e6-4a03-829d-17c29ed4b286-kube-api-access-6mpj4\") pod \"maas-api-e2e-ait-cccd540e-87788ccd8-57vc9\" (UID: \"ebeec626-f9e6-4a03-829d-17c29ed4b286\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-cccd540e-87788ccd8-57vc9"
Jul 28 23:51:05.260230 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:05.260194    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ebeec626-f9e6-4a03-829d-17c29ed4b286-maas-api-tls\") pod \"maas-api-e2e-ait-cccd540e-87788ccd8-57vc9\" (UID: \"ebeec626-f9e6-4a03-829d-17c29ed4b286\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-cccd540e-87788ccd8-57vc9"
Jul 28 23:51:05.262667 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:05.262642    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ebeec626-f9e6-4a03-829d-17c29ed4b286-maas-api-tls\") pod \"maas-api-e2e-ait-cccd540e-87788ccd8-57vc9\" (UID: \"ebeec626-f9e6-4a03-829d-17c29ed4b286\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-cccd540e-87788ccd8-57vc9"
Jul 28 23:51:05.406665 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:05.406609    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-cccd540e-87788ccd8-57vc9"
Jul 28 23:51:05.539585 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:05.539557    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-cccd540e-87788ccd8-57vc9"]
Jul 28 23:51:05.541598 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:51:05.541565    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podebeec626_f9e6_4a03_829d_17c29ed4b286.slice/crio-bc2c000ec0a75a8aa21e89a84a9d028e465df2f693fa5aa782ee489e86ccd032 WatchSource:0}: Error finding container bc2c000ec0a75a8aa21e89a84a9d028e465df2f693fa5aa782ee489e86ccd032: Status 404 returned error can't find the container with id bc2c000ec0a75a8aa21e89a84a9d028e465df2f693fa5aa782ee489e86ccd032
Jul 28 23:51:05.543797 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:05.543778    2572 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 23:51:06.067374 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:06.067342    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-cccd540e-87788ccd8-57vc9" event={"ID":"ebeec626-f9e6-4a03-829d-17c29ed4b286","Type":"ContainerStarted","Data":"bc2c000ec0a75a8aa21e89a84a9d028e465df2f693fa5aa782ee489e86ccd032"}
Jul 28 23:51:07.073289 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:07.073252    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-cccd540e-87788ccd8-57vc9" event={"ID":"ebeec626-f9e6-4a03-829d-17c29ed4b286","Type":"ContainerStarted","Data":"8c69e54ad6263bcabe1043597a50fd9b30161cd4c703833801877832da846486"}
Jul 28 23:51:07.073712 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:07.073480    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-cccd540e-87788ccd8-57vc9"
Jul 28 23:51:07.090436 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:07.090389    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-cccd540e-87788ccd8-57vc9" podStartSLOduration=2.623622061 podStartE2EDuration="3.090373366s" totalImagesPullingTime="466.751305ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 23:51:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 23:51:07.088792914 +0000 UTC m=+1743.781467420" watchObservedRunningTime="2026-07-28 23:51:07.090373366 +0000 UTC m=+1743.783047875"
Jul 28 23:51:13.082802 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:13.082769    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-cccd540e-87788ccd8-57vc9"
Jul 28 23:51:19.059314 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:19.059278    2572 patch_prober.go:28] interesting pod/e2e-ait-cccd540e-openshift-default-7dc5c98ff4-qc7dc container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused" start-of-body=
Jul 28 23:51:19.059777 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:19.059334    2572 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-cccd540e-openshift-default-7dc5c98ff4-qc7dc" podUID="e5c4fac5-120e-460a-a55c-3af87f0304bb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused"
Jul 28 23:51:19.129522 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:19.129482    2572 generic.go:356] "Generic (PLEG): container finished" podID="e5c4fac5-120e-460a-a55c-3af87f0304bb" containerID="70755797a2818943a0e006cfa5400eab5c3ca37cc76d27018e23e92cda68ad8c" exitCode=0
Jul 28 23:51:19.129729 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:19.129526    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-cccd540e-openshift-default-7dc5c98ff4-qc7dc" event={"ID":"e5c4fac5-120e-460a-a55c-3af87f0304bb","Type":"ContainerDied","Data":"70755797a2818943a0e006cfa5400eab5c3ca37cc76d27018e23e92cda68ad8c"}
Jul 28 23:51:19.129975 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:19.129957    2572 scope.go:117] "RemoveContainer" containerID="70755797a2818943a0e006cfa5400eab5c3ca37cc76d27018e23e92cda68ad8c"
Jul 28 23:51:19.130991 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:19.130963    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 23:51:19.131108 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:19.131032    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 23:51:19.131108 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:19.131063    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 23:51:20.136301 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:20.136261    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-cccd540e-openshift-default-7dc5c98ff4-qc7dc" event={"ID":"e5c4fac5-120e-460a-a55c-3af87f0304bb","Type":"ContainerStarted","Data":"c60afaf19d4f4542ab9ae79938de777150b54e96506066c612bd408329ce309f"}
Jul 28 23:51:21.117775 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:21.117734    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-cccd540e-openshift-default-7dc5c98ff4-qc7dc"
Jul 28 23:51:21.117937 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:21.117868    2572 patch_prober.go:28] interesting pod/e2e-ait-cccd540e-openshift-default-7dc5c98ff4-qc7dc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused" start-of-body=
Jul 28 23:51:21.117985 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:21.117928    2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-cccd540e-openshift-default-7dc5c98ff4-qc7dc" podUID="e5c4fac5-120e-460a-a55c-3af87f0304bb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused"
Jul 28 23:51:22.118636 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:22.118587    2572 patch_prober.go:28] interesting pod/e2e-ait-cccd540e-openshift-default-7dc5c98ff4-qc7dc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused" start-of-body=
Jul 28 23:51:22.119002 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:22.118657    2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-cccd540e-openshift-default-7dc5c98ff4-qc7dc" podUID="e5c4fac5-120e-460a-a55c-3af87f0304bb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused"
Jul 28 23:51:23.118507 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:23.118463    2572 patch_prober.go:28] interesting pod/e2e-ait-cccd540e-openshift-default-7dc5c98ff4-qc7dc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused" start-of-body=
Jul 28 23:51:23.118789 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:23.118527    2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-cccd540e-openshift-default-7dc5c98ff4-qc7dc" podUID="e5c4fac5-120e-460a-a55c-3af87f0304bb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused"
Jul 28 23:51:24.118985 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:24.118951    2572 patch_prober.go:28] interesting pod/e2e-ait-cccd540e-openshift-default-7dc5c98ff4-qc7dc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused" start-of-body=
Jul 28 23:51:24.119376 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:24.119004    2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-cccd540e-openshift-default-7dc5c98ff4-qc7dc" podUID="e5c4fac5-120e-460a-a55c-3af87f0304bb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused"
Jul 28 23:51:25.122363 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:25.122332    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-cccd540e-openshift-default-7dc5c98ff4-qc7dc"
Jul 28 23:51:25.122747 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:25.122573    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-cccd540e-openshift-default-7dc5c98ff4-qc7dc"
Jul 28 23:51:25.123252 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:25.123233    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-cccd540e-openshift-default-7dc5c98ff4-qc7dc"
Jul 28 23:51:35.180373 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:35.180335    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-cccd540e-87788ccd8-57vc9"]
Jul 28 23:51:35.180904 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:35.180590    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-cccd540e-87788ccd8-57vc9" podUID="ebeec626-f9e6-4a03-829d-17c29ed4b286" containerName="maas-api" containerID="cri-o://8c69e54ad6263bcabe1043597a50fd9b30161cd4c703833801877832da846486" gracePeriod=30
Jul 28 23:51:35.437284 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:35.437220    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-cccd540e-87788ccd8-57vc9"
Jul 28 23:51:35.479906 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:35.479871    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-cccd540e-openshift-default-7dc5c98ff4-qc7dc"]
Jul 28 23:51:35.480140 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:35.480115    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-cccd540e-openshift-default-7dc5c98ff4-qc7dc" podUID="e5c4fac5-120e-460a-a55c-3af87f0304bb" containerName="istio-proxy" containerID="cri-o://c60afaf19d4f4542ab9ae79938de777150b54e96506066c612bd408329ce309f" gracePeriod=30
Jul 28 23:51:35.527188 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:35.527158    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ebeec626-f9e6-4a03-829d-17c29ed4b286-openshift-service-ca\") pod \"ebeec626-f9e6-4a03-829d-17c29ed4b286\" (UID: \"ebeec626-f9e6-4a03-829d-17c29ed4b286\") "
Jul 28 23:51:35.527351 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:35.527228    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6mpj4\" (UniqueName: \"kubernetes.io/projected/ebeec626-f9e6-4a03-829d-17c29ed4b286-kube-api-access-6mpj4\") pod \"ebeec626-f9e6-4a03-829d-17c29ed4b286\" (UID: \"ebeec626-f9e6-4a03-829d-17c29ed4b286\") "
Jul 28 23:51:35.527351 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:35.527284    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ebeec626-f9e6-4a03-829d-17c29ed4b286-maas-api-tls\") pod \"ebeec626-f9e6-4a03-829d-17c29ed4b286\" (UID: \"ebeec626-f9e6-4a03-829d-17c29ed4b286\") "
Jul 28 23:51:35.527583 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:35.527558    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ebeec626-f9e6-4a03-829d-17c29ed4b286-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ebeec626-f9e6-4a03-829d-17c29ed4b286" (UID: "ebeec626-f9e6-4a03-829d-17c29ed4b286"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 23:51:35.529459 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:35.529429    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ebeec626-f9e6-4a03-829d-17c29ed4b286-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ebeec626-f9e6-4a03-829d-17c29ed4b286" (UID: "ebeec626-f9e6-4a03-829d-17c29ed4b286"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 23:51:35.529558 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:35.529480    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ebeec626-f9e6-4a03-829d-17c29ed4b286-kube-api-access-6mpj4" (OuterVolumeSpecName: "kube-api-access-6mpj4") pod "ebeec626-f9e6-4a03-829d-17c29ed4b286" (UID: "ebeec626-f9e6-4a03-829d-17c29ed4b286"). InnerVolumeSpecName "kube-api-access-6mpj4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 23:51:35.628088 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:35.628052    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ebeec626-f9e6-4a03-829d-17c29ed4b286-maas-api-tls\") on node \"ip-10-0-131-112.ec2.internal\" DevicePath \"\""
Jul 28 23:51:35.628088 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:35.628081    2572 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ebeec626-f9e6-4a03-829d-17c29ed4b286-openshift-service-ca\") on node \"ip-10-0-131-112.ec2.internal\" DevicePath \"\""
Jul 28 23:51:35.628088 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:35.628092    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6mpj4\" (UniqueName: \"kubernetes.io/projected/ebeec626-f9e6-4a03-829d-17c29ed4b286-kube-api-access-6mpj4\") on node \"ip-10-0-131-112.ec2.internal\" DevicePath \"\""
Jul 28 23:51:36.220598 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:36.220503    2572 generic.go:356] "Generic (PLEG): container finished" podID="ebeec626-f9e6-4a03-829d-17c29ed4b286" containerID="8c69e54ad6263bcabe1043597a50fd9b30161cd4c703833801877832da846486" exitCode=0
Jul 28 23:51:36.220598 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:36.220553    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-cccd540e-87788ccd8-57vc9"
Jul 28 23:51:36.220598 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:36.220574    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-cccd540e-87788ccd8-57vc9" event={"ID":"ebeec626-f9e6-4a03-829d-17c29ed4b286","Type":"ContainerDied","Data":"8c69e54ad6263bcabe1043597a50fd9b30161cd4c703833801877832da846486"}
Jul 28 23:51:36.221153 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:36.220616    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-cccd540e-87788ccd8-57vc9" event={"ID":"ebeec626-f9e6-4a03-829d-17c29ed4b286","Type":"ContainerDied","Data":"bc2c000ec0a75a8aa21e89a84a9d028e465df2f693fa5aa782ee489e86ccd032"}
Jul 28 23:51:36.221153 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:36.220654    2572 scope.go:117] "RemoveContainer" containerID="8c69e54ad6263bcabe1043597a50fd9b30161cd4c703833801877832da846486"
Jul 28 23:51:36.229115 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:36.229094    2572 scope.go:117] "RemoveContainer" containerID="8c69e54ad6263bcabe1043597a50fd9b30161cd4c703833801877832da846486"
Jul 28 23:51:36.229364 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:51:36.229344    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8c69e54ad6263bcabe1043597a50fd9b30161cd4c703833801877832da846486\": container with ID starting with 8c69e54ad6263bcabe1043597a50fd9b30161cd4c703833801877832da846486 not found: ID does not exist" containerID="8c69e54ad6263bcabe1043597a50fd9b30161cd4c703833801877832da846486"
Jul 28 23:51:36.229412 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:36.229372    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8c69e54ad6263bcabe1043597a50fd9b30161cd4c703833801877832da846486"} err="failed to get container status \"8c69e54ad6263bcabe1043597a50fd9b30161cd4c703833801877832da846486\": rpc error: code = NotFound desc = could not find container \"8c69e54ad6263bcabe1043597a50fd9b30161cd4c703833801877832da846486\": container with ID starting with 8c69e54ad6263bcabe1043597a50fd9b30161cd4c703833801877832da846486 not found: ID does not exist"
Jul 28 23:51:36.237649 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:36.237611    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-cccd540e-87788ccd8-57vc9"]
Jul 28 23:51:36.245863 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:36.245842    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-cccd540e-87788ccd8-57vc9"]
Jul 28 23:51:36.450098 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:36.450067    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-47ffb609-gw-openshift-default-588bbd686d-bg6xg"]
Jul 28 23:51:36.450388 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:36.450366    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ebeec626-f9e6-4a03-829d-17c29ed4b286" containerName="maas-api"
Jul 28 23:51:36.450388 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:36.450381    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="ebeec626-f9e6-4a03-829d-17c29ed4b286" containerName="maas-api"
Jul 28 23:51:36.450507 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:36.450427    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="ebeec626-f9e6-4a03-829d-17c29ed4b286" containerName="maas-api"
Jul 28 23:51:36.454680 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:36.454662    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-47ffb609-gw-openshift-default-588bbd686d-bg6xg"
Jul 28 23:51:36.457131 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:36.457109    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-47ffb609-gw-openshift-default-dockercfg-lgnpr\""
Jul 28 23:51:36.463700 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:36.463675    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-47ffb609-gw-openshift-default-588bbd686d-bg6xg"]
Jul 28 23:51:36.537642 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:36.537453    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qpfnk\" (UniqueName: \"kubernetes.io/projected/9c755016-49d5-4ba4-8ce9-6b457d8a9bcf-kube-api-access-qpfnk\") pod \"e2e-migrate-47ffb609-gw-openshift-default-588bbd686d-bg6xg\" (UID: \"9c755016-49d5-4ba4-8ce9-6b457d8a9bcf\") " pod="openshift-ingress/e2e-migrate-47ffb609-gw-openshift-default-588bbd686d-bg6xg"
Jul 28 23:51:36.537642 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:36.537503    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9c755016-49d5-4ba4-8ce9-6b457d8a9bcf-istio-token\") pod \"e2e-migrate-47ffb609-gw-openshift-default-588bbd686d-bg6xg\" (UID: \"9c755016-49d5-4ba4-8ce9-6b457d8a9bcf\") " pod="openshift-ingress/e2e-migrate-47ffb609-gw-openshift-default-588bbd686d-bg6xg"
Jul 28 23:51:36.537642 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:36.537532    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9c755016-49d5-4ba4-8ce9-6b457d8a9bcf-istio-podinfo\") pod \"e2e-migrate-47ffb609-gw-openshift-default-588bbd686d-bg6xg\" (UID: \"9c755016-49d5-4ba4-8ce9-6b457d8a9bcf\") " pod="openshift-ingress/e2e-migrate-47ffb609-gw-openshift-default-588bbd686d-bg6xg"
Jul 28 23:51:36.537642 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:36.537556    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9c755016-49d5-4ba4-8ce9-6b457d8a9bcf-istiod-ca-cert\") pod \"e2e-migrate-47ffb609-gw-openshift-default-588bbd686d-bg6xg\" (UID: \"9c755016-49d5-4ba4-8ce9-6b457d8a9bcf\") " pod="openshift-ingress/e2e-migrate-47ffb609-gw-openshift-default-588bbd686d-bg6xg"
Jul 28 23:51:36.537957 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:36.537653    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9c755016-49d5-4ba4-8ce9-6b457d8a9bcf-workload-certs\") pod \"e2e-migrate-47ffb609-gw-openshift-default-588bbd686d-bg6xg\" (UID: \"9c755016-49d5-4ba4-8ce9-6b457d8a9bcf\") " pod="openshift-ingress/e2e-migrate-47ffb609-gw-openshift-default-588bbd686d-bg6xg"
Jul 28 23:51:36.537957 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:36.537749    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9c755016-49d5-4ba4-8ce9-6b457d8a9bcf-istio-envoy\") pod \"e2e-migrate-47ffb609-gw-openshift-default-588bbd686d-bg6xg\" (UID: \"9c755016-49d5-4ba4-8ce9-6b457d8a9bcf\") " pod="openshift-ingress/e2e-migrate-47ffb609-gw-openshift-default-588bbd686d-bg6xg"
Jul 28 23:51:36.537957 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:36.537786    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9c755016-49d5-4ba4-8ce9-6b457d8a9bcf-credential-socket\") pod \"e2e-migrate-47ffb609-gw-openshift-default-588bbd686d-bg6xg\" (UID: \"9c755016-49d5-4ba4-8ce9-6b457d8a9bcf\") " pod="openshift-ingress/e2e-migrate-47ffb609-gw-openshift-default-588bbd686d-bg6xg"
Jul 28 23:51:36.537957 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:36.537833    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9c755016-49d5-4ba4-8ce9-6b457d8a9bcf-workload-socket\") pod \"e2e-migrate-47ffb609-gw-openshift-default-588bbd686d-bg6xg\" (UID: \"9c755016-49d5-4ba4-8ce9-6b457d8a9bcf\") " pod="openshift-ingress/e2e-migrate-47ffb609-gw-openshift-default-588bbd686d-bg6xg"
Jul 28 23:51:36.537957 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:36.537881    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9c755016-49d5-4ba4-8ce9-6b457d8a9bcf-istio-data\") pod \"e2e-migrate-47ffb609-gw-openshift-default-588bbd686d-bg6xg\" (UID: \"9c755016-49d5-4ba4-8ce9-6b457d8a9bcf\") " pod="openshift-ingress/e2e-migrate-47ffb609-gw-openshift-default-588bbd686d-bg6xg"
Jul 28 23:51:36.638874 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:36.638835    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9c755016-49d5-4ba4-8ce9-6b457d8a9bcf-istio-data\") pod \"e2e-migrate-47ffb609-gw-openshift-default-588bbd686d-bg6xg\" (UID: \"9c755016-49d5-4ba4-8ce9-6b457d8a9bcf\") " pod="openshift-ingress/e2e-migrate-47ffb609-gw-openshift-default-588bbd686d-bg6xg"
Jul 28 23:51:36.639041 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:36.638887    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qpfnk\" (UniqueName: \"kubernetes.io/projected/9c755016-49d5-4ba4-8ce9-6b457d8a9bcf-kube-api-access-qpfnk\") pod \"e2e-migrate-47ffb609-gw-openshift-default-588bbd686d-bg6xg\" (UID: \"9c755016-49d5-4ba4-8ce9-6b457d8a9bcf\") " pod="openshift-ingress/e2e-migrate-47ffb609-gw-openshift-default-588bbd686d-bg6xg"
Jul 28 23:51:36.639041 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:36.639018    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9c755016-49d5-4ba4-8ce9-6b457d8a9bcf-istio-token\") pod \"e2e-migrate-47ffb609-gw-openshift-default-588bbd686d-bg6xg\" (UID: \"9c755016-49d5-4ba4-8ce9-6b457d8a9bcf\") " pod="openshift-ingress/e2e-migrate-47ffb609-gw-openshift-default-588bbd686d-bg6xg"
Jul 28 23:51:36.639136 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:36.639067    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9c755016-49d5-4ba4-8ce9-6b457d8a9bcf-istio-podinfo\") pod \"e2e-migrate-47ffb609-gw-openshift-default-588bbd686d-bg6xg\" (UID: \"9c755016-49d5-4ba4-8ce9-6b457d8a9bcf\") " pod="openshift-ingress/e2e-migrate-47ffb609-gw-openshift-default-588bbd686d-bg6xg"
Jul 28 23:51:36.639136 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:36.639097    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9c755016-49d5-4ba4-8ce9-6b457d8a9bcf-istiod-ca-cert\") pod \"e2e-migrate-47ffb609-gw-openshift-default-588bbd686d-bg6xg\" (UID: \"9c755016-49d5-4ba4-8ce9-6b457d8a9bcf\") " pod="openshift-ingress/e2e-migrate-47ffb609-gw-openshift-default-588bbd686d-bg6xg"
Jul 28 23:51:36.639136 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:36.639128    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9c755016-49d5-4ba4-8ce9-6b457d8a9bcf-workload-certs\") pod \"e2e-migrate-47ffb609-gw-openshift-default-588bbd686d-bg6xg\" (UID: \"9c755016-49d5-4ba4-8ce9-6b457d8a9bcf\") " pod="openshift-ingress/e2e-migrate-47ffb609-gw-openshift-default-588bbd686d-bg6xg"
Jul 28 23:51:36.639281 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:36.639218    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9c755016-49d5-4ba4-8ce9-6b457d8a9bcf-istio-envoy\") pod \"e2e-migrate-47ffb609-gw-openshift-default-588bbd686d-bg6xg\" (UID: \"9c755016-49d5-4ba4-8ce9-6b457d8a9bcf\") " pod="openshift-ingress/e2e-migrate-47ffb609-gw-openshift-default-588bbd686d-bg6xg"
Jul 28 23:51:36.639281 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:36.639249    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9c755016-49d5-4ba4-8ce9-6b457d8a9bcf-credential-socket\") pod \"e2e-migrate-47ffb609-gw-openshift-default-588bbd686d-bg6xg\" (UID: \"9c755016-49d5-4ba4-8ce9-6b457d8a9bcf\") " pod="openshift-ingress/e2e-migrate-47ffb609-gw-openshift-default-588bbd686d-bg6xg"
Jul 28 23:51:36.639281 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:36.639271    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9c755016-49d5-4ba4-8ce9-6b457d8a9bcf-istio-data\") pod \"e2e-migrate-47ffb609-gw-openshift-default-588bbd686d-bg6xg\" (UID: \"9c755016-49d5-4ba4-8ce9-6b457d8a9bcf\") " pod="openshift-ingress/e2e-migrate-47ffb609-gw-openshift-default-588bbd686d-bg6xg"
Jul 28 23:51:36.639431 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:36.639295    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9c755016-49d5-4ba4-8ce9-6b457d8a9bcf-workload-socket\") pod \"e2e-migrate-47ffb609-gw-openshift-default-588bbd686d-bg6xg\" (UID: \"9c755016-49d5-4ba4-8ce9-6b457d8a9bcf\") " pod="openshift-ingress/e2e-migrate-47ffb609-gw-openshift-default-588bbd686d-bg6xg"
Jul 28 23:51:36.639556 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:36.639530    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9c755016-49d5-4ba4-8ce9-6b457d8a9bcf-credential-socket\") pod \"e2e-migrate-47ffb609-gw-openshift-default-588bbd686d-bg6xg\" (UID: \"9c755016-49d5-4ba4-8ce9-6b457d8a9bcf\") " pod="openshift-ingress/e2e-migrate-47ffb609-gw-openshift-default-588bbd686d-bg6xg"
Jul 28 23:51:36.639714 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:36.639660    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9c755016-49d5-4ba4-8ce9-6b457d8a9bcf-workload-socket\") pod \"e2e-migrate-47ffb609-gw-openshift-default-588bbd686d-bg6xg\" (UID: \"9c755016-49d5-4ba4-8ce9-6b457d8a9bcf\") " pod="openshift-ingress/e2e-migrate-47ffb609-gw-openshift-default-588bbd686d-bg6xg"
Jul 28 23:51:36.639854 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:36.639832    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9c755016-49d5-4ba4-8ce9-6b457d8a9bcf-workload-certs\") pod \"e2e-migrate-47ffb609-gw-openshift-default-588bbd686d-bg6xg\" (UID: \"9c755016-49d5-4ba4-8ce9-6b457d8a9bcf\") " pod="openshift-ingress/e2e-migrate-47ffb609-gw-openshift-default-588bbd686d-bg6xg"
Jul 28 23:51:36.639930 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:36.639885    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9c755016-49d5-4ba4-8ce9-6b457d8a9bcf-istiod-ca-cert\") pod \"e2e-migrate-47ffb609-gw-openshift-default-588bbd686d-bg6xg\" (UID: \"9c755016-49d5-4ba4-8ce9-6b457d8a9bcf\") " pod="openshift-ingress/e2e-migrate-47ffb609-gw-openshift-default-588bbd686d-bg6xg"
Jul 28 23:51:36.641480 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:36.641458    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9c755016-49d5-4ba4-8ce9-6b457d8a9bcf-istio-envoy\") pod \"e2e-migrate-47ffb609-gw-openshift-default-588bbd686d-bg6xg\" (UID: \"9c755016-49d5-4ba4-8ce9-6b457d8a9bcf\") " pod="openshift-ingress/e2e-migrate-47ffb609-gw-openshift-default-588bbd686d-bg6xg"
Jul 28 23:51:36.641643 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:36.641609    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9c755016-49d5-4ba4-8ce9-6b457d8a9bcf-istio-podinfo\") pod \"e2e-migrate-47ffb609-gw-openshift-default-588bbd686d-bg6xg\" (UID: \"9c755016-49d5-4ba4-8ce9-6b457d8a9bcf\") " pod="openshift-ingress/e2e-migrate-47ffb609-gw-openshift-default-588bbd686d-bg6xg"
Jul 28 23:51:36.653008 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:36.652978    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qpfnk\" (UniqueName: \"kubernetes.io/projected/9c755016-49d5-4ba4-8ce9-6b457d8a9bcf-kube-api-access-qpfnk\") pod \"e2e-migrate-47ffb609-gw-openshift-default-588bbd686d-bg6xg\" (UID: \"9c755016-49d5-4ba4-8ce9-6b457d8a9bcf\") " pod="openshift-ingress/e2e-migrate-47ffb609-gw-openshift-default-588bbd686d-bg6xg"
Jul 28 23:51:36.656205 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:36.656181    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9c755016-49d5-4ba4-8ce9-6b457d8a9bcf-istio-token\") pod \"e2e-migrate-47ffb609-gw-openshift-default-588bbd686d-bg6xg\" (UID: \"9c755016-49d5-4ba4-8ce9-6b457d8a9bcf\") " pod="openshift-ingress/e2e-migrate-47ffb609-gw-openshift-default-588bbd686d-bg6xg"
Jul 28 23:51:36.765930 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:36.765889    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-47ffb609-gw-openshift-default-588bbd686d-bg6xg"
Jul 28 23:51:36.914393 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:36.914372    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-47ffb609-gw-openshift-default-588bbd686d-bg6xg"]
Jul 28 23:51:36.916525 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:51:36.916498    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c755016_49d5_4ba4_8ce9_6b457d8a9bcf.slice/crio-5eba3c3e5b7234c0d935b25a4d5cc8d8b9c6244eae3a599591e0ce2671cbc875 WatchSource:0}: Error finding container 5eba3c3e5b7234c0d935b25a4d5cc8d8b9c6244eae3a599591e0ce2671cbc875: Status 404 returned error can't find the container with id 5eba3c3e5b7234c0d935b25a4d5cc8d8b9c6244eae3a599591e0ce2671cbc875
Jul 28 23:51:36.918759 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:36.918728    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 23:51:36.918842 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:36.918792    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 23:51:36.918842 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:36.918820    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 23:51:37.226955 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:37.226923    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-47ffb609-gw-openshift-default-588bbd686d-bg6xg" event={"ID":"9c755016-49d5-4ba4-8ce9-6b457d8a9bcf","Type":"ContainerStarted","Data":"cc1f389e0a0f0b5b9bd7b48d6b058d668d6108b519a42c68def4c735ce17fe7e"}
Jul 28 23:51:37.227388 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:37.226988    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-47ffb609-gw-openshift-default-588bbd686d-bg6xg" event={"ID":"9c755016-49d5-4ba4-8ce9-6b457d8a9bcf","Type":"ContainerStarted","Data":"5eba3c3e5b7234c0d935b25a4d5cc8d8b9c6244eae3a599591e0ce2671cbc875"}
Jul 28 23:51:37.254238 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:37.254183    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-47ffb609-gw-openshift-default-588bbd686d-bg6xg" podStartSLOduration=1.2541673 podStartE2EDuration="1.2541673s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 23:51:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 23:51:37.252041525 +0000 UTC m=+1773.944716036" watchObservedRunningTime="2026-07-28 23:51:37.2541673 +0000 UTC m=+1773.946841810"
Jul 28 23:51:37.766086 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:37.766050    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-47ffb609-gw-openshift-default-588bbd686d-bg6xg"
Jul 28 23:51:37.777005 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:37.776958    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-47ffb609-gw-openshift-default-588bbd686d-bg6xg"
Jul 28 23:51:37.865230 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:37.865194    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ebeec626-f9e6-4a03-829d-17c29ed4b286" path="/var/lib/kubelet/pods/ebeec626-f9e6-4a03-829d-17c29ed4b286/volumes"
Jul 28 23:51:38.231924 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:38.231893    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-47ffb609-gw-openshift-default-588bbd686d-bg6xg"
Jul 28 23:51:38.232780 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:38.232759    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-47ffb609-gw-openshift-default-588bbd686d-bg6xg"
Jul 28 23:51:40.125266 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:40.125228    2572 patch_prober.go:28] interesting pod/e2e-ait-cccd540e-openshift-default-7dc5c98ff4-qc7dc container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 28 23:51:40.125669 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:40.125285    2572 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-cccd540e-openshift-default-7dc5c98ff4-qc7dc" podUID="e5c4fac5-120e-460a-a55c-3af87f0304bb" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 23:51:40.739215 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:40.739156    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-cccd540e-openshift-default-7dc5c98ff4-qc7dc"
Jul 28 23:51:40.876042 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:40.876008    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e5c4fac5-120e-460a-a55c-3af87f0304bb-credential-socket\") pod \"e5c4fac5-120e-460a-a55c-3af87f0304bb\" (UID: \"e5c4fac5-120e-460a-a55c-3af87f0304bb\") "
Jul 28 23:51:40.876188 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:40.876066    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e5c4fac5-120e-460a-a55c-3af87f0304bb-workload-socket\") pod \"e5c4fac5-120e-460a-a55c-3af87f0304bb\" (UID: \"e5c4fac5-120e-460a-a55c-3af87f0304bb\") "
Jul 28 23:51:40.876188 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:40.876089    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e5c4fac5-120e-460a-a55c-3af87f0304bb-istio-token\") pod \"e5c4fac5-120e-460a-a55c-3af87f0304bb\" (UID: \"e5c4fac5-120e-460a-a55c-3af87f0304bb\") "
Jul 28 23:51:40.876188 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:40.876128    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e5c4fac5-120e-460a-a55c-3af87f0304bb-istio-podinfo\") pod \"e5c4fac5-120e-460a-a55c-3af87f0304bb\" (UID: \"e5c4fac5-120e-460a-a55c-3af87f0304bb\") "
Jul 28 23:51:40.876188 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:40.876151    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e5c4fac5-120e-460a-a55c-3af87f0304bb-istiod-ca-cert\") pod \"e5c4fac5-120e-460a-a55c-3af87f0304bb\" (UID: \"e5c4fac5-120e-460a-a55c-3af87f0304bb\") "
Jul 28 23:51:40.876188 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:40.876177    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e5c4fac5-120e-460a-a55c-3af87f0304bb-workload-certs\") pod \"e5c4fac5-120e-460a-a55c-3af87f0304bb\" (UID: \"e5c4fac5-120e-460a-a55c-3af87f0304bb\") "
Jul 28 23:51:40.876447 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:40.876268    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e5c4fac5-120e-460a-a55c-3af87f0304bb-istio-data\") pod \"e5c4fac5-120e-460a-a55c-3af87f0304bb\" (UID: \"e5c4fac5-120e-460a-a55c-3af87f0304bb\") "
Jul 28 23:51:40.876447 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:40.876317    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e5c4fac5-120e-460a-a55c-3af87f0304bb-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e5c4fac5-120e-460a-a55c-3af87f0304bb" (UID: "e5c4fac5-120e-460a-a55c-3af87f0304bb"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 23:51:40.876447 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:40.876355    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e5c4fac5-120e-460a-a55c-3af87f0304bb-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e5c4fac5-120e-460a-a55c-3af87f0304bb" (UID: "e5c4fac5-120e-460a-a55c-3af87f0304bb"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 23:51:40.876447 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:40.876367    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d9mn2\" (UniqueName: \"kubernetes.io/projected/e5c4fac5-120e-460a-a55c-3af87f0304bb-kube-api-access-d9mn2\") pod \"e5c4fac5-120e-460a-a55c-3af87f0304bb\" (UID: \"e5c4fac5-120e-460a-a55c-3af87f0304bb\") "
Jul 28 23:51:40.876447 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:40.876405    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e5c4fac5-120e-460a-a55c-3af87f0304bb-istio-envoy\") pod \"e5c4fac5-120e-460a-a55c-3af87f0304bb\" (UID: \"e5c4fac5-120e-460a-a55c-3af87f0304bb\") "
Jul 28 23:51:40.876447 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:40.876445    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e5c4fac5-120e-460a-a55c-3af87f0304bb-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e5c4fac5-120e-460a-a55c-3af87f0304bb" (UID: "e5c4fac5-120e-460a-a55c-3af87f0304bb"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 23:51:40.876779 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:40.876590    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e5c4fac5-120e-460a-a55c-3af87f0304bb-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e5c4fac5-120e-460a-a55c-3af87f0304bb" (UID: "e5c4fac5-120e-460a-a55c-3af87f0304bb"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 23:51:40.876872 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:40.876809    2572 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e5c4fac5-120e-460a-a55c-3af87f0304bb-credential-socket\") on node \"ip-10-0-131-112.ec2.internal\" DevicePath \"\""
Jul 28 23:51:40.876872 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:40.876833    2572 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e5c4fac5-120e-460a-a55c-3af87f0304bb-workload-socket\") on node \"ip-10-0-131-112.ec2.internal\" DevicePath \"\""
Jul 28 23:51:40.876872 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:40.876849    2572 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e5c4fac5-120e-460a-a55c-3af87f0304bb-istiod-ca-cert\") on node \"ip-10-0-131-112.ec2.internal\" DevicePath \"\""
Jul 28 23:51:40.876872 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:40.876862    2572 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e5c4fac5-120e-460a-a55c-3af87f0304bb-workload-certs\") on node \"ip-10-0-131-112.ec2.internal\" DevicePath \"\""
Jul 28 23:51:40.877504 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:40.877476    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e5c4fac5-120e-460a-a55c-3af87f0304bb-istio-data" (OuterVolumeSpecName: "istio-data") pod "e5c4fac5-120e-460a-a55c-3af87f0304bb" (UID: "e5c4fac5-120e-460a-a55c-3af87f0304bb"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 23:51:40.878586 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:40.878565    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e5c4fac5-120e-460a-a55c-3af87f0304bb-istio-token" (OuterVolumeSpecName: "istio-token") pod "e5c4fac5-120e-460a-a55c-3af87f0304bb" (UID: "e5c4fac5-120e-460a-a55c-3af87f0304bb"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 23:51:40.878690 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:40.878563    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e5c4fac5-120e-460a-a55c-3af87f0304bb-kube-api-access-d9mn2" (OuterVolumeSpecName: "kube-api-access-d9mn2") pod "e5c4fac5-120e-460a-a55c-3af87f0304bb" (UID: "e5c4fac5-120e-460a-a55c-3af87f0304bb"). InnerVolumeSpecName "kube-api-access-d9mn2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 23:51:40.878745 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:40.878713    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e5c4fac5-120e-460a-a55c-3af87f0304bb-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e5c4fac5-120e-460a-a55c-3af87f0304bb" (UID: "e5c4fac5-120e-460a-a55c-3af87f0304bb"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 28 23:51:40.878745 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:40.878731    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e5c4fac5-120e-460a-a55c-3af87f0304bb-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e5c4fac5-120e-460a-a55c-3af87f0304bb" (UID: "e5c4fac5-120e-460a-a55c-3af87f0304bb"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 23:51:40.977533 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:40.977480    2572 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e5c4fac5-120e-460a-a55c-3af87f0304bb-istio-token\") on node \"ip-10-0-131-112.ec2.internal\" DevicePath \"\""
Jul 28 23:51:40.977533 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:40.977525    2572 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e5c4fac5-120e-460a-a55c-3af87f0304bb-istio-podinfo\") on node \"ip-10-0-131-112.ec2.internal\" DevicePath \"\""
Jul 28 23:51:40.977533 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:40.977535    2572 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e5c4fac5-120e-460a-a55c-3af87f0304bb-istio-data\") on node \"ip-10-0-131-112.ec2.internal\" DevicePath \"\""
Jul 28 23:51:40.977533 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:40.977545    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d9mn2\" (UniqueName: \"kubernetes.io/projected/e5c4fac5-120e-460a-a55c-3af87f0304bb-kube-api-access-d9mn2\") on node \"ip-10-0-131-112.ec2.internal\" DevicePath \"\""
Jul 28 23:51:40.977533 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:40.977555    2572 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e5c4fac5-120e-460a-a55c-3af87f0304bb-istio-envoy\") on node \"ip-10-0-131-112.ec2.internal\" DevicePath \"\""
Jul 28 23:51:41.246694 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:41.246658    2572 generic.go:356] "Generic (PLEG): container finished" podID="e5c4fac5-120e-460a-a55c-3af87f0304bb" containerID="c60afaf19d4f4542ab9ae79938de777150b54e96506066c612bd408329ce309f" exitCode=0
Jul 28 23:51:41.247104 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:41.246726    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-cccd540e-openshift-default-7dc5c98ff4-qc7dc" event={"ID":"e5c4fac5-120e-460a-a55c-3af87f0304bb","Type":"ContainerDied","Data":"c60afaf19d4f4542ab9ae79938de777150b54e96506066c612bd408329ce309f"}
Jul 28 23:51:41.247104 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:41.246760    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-cccd540e-openshift-default-7dc5c98ff4-qc7dc"
Jul 28 23:51:41.247104 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:41.246778    2572 scope.go:117] "RemoveContainer" containerID="c60afaf19d4f4542ab9ae79938de777150b54e96506066c612bd408329ce309f"
Jul 28 23:51:41.247104 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:41.246767    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-cccd540e-openshift-default-7dc5c98ff4-qc7dc" event={"ID":"e5c4fac5-120e-460a-a55c-3af87f0304bb","Type":"ContainerDied","Data":"7e5c31a4a63f643fb2dab3c028023777999b73a152b5a610ad90135ac6a3e6e6"}
Jul 28 23:51:41.255418 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:41.255402    2572 scope.go:117] "RemoveContainer" containerID="70755797a2818943a0e006cfa5400eab5c3ca37cc76d27018e23e92cda68ad8c"
Jul 28 23:51:41.262980 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:41.262961    2572 scope.go:117] "RemoveContainer" containerID="c60afaf19d4f4542ab9ae79938de777150b54e96506066c612bd408329ce309f"
Jul 28 23:51:41.263285 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:51:41.263266    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c60afaf19d4f4542ab9ae79938de777150b54e96506066c612bd408329ce309f\": container with ID starting with c60afaf19d4f4542ab9ae79938de777150b54e96506066c612bd408329ce309f not found: ID does not exist" containerID="c60afaf19d4f4542ab9ae79938de777150b54e96506066c612bd408329ce309f"
Jul 28 23:51:41.263334 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:41.263293    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c60afaf19d4f4542ab9ae79938de777150b54e96506066c612bd408329ce309f"} err="failed to get container status \"c60afaf19d4f4542ab9ae79938de777150b54e96506066c612bd408329ce309f\": rpc error: code = NotFound desc = could not find container \"c60afaf19d4f4542ab9ae79938de777150b54e96506066c612bd408329ce309f\": container with ID starting with c60afaf19d4f4542ab9ae79938de777150b54e96506066c612bd408329ce309f not found: ID does not exist"
Jul 28 23:51:41.263334 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:41.263311    2572 scope.go:117] "RemoveContainer" containerID="70755797a2818943a0e006cfa5400eab5c3ca37cc76d27018e23e92cda68ad8c"
Jul 28 23:51:41.263536 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:51:41.263521    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"70755797a2818943a0e006cfa5400eab5c3ca37cc76d27018e23e92cda68ad8c\": container with ID starting with 70755797a2818943a0e006cfa5400eab5c3ca37cc76d27018e23e92cda68ad8c not found: ID does not exist" containerID="70755797a2818943a0e006cfa5400eab5c3ca37cc76d27018e23e92cda68ad8c"
Jul 28 23:51:41.263585 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:41.263545    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"70755797a2818943a0e006cfa5400eab5c3ca37cc76d27018e23e92cda68ad8c"} err="failed to get container status \"70755797a2818943a0e006cfa5400eab5c3ca37cc76d27018e23e92cda68ad8c\": rpc error: code = NotFound desc = could not find container \"70755797a2818943a0e006cfa5400eab5c3ca37cc76d27018e23e92cda68ad8c\": container with ID starting with 70755797a2818943a0e006cfa5400eab5c3ca37cc76d27018e23e92cda68ad8c not found: ID does not exist"
Jul 28 23:51:41.270641 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:41.270602    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-cccd540e-openshift-default-7dc5c98ff4-qc7dc"]
Jul 28 23:51:41.275663 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:41.275637    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-cccd540e-openshift-default-7dc5c98ff4-qc7dc"]
Jul 28 23:51:41.873908 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:41.873873    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e5c4fac5-120e-460a-a55c-3af87f0304bb" path="/var/lib/kubelet/pods/e5c4fac5-120e-460a-a55c-3af87f0304bb/volumes"
Jul 28 23:51:53.233863 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:53.233821    2572 patch_prober.go:28] interesting pod/e2e-migrate-47ffb609-gw-openshift-default-588bbd686d-bg6xg container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" start-of-body=
Jul 28 23:51:53.234332 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:53.233886    2572 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-migrate-47ffb609-gw-openshift-default-588bbd686d-bg6xg" podUID="9c755016-49d5-4ba4-8ce9-6b457d8a9bcf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused"
Jul 28 23:51:53.298656 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:53.298558    2572 generic.go:356] "Generic (PLEG): container finished" podID="9c755016-49d5-4ba4-8ce9-6b457d8a9bcf" containerID="cc1f389e0a0f0b5b9bd7b48d6b058d668d6108b519a42c68def4c735ce17fe7e" exitCode=0
Jul 28 23:51:53.298656 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:53.298638    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-47ffb609-gw-openshift-default-588bbd686d-bg6xg" event={"ID":"9c755016-49d5-4ba4-8ce9-6b457d8a9bcf","Type":"ContainerDied","Data":"cc1f389e0a0f0b5b9bd7b48d6b058d668d6108b519a42c68def4c735ce17fe7e"}
Jul 28 23:51:53.299068 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:53.299050    2572 scope.go:117] "RemoveContainer" containerID="cc1f389e0a0f0b5b9bd7b48d6b058d668d6108b519a42c68def4c735ce17fe7e"
Jul 28 23:51:53.300063 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:53.300036    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 23:51:53.300123 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:53.300103    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 23:51:53.300162 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:53.300135    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 23:51:54.305252 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:54.305214    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-47ffb609-gw-openshift-default-588bbd686d-bg6xg" event={"ID":"9c755016-49d5-4ba4-8ce9-6b457d8a9bcf","Type":"ContainerStarted","Data":"4c512b59a0fb178ebb69dbe374a51bc8de13ba3d741542d7df0164b415b27d9c"}
Jul 28 23:51:54.766698 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:54.766661    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-47ffb609-gw-openshift-default-588bbd686d-bg6xg"
Jul 28 23:51:54.766889 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:54.766857    2572 patch_prober.go:28] interesting pod/e2e-migrate-47ffb609-gw-openshift-default-588bbd686d-bg6xg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" start-of-body=
Jul 28 23:51:54.766939 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:54.766903    2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-47ffb609-gw-openshift-default-588bbd686d-bg6xg" podUID="9c755016-49d5-4ba4-8ce9-6b457d8a9bcf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused"
Jul 28 23:51:55.766468 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:55.766428    2572 patch_prober.go:28] interesting pod/e2e-migrate-47ffb609-gw-openshift-default-588bbd686d-bg6xg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" start-of-body=
Jul 28 23:51:55.766876 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:55.766487    2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-47ffb609-gw-openshift-default-588bbd686d-bg6xg" podUID="9c755016-49d5-4ba4-8ce9-6b457d8a9bcf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused"
Jul 28 23:51:56.766926 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:56.766888    2572 patch_prober.go:28] interesting pod/e2e-migrate-47ffb609-gw-openshift-default-588bbd686d-bg6xg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" start-of-body=
Jul 28 23:51:56.767346 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:56.766971    2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-47ffb609-gw-openshift-default-588bbd686d-bg6xg" podUID="9c755016-49d5-4ba4-8ce9-6b457d8a9bcf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused"
Jul 28 23:51:57.766874 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:57.766819    2572 patch_prober.go:28] interesting pod/e2e-migrate-47ffb609-gw-openshift-default-588bbd686d-bg6xg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" start-of-body=
Jul 28 23:51:57.767057 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:57.766892    2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-47ffb609-gw-openshift-default-588bbd686d-bg6xg" podUID="9c755016-49d5-4ba4-8ce9-6b457d8a9bcf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused"
Jul 28 23:51:58.770861 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:58.770831    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-47ffb609-gw-openshift-default-588bbd686d-bg6xg"
Jul 28 23:51:58.771224 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:58.771207    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-47ffb609-gw-openshift-default-588bbd686d-bg6xg"
Jul 28 23:51:58.771999 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:51:58.771982    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-47ffb609-gw-openshift-default-588bbd686d-bg6xg"
Jul 28 23:52:10.566285 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:10.566249    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-47ffb609-gw-openshift-default-588bbd686d-bg6xg"]
Jul 28 23:52:10.566749 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:10.566533    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-47ffb609-gw-openshift-default-588bbd686d-bg6xg" podUID="9c755016-49d5-4ba4-8ce9-6b457d8a9bcf" containerName="istio-proxy" containerID="cri-o://4c512b59a0fb178ebb69dbe374a51bc8de13ba3d741542d7df0164b415b27d9c" gracePeriod=30
Jul 28 23:52:13.773710 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:13.773667    2572 patch_prober.go:28] interesting pod/e2e-migrate-47ffb609-gw-openshift-default-588bbd686d-bg6xg container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 28 23:52:13.774094 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:13.773742    2572 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-migrate-47ffb609-gw-openshift-default-588bbd686d-bg6xg" podUID="9c755016-49d5-4ba4-8ce9-6b457d8a9bcf" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 23:52:14.146739 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:14.146652    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-2aaf48a6-66b75f75f4-s5zgs"]
Jul 28 23:52:14.147053 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:14.147034    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e5c4fac5-120e-460a-a55c-3af87f0304bb" containerName="istio-proxy"
Jul 28 23:52:14.147147 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:14.147056    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="e5c4fac5-120e-460a-a55c-3af87f0304bb" containerName="istio-proxy"
Jul 28 23:52:14.147204 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:14.147170    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="e5c4fac5-120e-460a-a55c-3af87f0304bb" containerName="istio-proxy"
Jul 28 23:52:14.147204 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:14.147185    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="e5c4fac5-120e-460a-a55c-3af87f0304bb" containerName="istio-proxy"
Jul 28 23:52:14.150227 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:14.150204    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2aaf48a6-66b75f75f4-s5zgs"
Jul 28 23:52:14.152663 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:14.152634    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-2aaf48a6-serving-cert\""
Jul 28 23:52:14.165274 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:14.165245    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-2aaf48a6-66b75f75f4-s5zgs"]
Jul 28 23:52:14.282347 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:14.282308    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/01b8cfc5-5721-454b-9f9a-33e7aed58379-openshift-service-ca\") pod \"maas-api-e2e-ait-2aaf48a6-66b75f75f4-s5zgs\" (UID: \"01b8cfc5-5721-454b-9f9a-33e7aed58379\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-2aaf48a6-66b75f75f4-s5zgs"
Jul 28 23:52:14.282515 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:14.282435    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/01b8cfc5-5721-454b-9f9a-33e7aed58379-maas-api-tls\") pod \"maas-api-e2e-ait-2aaf48a6-66b75f75f4-s5zgs\" (UID: \"01b8cfc5-5721-454b-9f9a-33e7aed58379\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-2aaf48a6-66b75f75f4-s5zgs"
Jul 28 23:52:14.282515 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:14.282484    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zz8c4\" (UniqueName: \"kubernetes.io/projected/01b8cfc5-5721-454b-9f9a-33e7aed58379-kube-api-access-zz8c4\") pod \"maas-api-e2e-ait-2aaf48a6-66b75f75f4-s5zgs\" (UID: \"01b8cfc5-5721-454b-9f9a-33e7aed58379\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-2aaf48a6-66b75f75f4-s5zgs"
Jul 28 23:52:14.382989 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:14.382952    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/01b8cfc5-5721-454b-9f9a-33e7aed58379-maas-api-tls\") pod \"maas-api-e2e-ait-2aaf48a6-66b75f75f4-s5zgs\" (UID: \"01b8cfc5-5721-454b-9f9a-33e7aed58379\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-2aaf48a6-66b75f75f4-s5zgs"
Jul 28 23:52:14.383157 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:14.383011    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zz8c4\" (UniqueName: \"kubernetes.io/projected/01b8cfc5-5721-454b-9f9a-33e7aed58379-kube-api-access-zz8c4\") pod \"maas-api-e2e-ait-2aaf48a6-66b75f75f4-s5zgs\" (UID: \"01b8cfc5-5721-454b-9f9a-33e7aed58379\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-2aaf48a6-66b75f75f4-s5zgs"
Jul 28 23:52:14.383157 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:14.383053    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/01b8cfc5-5721-454b-9f9a-33e7aed58379-openshift-service-ca\") pod \"maas-api-e2e-ait-2aaf48a6-66b75f75f4-s5zgs\" (UID: \"01b8cfc5-5721-454b-9f9a-33e7aed58379\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-2aaf48a6-66b75f75f4-s5zgs"
Jul 28 23:52:14.383774 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:14.383756    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/01b8cfc5-5721-454b-9f9a-33e7aed58379-openshift-service-ca\") pod \"maas-api-e2e-ait-2aaf48a6-66b75f75f4-s5zgs\" (UID: \"01b8cfc5-5721-454b-9f9a-33e7aed58379\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-2aaf48a6-66b75f75f4-s5zgs"
Jul 28 23:52:14.385574 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:14.385554    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/01b8cfc5-5721-454b-9f9a-33e7aed58379-maas-api-tls\") pod \"maas-api-e2e-ait-2aaf48a6-66b75f75f4-s5zgs\" (UID: \"01b8cfc5-5721-454b-9f9a-33e7aed58379\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-2aaf48a6-66b75f75f4-s5zgs"
Jul 28 23:52:14.391199 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:14.391170    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zz8c4\" (UniqueName: \"kubernetes.io/projected/01b8cfc5-5721-454b-9f9a-33e7aed58379-kube-api-access-zz8c4\") pod \"maas-api-e2e-ait-2aaf48a6-66b75f75f4-s5zgs\" (UID: \"01b8cfc5-5721-454b-9f9a-33e7aed58379\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-2aaf48a6-66b75f75f4-s5zgs"
Jul 28 23:52:14.463526 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:14.463442    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2aaf48a6-66b75f75f4-s5zgs"
Jul 28 23:52:14.597699 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:14.597674    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-2aaf48a6-66b75f75f4-s5zgs"]
Jul 28 23:52:14.599820 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:52:14.599794    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod01b8cfc5_5721_454b_9f9a_33e7aed58379.slice/crio-18d853e0c47bdfa196eff826c6ca40cee96bd9b1552b682e120e3e4d48e7a89d WatchSource:0}: Error finding container 18d853e0c47bdfa196eff826c6ca40cee96bd9b1552b682e120e3e4d48e7a89d: Status 404 returned error can't find the container with id 18d853e0c47bdfa196eff826c6ca40cee96bd9b1552b682e120e3e4d48e7a89d
Jul 28 23:52:15.396567 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:15.396480    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2aaf48a6-66b75f75f4-s5zgs" event={"ID":"01b8cfc5-5721-454b-9f9a-33e7aed58379","Type":"ContainerStarted","Data":"1405b4a7fc7f9b550166693d5a53ae1208b47119aad6243ce51633b80062c0f4"}
Jul 28 23:52:15.396567 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:15.396532    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2aaf48a6-66b75f75f4-s5zgs" event={"ID":"01b8cfc5-5721-454b-9f9a-33e7aed58379","Type":"ContainerStarted","Data":"18d853e0c47bdfa196eff826c6ca40cee96bd9b1552b682e120e3e4d48e7a89d"}
Jul 28 23:52:15.396988 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:15.396612    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2aaf48a6-66b75f75f4-s5zgs"
Jul 28 23:52:15.414150 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:15.414102    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2aaf48a6-66b75f75f4-s5zgs" podStartSLOduration=1.011944494 podStartE2EDuration="1.414085994s" totalImagesPullingTime="402.1415ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 23:52:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 23:52:15.412703811 +0000 UTC m=+1812.105378321" watchObservedRunningTime="2026-07-28 23:52:15.414085994 +0000 UTC m=+1812.106760503"
Jul 28 23:52:15.821331 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:15.821304    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-47ffb609-gw-openshift-default-588bbd686d-bg6xg"
Jul 28 23:52:15.895088 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:15.895052    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qpfnk\" (UniqueName: \"kubernetes.io/projected/9c755016-49d5-4ba4-8ce9-6b457d8a9bcf-kube-api-access-qpfnk\") pod \"9c755016-49d5-4ba4-8ce9-6b457d8a9bcf\" (UID: \"9c755016-49d5-4ba4-8ce9-6b457d8a9bcf\") "
Jul 28 23:52:15.895276 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:15.895126    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9c755016-49d5-4ba4-8ce9-6b457d8a9bcf-workload-socket\") pod \"9c755016-49d5-4ba4-8ce9-6b457d8a9bcf\" (UID: \"9c755016-49d5-4ba4-8ce9-6b457d8a9bcf\") "
Jul 28 23:52:15.895276 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:15.895151    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9c755016-49d5-4ba4-8ce9-6b457d8a9bcf-credential-socket\") pod \"9c755016-49d5-4ba4-8ce9-6b457d8a9bcf\" (UID: \"9c755016-49d5-4ba4-8ce9-6b457d8a9bcf\") "
Jul 28 23:52:15.895276 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:15.895202    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9c755016-49d5-4ba4-8ce9-6b457d8a9bcf-workload-certs\") pod \"9c755016-49d5-4ba4-8ce9-6b457d8a9bcf\" (UID: \"9c755016-49d5-4ba4-8ce9-6b457d8a9bcf\") "
Jul 28 23:52:15.895276 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:15.895251    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9c755016-49d5-4ba4-8ce9-6b457d8a9bcf-istio-podinfo\") pod \"9c755016-49d5-4ba4-8ce9-6b457d8a9bcf\" (UID: \"9c755016-49d5-4ba4-8ce9-6b457d8a9bcf\") "
Jul 28 23:52:15.895486 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:15.895277    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9c755016-49d5-4ba4-8ce9-6b457d8a9bcf-istio-token\") pod \"9c755016-49d5-4ba4-8ce9-6b457d8a9bcf\" (UID: \"9c755016-49d5-4ba4-8ce9-6b457d8a9bcf\") "
Jul 28 23:52:15.895486 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:15.895299    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9c755016-49d5-4ba4-8ce9-6b457d8a9bcf-istiod-ca-cert\") pod \"9c755016-49d5-4ba4-8ce9-6b457d8a9bcf\" (UID: \"9c755016-49d5-4ba4-8ce9-6b457d8a9bcf\") "
Jul 28 23:52:15.895486 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:15.895329    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9c755016-49d5-4ba4-8ce9-6b457d8a9bcf-istio-data\") pod \"9c755016-49d5-4ba4-8ce9-6b457d8a9bcf\" (UID: \"9c755016-49d5-4ba4-8ce9-6b457d8a9bcf\") "
Jul 28 23:52:15.895486 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:15.895370    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9c755016-49d5-4ba4-8ce9-6b457d8a9bcf-istio-envoy\") pod \"9c755016-49d5-4ba4-8ce9-6b457d8a9bcf\" (UID: \"9c755016-49d5-4ba4-8ce9-6b457d8a9bcf\") "
Jul 28 23:52:15.895486 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:15.895432    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9c755016-49d5-4ba4-8ce9-6b457d8a9bcf-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "9c755016-49d5-4ba4-8ce9-6b457d8a9bcf" (UID: "9c755016-49d5-4ba4-8ce9-6b457d8a9bcf"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 23:52:15.895486 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:15.895465    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9c755016-49d5-4ba4-8ce9-6b457d8a9bcf-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "9c755016-49d5-4ba4-8ce9-6b457d8a9bcf" (UID: "9c755016-49d5-4ba4-8ce9-6b457d8a9bcf"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 23:52:15.895810 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:15.895552    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9c755016-49d5-4ba4-8ce9-6b457d8a9bcf-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "9c755016-49d5-4ba4-8ce9-6b457d8a9bcf" (UID: "9c755016-49d5-4ba4-8ce9-6b457d8a9bcf"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 23:52:15.895810 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:15.895715    2572 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9c755016-49d5-4ba4-8ce9-6b457d8a9bcf-workload-socket\") on node \"ip-10-0-131-112.ec2.internal\" DevicePath \"\""
Jul 28 23:52:15.895810 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:15.895741    2572 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9c755016-49d5-4ba4-8ce9-6b457d8a9bcf-credential-socket\") on node \"ip-10-0-131-112.ec2.internal\" DevicePath \"\""
Jul 28 23:52:15.895810 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:15.895757    2572 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9c755016-49d5-4ba4-8ce9-6b457d8a9bcf-workload-certs\") on node \"ip-10-0-131-112.ec2.internal\" DevicePath \"\""
Jul 28 23:52:15.895810 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:15.895741    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9c755016-49d5-4ba4-8ce9-6b457d8a9bcf-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "9c755016-49d5-4ba4-8ce9-6b457d8a9bcf" (UID: "9c755016-49d5-4ba4-8ce9-6b457d8a9bcf"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 23:52:15.896729 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:15.896700    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9c755016-49d5-4ba4-8ce9-6b457d8a9bcf-istio-data" (OuterVolumeSpecName: "istio-data") pod "9c755016-49d5-4ba4-8ce9-6b457d8a9bcf" (UID: "9c755016-49d5-4ba4-8ce9-6b457d8a9bcf"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 23:52:15.897697 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:15.897663    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9c755016-49d5-4ba4-8ce9-6b457d8a9bcf-istio-token" (OuterVolumeSpecName: "istio-token") pod "9c755016-49d5-4ba4-8ce9-6b457d8a9bcf" (UID: "9c755016-49d5-4ba4-8ce9-6b457d8a9bcf"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 23:52:15.897837 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:15.897816    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9c755016-49d5-4ba4-8ce9-6b457d8a9bcf-kube-api-access-qpfnk" (OuterVolumeSpecName: "kube-api-access-qpfnk") pod "9c755016-49d5-4ba4-8ce9-6b457d8a9bcf" (UID: "9c755016-49d5-4ba4-8ce9-6b457d8a9bcf"). InnerVolumeSpecName "kube-api-access-qpfnk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 23:52:15.897899 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:15.897821    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9c755016-49d5-4ba4-8ce9-6b457d8a9bcf-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "9c755016-49d5-4ba4-8ce9-6b457d8a9bcf" (UID: "9c755016-49d5-4ba4-8ce9-6b457d8a9bcf"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 23:52:15.898348 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:15.898328    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9c755016-49d5-4ba4-8ce9-6b457d8a9bcf-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "9c755016-49d5-4ba4-8ce9-6b457d8a9bcf" (UID: "9c755016-49d5-4ba4-8ce9-6b457d8a9bcf"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 28 23:52:15.997104 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:15.997070    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qpfnk\" (UniqueName: \"kubernetes.io/projected/9c755016-49d5-4ba4-8ce9-6b457d8a9bcf-kube-api-access-qpfnk\") on node \"ip-10-0-131-112.ec2.internal\" DevicePath \"\""
Jul 28 23:52:15.997104 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:15.997103    2572 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9c755016-49d5-4ba4-8ce9-6b457d8a9bcf-istio-podinfo\") on node \"ip-10-0-131-112.ec2.internal\" DevicePath \"\""
Jul 28 23:52:15.997320 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:15.997117    2572 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9c755016-49d5-4ba4-8ce9-6b457d8a9bcf-istio-token\") on node \"ip-10-0-131-112.ec2.internal\" DevicePath \"\""
Jul 28 23:52:15.997320 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:15.997129    2572 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9c755016-49d5-4ba4-8ce9-6b457d8a9bcf-istiod-ca-cert\") on node \"ip-10-0-131-112.ec2.internal\" DevicePath \"\""
Jul 28 23:52:15.997320 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:15.997140    2572 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9c755016-49d5-4ba4-8ce9-6b457d8a9bcf-istio-data\") on node \"ip-10-0-131-112.ec2.internal\" DevicePath \"\""
Jul 28 23:52:15.997320 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:15.997150    2572 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9c755016-49d5-4ba4-8ce9-6b457d8a9bcf-istio-envoy\") on node \"ip-10-0-131-112.ec2.internal\" DevicePath \"\""
Jul 28 23:52:16.402773 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:16.402689    2572 generic.go:356] "Generic (PLEG): container finished" podID="9c755016-49d5-4ba4-8ce9-6b457d8a9bcf" containerID="4c512b59a0fb178ebb69dbe374a51bc8de13ba3d741542d7df0164b415b27d9c" exitCode=0
Jul 28 23:52:16.402773 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:16.402736    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-47ffb609-gw-openshift-default-588bbd686d-bg6xg"
Jul 28 23:52:16.403325 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:16.402782    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-47ffb609-gw-openshift-default-588bbd686d-bg6xg" event={"ID":"9c755016-49d5-4ba4-8ce9-6b457d8a9bcf","Type":"ContainerDied","Data":"4c512b59a0fb178ebb69dbe374a51bc8de13ba3d741542d7df0164b415b27d9c"}
Jul 28 23:52:16.403325 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:16.402846    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-47ffb609-gw-openshift-default-588bbd686d-bg6xg" event={"ID":"9c755016-49d5-4ba4-8ce9-6b457d8a9bcf","Type":"ContainerDied","Data":"5eba3c3e5b7234c0d935b25a4d5cc8d8b9c6244eae3a599591e0ce2671cbc875"}
Jul 28 23:52:16.403325 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:16.402869    2572 scope.go:117] "RemoveContainer" containerID="4c512b59a0fb178ebb69dbe374a51bc8de13ba3d741542d7df0164b415b27d9c"
Jul 28 23:52:16.412050 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:16.412030    2572 scope.go:117] "RemoveContainer" containerID="cc1f389e0a0f0b5b9bd7b48d6b058d668d6108b519a42c68def4c735ce17fe7e"
Jul 28 23:52:16.420150 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:16.420126    2572 scope.go:117] "RemoveContainer" containerID="4c512b59a0fb178ebb69dbe374a51bc8de13ba3d741542d7df0164b415b27d9c"
Jul 28 23:52:16.420445 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:52:16.420424    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4c512b59a0fb178ebb69dbe374a51bc8de13ba3d741542d7df0164b415b27d9c\": container with ID starting with 4c512b59a0fb178ebb69dbe374a51bc8de13ba3d741542d7df0164b415b27d9c not found: ID does not exist" containerID="4c512b59a0fb178ebb69dbe374a51bc8de13ba3d741542d7df0164b415b27d9c"
Jul 28 23:52:16.420496 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:16.420455    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4c512b59a0fb178ebb69dbe374a51bc8de13ba3d741542d7df0164b415b27d9c"} err="failed to get container status \"4c512b59a0fb178ebb69dbe374a51bc8de13ba3d741542d7df0164b415b27d9c\": rpc error: code = NotFound desc = could not find container \"4c512b59a0fb178ebb69dbe374a51bc8de13ba3d741542d7df0164b415b27d9c\": container with ID starting with 4c512b59a0fb178ebb69dbe374a51bc8de13ba3d741542d7df0164b415b27d9c not found: ID does not exist"
Jul 28 23:52:16.420496 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:16.420476    2572 scope.go:117] "RemoveContainer" containerID="cc1f389e0a0f0b5b9bd7b48d6b058d668d6108b519a42c68def4c735ce17fe7e"
Jul 28 23:52:16.420773 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:52:16.420749    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cc1f389e0a0f0b5b9bd7b48d6b058d668d6108b519a42c68def4c735ce17fe7e\": container with ID starting with cc1f389e0a0f0b5b9bd7b48d6b058d668d6108b519a42c68def4c735ce17fe7e not found: ID does not exist" containerID="cc1f389e0a0f0b5b9bd7b48d6b058d668d6108b519a42c68def4c735ce17fe7e"
Jul 28 23:52:16.420830 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:16.420785    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cc1f389e0a0f0b5b9bd7b48d6b058d668d6108b519a42c68def4c735ce17fe7e"} err="failed to get container status \"cc1f389e0a0f0b5b9bd7b48d6b058d668d6108b519a42c68def4c735ce17fe7e\": rpc error: code = NotFound desc = could not find container \"cc1f389e0a0f0b5b9bd7b48d6b058d668d6108b519a42c68def4c735ce17fe7e\": container with ID starting with cc1f389e0a0f0b5b9bd7b48d6b058d668d6108b519a42c68def4c735ce17fe7e not found: ID does not exist"
Jul 28 23:52:16.426037 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:16.426000    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-47ffb609-gw-openshift-default-588bbd686d-bg6xg"]
Jul 28 23:52:16.430271 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:16.430246    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-47ffb609-gw-openshift-default-588bbd686d-bg6xg"]
Jul 28 23:52:17.871465 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:17.871431    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9c755016-49d5-4ba4-8ce9-6b457d8a9bcf" path="/var/lib/kubelet/pods/9c755016-49d5-4ba4-8ce9-6b457d8a9bcf/volumes"
Jul 28 23:52:21.408690 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:21.408661    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2aaf48a6-66b75f75f4-s5zgs"
Jul 28 23:52:39.733241 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:39.733200    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2aaf48a6-klszg"]
Jul 28 23:52:39.733718 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:39.733702    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e5c4fac5-120e-460a-a55c-3af87f0304bb" containerName="istio-proxy"
Jul 28 23:52:39.733766 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:39.733722    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="e5c4fac5-120e-460a-a55c-3af87f0304bb" containerName="istio-proxy"
Jul 28 23:52:39.733766 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:39.733749    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9c755016-49d5-4ba4-8ce9-6b457d8a9bcf" containerName="istio-proxy"
Jul 28 23:52:39.733766 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:39.733757    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="9c755016-49d5-4ba4-8ce9-6b457d8a9bcf" containerName="istio-proxy"
Jul 28 23:52:39.733854 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:39.733770    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9c755016-49d5-4ba4-8ce9-6b457d8a9bcf" containerName="istio-proxy"
Jul 28 23:52:39.733854 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:39.733780    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="9c755016-49d5-4ba4-8ce9-6b457d8a9bcf" containerName="istio-proxy"
Jul 28 23:52:39.733917 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:39.733856    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="9c755016-49d5-4ba4-8ce9-6b457d8a9bcf" containerName="istio-proxy"
Jul 28 23:52:39.733917 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:39.733867    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="9c755016-49d5-4ba4-8ce9-6b457d8a9bcf" containerName="istio-proxy"
Jul 28 23:52:39.737231 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:39.737209    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2aaf48a6-klszg"
Jul 28 23:52:39.739808 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:39.739778    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-rszgs\""
Jul 28 23:52:39.741924 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:39.741895    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2aaf48a6-klszg"]
Jul 28 23:52:39.901480 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:39.901442    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/27e39bbc-eb75-432d-bb04-c1b8fe2d3397-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-2aaf48a6-klszg\" (UID: \"27e39bbc-eb75-432d-bb04-c1b8fe2d3397\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2aaf48a6-klszg"
Jul 28 23:52:40.002611 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:40.002524    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/27e39bbc-eb75-432d-bb04-c1b8fe2d3397-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-2aaf48a6-klszg\" (UID: \"27e39bbc-eb75-432d-bb04-c1b8fe2d3397\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2aaf48a6-klszg"
Jul 28 23:52:40.003171 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:40.003150    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/27e39bbc-eb75-432d-bb04-c1b8fe2d3397-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-2aaf48a6-klszg\" (UID: \"27e39bbc-eb75-432d-bb04-c1b8fe2d3397\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2aaf48a6-klszg"
Jul 28 23:52:40.051261 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:40.051224    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2aaf48a6-klszg"
Jul 28 23:52:40.183746 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:40.183714    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2aaf48a6-klszg"]
Jul 28 23:52:40.186408 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:52:40.186376    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod27e39bbc_eb75_432d_bb04_c1b8fe2d3397.slice/crio-1ed0353abb44ac3e408889c492952bb81a4785d4278061bbb7ed619a7cc0a8a9 WatchSource:0}: Error finding container 1ed0353abb44ac3e408889c492952bb81a4785d4278061bbb7ed619a7cc0a8a9: Status 404 returned error can't find the container with id 1ed0353abb44ac3e408889c492952bb81a4785d4278061bbb7ed619a7cc0a8a9
Jul 28 23:52:40.508681 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:40.508638    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2aaf48a6-klszg" event={"ID":"27e39bbc-eb75-432d-bb04-c1b8fe2d3397","Type":"ContainerStarted","Data":"1ed0353abb44ac3e408889c492952bb81a4785d4278061bbb7ed619a7cc0a8a9"}
Jul 28 23:52:43.524492 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:43.524454    2572 generic.go:356] "Generic (PLEG): container finished" podID="27e39bbc-eb75-432d-bb04-c1b8fe2d3397" containerID="b2f825e49a093ccbcfdcffef3f322ff032e5e497638f243fcdde674e04a25b78" exitCode=0
Jul 28 23:52:43.524871 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:43.524536    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2aaf48a6-klszg" event={"ID":"27e39bbc-eb75-432d-bb04-c1b8fe2d3397","Type":"ContainerDied","Data":"b2f825e49a093ccbcfdcffef3f322ff032e5e497638f243fcdde674e04a25b78"}
Jul 28 23:52:44.656447 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:44.656422    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2aaf48a6-klszg"
Jul 28 23:52:44.842263 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:44.842173    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/27e39bbc-eb75-432d-bb04-c1b8fe2d3397-maas-api-service-ca\") pod \"27e39bbc-eb75-432d-bb04-c1b8fe2d3397\" (UID: \"27e39bbc-eb75-432d-bb04-c1b8fe2d3397\") "
Jul 28 23:52:44.842528 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:44.842506    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/27e39bbc-eb75-432d-bb04-c1b8fe2d3397-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "27e39bbc-eb75-432d-bb04-c1b8fe2d3397" (UID: "27e39bbc-eb75-432d-bb04-c1b8fe2d3397"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 23:52:44.943310 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:44.943274    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/27e39bbc-eb75-432d-bb04-c1b8fe2d3397-maas-api-service-ca\") on node \"ip-10-0-131-112.ec2.internal\" DevicePath \"\""
Jul 28 23:52:45.534218 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:45.534190    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2aaf48a6-klszg"
Jul 28 23:52:45.534218 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:45.534199    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2aaf48a6-klszg" event={"ID":"27e39bbc-eb75-432d-bb04-c1b8fe2d3397","Type":"ContainerDied","Data":"1ed0353abb44ac3e408889c492952bb81a4785d4278061bbb7ed619a7cc0a8a9"}
Jul 28 23:52:45.534429 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:45.534251    2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1ed0353abb44ac3e408889c492952bb81a4785d4278061bbb7ed619a7cc0a8a9"
Jul 28 23:52:54.811645 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:54.807735    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-2aaf48a6-66b75f75f4-s5zgs"]
Jul 28 23:52:54.811645 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:54.808042    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2aaf48a6-66b75f75f4-s5zgs" podUID="01b8cfc5-5721-454b-9f9a-33e7aed58379" containerName="maas-api" containerID="cri-o://1405b4a7fc7f9b550166693d5a53ae1208b47119aad6243ce51633b80062c0f4" gracePeriod=30
Jul 28 23:52:54.997920 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:54.997891    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2aaf48a6-klszg"]
Jul 28 23:52:55.000271 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:55.000243    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2aaf48a6-klszg"]
Jul 28 23:52:55.062050 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:55.061989    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2aaf48a6-66b75f75f4-s5zgs"
Jul 28 23:52:55.121441 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:55.121409    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/01b8cfc5-5721-454b-9f9a-33e7aed58379-openshift-service-ca\") pod \"01b8cfc5-5721-454b-9f9a-33e7aed58379\" (UID: \"01b8cfc5-5721-454b-9f9a-33e7aed58379\") "
Jul 28 23:52:55.121607 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:55.121458    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/01b8cfc5-5721-454b-9f9a-33e7aed58379-maas-api-tls\") pod \"01b8cfc5-5721-454b-9f9a-33e7aed58379\" (UID: \"01b8cfc5-5721-454b-9f9a-33e7aed58379\") "
Jul 28 23:52:55.121607 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:55.121498    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zz8c4\" (UniqueName: \"kubernetes.io/projected/01b8cfc5-5721-454b-9f9a-33e7aed58379-kube-api-access-zz8c4\") pod \"01b8cfc5-5721-454b-9f9a-33e7aed58379\" (UID: \"01b8cfc5-5721-454b-9f9a-33e7aed58379\") "
Jul 28 23:52:55.121799 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:55.121777    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/01b8cfc5-5721-454b-9f9a-33e7aed58379-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "01b8cfc5-5721-454b-9f9a-33e7aed58379" (UID: "01b8cfc5-5721-454b-9f9a-33e7aed58379"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 23:52:55.123576 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:55.123547    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/01b8cfc5-5721-454b-9f9a-33e7aed58379-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "01b8cfc5-5721-454b-9f9a-33e7aed58379" (UID: "01b8cfc5-5721-454b-9f9a-33e7aed58379"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 23:52:55.123689 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:55.123582    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/01b8cfc5-5721-454b-9f9a-33e7aed58379-kube-api-access-zz8c4" (OuterVolumeSpecName: "kube-api-access-zz8c4") pod "01b8cfc5-5721-454b-9f9a-33e7aed58379" (UID: "01b8cfc5-5721-454b-9f9a-33e7aed58379"). InnerVolumeSpecName "kube-api-access-zz8c4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 23:52:55.222101 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:55.222045    2572 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/01b8cfc5-5721-454b-9f9a-33e7aed58379-openshift-service-ca\") on node \"ip-10-0-131-112.ec2.internal\" DevicePath \"\""
Jul 28 23:52:55.222101 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:55.222095    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/01b8cfc5-5721-454b-9f9a-33e7aed58379-maas-api-tls\") on node \"ip-10-0-131-112.ec2.internal\" DevicePath \"\""
Jul 28 23:52:55.222101 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:55.222108    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zz8c4\" (UniqueName: \"kubernetes.io/projected/01b8cfc5-5721-454b-9f9a-33e7aed58379-kube-api-access-zz8c4\") on node \"ip-10-0-131-112.ec2.internal\" DevicePath \"\""
Jul 28 23:52:55.579159 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:55.579123    2572 generic.go:356] "Generic (PLEG): container finished" podID="01b8cfc5-5721-454b-9f9a-33e7aed58379" containerID="1405b4a7fc7f9b550166693d5a53ae1208b47119aad6243ce51633b80062c0f4" exitCode=0
Jul 28 23:52:55.579342 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:55.579167    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2aaf48a6-66b75f75f4-s5zgs"
Jul 28 23:52:55.579342 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:55.579210    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2aaf48a6-66b75f75f4-s5zgs" event={"ID":"01b8cfc5-5721-454b-9f9a-33e7aed58379","Type":"ContainerDied","Data":"1405b4a7fc7f9b550166693d5a53ae1208b47119aad6243ce51633b80062c0f4"}
Jul 28 23:52:55.579342 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:55.579258    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2aaf48a6-66b75f75f4-s5zgs" event={"ID":"01b8cfc5-5721-454b-9f9a-33e7aed58379","Type":"ContainerDied","Data":"18d853e0c47bdfa196eff826c6ca40cee96bd9b1552b682e120e3e4d48e7a89d"}
Jul 28 23:52:55.579342 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:55.579280    2572 scope.go:117] "RemoveContainer" containerID="1405b4a7fc7f9b550166693d5a53ae1208b47119aad6243ce51633b80062c0f4"
Jul 28 23:52:55.587730 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:55.587705    2572 scope.go:117] "RemoveContainer" containerID="1405b4a7fc7f9b550166693d5a53ae1208b47119aad6243ce51633b80062c0f4"
Jul 28 23:52:55.587965 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:52:55.587945    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1405b4a7fc7f9b550166693d5a53ae1208b47119aad6243ce51633b80062c0f4\": container with ID starting with 1405b4a7fc7f9b550166693d5a53ae1208b47119aad6243ce51633b80062c0f4 not found: ID does not exist" containerID="1405b4a7fc7f9b550166693d5a53ae1208b47119aad6243ce51633b80062c0f4"
Jul 28 23:52:55.588021 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:55.587979    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1405b4a7fc7f9b550166693d5a53ae1208b47119aad6243ce51633b80062c0f4"} err="failed to get container status \"1405b4a7fc7f9b550166693d5a53ae1208b47119aad6243ce51633b80062c0f4\": rpc error: code = NotFound desc = could not find container \"1405b4a7fc7f9b550166693d5a53ae1208b47119aad6243ce51633b80062c0f4\": container with ID starting with 1405b4a7fc7f9b550166693d5a53ae1208b47119aad6243ce51633b80062c0f4 not found: ID does not exist"
Jul 28 23:52:55.601748 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:55.601710    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-2aaf48a6-66b75f75f4-s5zgs"]
Jul 28 23:52:55.605718 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:55.604415    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-2aaf48a6-66b75f75f4-s5zgs"]
Jul 28 23:52:55.865605 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:55.865522    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="01b8cfc5-5721-454b-9f9a-33e7aed58379" path="/var/lib/kubelet/pods/01b8cfc5-5721-454b-9f9a-33e7aed58379/volumes"
Jul 28 23:52:55.865950 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:55.865875    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="27e39bbc-eb75-432d-bb04-c1b8fe2d3397" path="/var/lib/kubelet/pods/27e39bbc-eb75-432d-bb04-c1b8fe2d3397/volumes"
Jul 28 23:52:56.844198 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:56.844160    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-765c35ee-openshift-default-8459fc56d-fjqd9"]
Jul 28 23:52:56.844635 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:56.844606    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="27e39bbc-eb75-432d-bb04-c1b8fe2d3397" containerName="revoke-keys"
Jul 28 23:52:56.844635 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:56.844639    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="27e39bbc-eb75-432d-bb04-c1b8fe2d3397" containerName="revoke-keys"
Jul 28 23:52:56.844825 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:56.844662    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="01b8cfc5-5721-454b-9f9a-33e7aed58379" containerName="maas-api"
Jul 28 23:52:56.844825 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:56.844671    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="01b8cfc5-5721-454b-9f9a-33e7aed58379" containerName="maas-api"
Jul 28 23:52:56.844825 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:56.844741    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="01b8cfc5-5721-454b-9f9a-33e7aed58379" containerName="maas-api"
Jul 28 23:52:56.844825 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:56.844755    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="27e39bbc-eb75-432d-bb04-c1b8fe2d3397" containerName="revoke-keys"
Jul 28 23:52:56.849092 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:56.849070    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-765c35ee-openshift-default-8459fc56d-fjqd9"
Jul 28 23:52:56.851520 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:56.851495    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-765c35ee-openshift-default-dockercfg-xpb4q\""
Jul 28 23:52:56.859975 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:56.859951    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-765c35ee-openshift-default-8459fc56d-fjqd9"]
Jul 28 23:52:56.938177 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:56.938141    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/813556e8-2865-43f7-8b03-580e0e0b9ad9-istio-data\") pod \"e2e-derive-765c35ee-openshift-default-8459fc56d-fjqd9\" (UID: \"813556e8-2865-43f7-8b03-580e0e0b9ad9\") " pod="openshift-ingress/e2e-derive-765c35ee-openshift-default-8459fc56d-fjqd9"
Jul 28 23:52:56.938754 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:56.938722    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/813556e8-2865-43f7-8b03-580e0e0b9ad9-workload-socket\") pod \"e2e-derive-765c35ee-openshift-default-8459fc56d-fjqd9\" (UID: \"813556e8-2865-43f7-8b03-580e0e0b9ad9\") " pod="openshift-ingress/e2e-derive-765c35ee-openshift-default-8459fc56d-fjqd9"
Jul 28 23:52:56.938904 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:56.938878    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/813556e8-2865-43f7-8b03-580e0e0b9ad9-istio-envoy\") pod \"e2e-derive-765c35ee-openshift-default-8459fc56d-fjqd9\" (UID: \"813556e8-2865-43f7-8b03-580e0e0b9ad9\") " pod="openshift-ingress/e2e-derive-765c35ee-openshift-default-8459fc56d-fjqd9"
Jul 28 23:52:56.939026 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:56.938936    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/813556e8-2865-43f7-8b03-580e0e0b9ad9-workload-certs\") pod \"e2e-derive-765c35ee-openshift-default-8459fc56d-fjqd9\" (UID: \"813556e8-2865-43f7-8b03-580e0e0b9ad9\") " pod="openshift-ingress/e2e-derive-765c35ee-openshift-default-8459fc56d-fjqd9"
Jul 28 23:52:56.939026 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:56.938976    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/813556e8-2865-43f7-8b03-580e0e0b9ad9-credential-socket\") pod \"e2e-derive-765c35ee-openshift-default-8459fc56d-fjqd9\" (UID: \"813556e8-2865-43f7-8b03-580e0e0b9ad9\") " pod="openshift-ingress/e2e-derive-765c35ee-openshift-default-8459fc56d-fjqd9"
Jul 28 23:52:56.939026 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:56.939005    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/813556e8-2865-43f7-8b03-580e0e0b9ad9-istiod-ca-cert\") pod \"e2e-derive-765c35ee-openshift-default-8459fc56d-fjqd9\" (UID: \"813556e8-2865-43f7-8b03-580e0e0b9ad9\") " pod="openshift-ingress/e2e-derive-765c35ee-openshift-default-8459fc56d-fjqd9"
Jul 28 23:52:56.939191 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:56.939114    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/813556e8-2865-43f7-8b03-580e0e0b9ad9-istio-podinfo\") pod \"e2e-derive-765c35ee-openshift-default-8459fc56d-fjqd9\" (UID: \"813556e8-2865-43f7-8b03-580e0e0b9ad9\") " pod="openshift-ingress/e2e-derive-765c35ee-openshift-default-8459fc56d-fjqd9"
Jul 28 23:52:56.939191 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:56.939142    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/813556e8-2865-43f7-8b03-580e0e0b9ad9-istio-token\") pod \"e2e-derive-765c35ee-openshift-default-8459fc56d-fjqd9\" (UID: \"813556e8-2865-43f7-8b03-580e0e0b9ad9\") " pod="openshift-ingress/e2e-derive-765c35ee-openshift-default-8459fc56d-fjqd9"
Jul 28 23:52:56.939191 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:56.939172    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pgps6\" (UniqueName: \"kubernetes.io/projected/813556e8-2865-43f7-8b03-580e0e0b9ad9-kube-api-access-pgps6\") pod \"e2e-derive-765c35ee-openshift-default-8459fc56d-fjqd9\" (UID: \"813556e8-2865-43f7-8b03-580e0e0b9ad9\") " pod="openshift-ingress/e2e-derive-765c35ee-openshift-default-8459fc56d-fjqd9"
Jul 28 23:52:57.040036 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:57.039998    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/813556e8-2865-43f7-8b03-580e0e0b9ad9-istio-podinfo\") pod \"e2e-derive-765c35ee-openshift-default-8459fc56d-fjqd9\" (UID: \"813556e8-2865-43f7-8b03-580e0e0b9ad9\") " pod="openshift-ingress/e2e-derive-765c35ee-openshift-default-8459fc56d-fjqd9"
Jul 28 23:52:57.040036 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:57.040041    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/813556e8-2865-43f7-8b03-580e0e0b9ad9-istio-token\") pod \"e2e-derive-765c35ee-openshift-default-8459fc56d-fjqd9\" (UID: \"813556e8-2865-43f7-8b03-580e0e0b9ad9\") " pod="openshift-ingress/e2e-derive-765c35ee-openshift-default-8459fc56d-fjqd9"
Jul 28 23:52:57.040357 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:57.040062    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pgps6\" (UniqueName: \"kubernetes.io/projected/813556e8-2865-43f7-8b03-580e0e0b9ad9-kube-api-access-pgps6\") pod \"e2e-derive-765c35ee-openshift-default-8459fc56d-fjqd9\" (UID: \"813556e8-2865-43f7-8b03-580e0e0b9ad9\") " pod="openshift-ingress/e2e-derive-765c35ee-openshift-default-8459fc56d-fjqd9"
Jul 28 23:52:57.040357 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:57.040087    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/813556e8-2865-43f7-8b03-580e0e0b9ad9-istio-data\") pod \"e2e-derive-765c35ee-openshift-default-8459fc56d-fjqd9\" (UID: \"813556e8-2865-43f7-8b03-580e0e0b9ad9\") " pod="openshift-ingress/e2e-derive-765c35ee-openshift-default-8459fc56d-fjqd9"
Jul 28 23:52:57.040357 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:57.040107    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/813556e8-2865-43f7-8b03-580e0e0b9ad9-workload-socket\") pod \"e2e-derive-765c35ee-openshift-default-8459fc56d-fjqd9\" (UID: \"813556e8-2865-43f7-8b03-580e0e0b9ad9\") " pod="openshift-ingress/e2e-derive-765c35ee-openshift-default-8459fc56d-fjqd9"
Jul 28 23:52:57.040357 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:57.040128    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/813556e8-2865-43f7-8b03-580e0e0b9ad9-istio-envoy\") pod \"e2e-derive-765c35ee-openshift-default-8459fc56d-fjqd9\" (UID: \"813556e8-2865-43f7-8b03-580e0e0b9ad9\") " pod="openshift-ingress/e2e-derive-765c35ee-openshift-default-8459fc56d-fjqd9"
Jul 28 23:52:57.040357 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:57.040162    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/813556e8-2865-43f7-8b03-580e0e0b9ad9-workload-certs\") pod \"e2e-derive-765c35ee-openshift-default-8459fc56d-fjqd9\" (UID: \"813556e8-2865-43f7-8b03-580e0e0b9ad9\") " pod="openshift-ingress/e2e-derive-765c35ee-openshift-default-8459fc56d-fjqd9"
Jul 28 23:52:57.040357 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:57.040186    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/813556e8-2865-43f7-8b03-580e0e0b9ad9-credential-socket\") pod \"e2e-derive-765c35ee-openshift-default-8459fc56d-fjqd9\" (UID: \"813556e8-2865-43f7-8b03-580e0e0b9ad9\") " pod="openshift-ingress/e2e-derive-765c35ee-openshift-default-8459fc56d-fjqd9"
Jul 28 23:52:57.040357 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:57.040210    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/813556e8-2865-43f7-8b03-580e0e0b9ad9-istiod-ca-cert\") pod \"e2e-derive-765c35ee-openshift-default-8459fc56d-fjqd9\" (UID: \"813556e8-2865-43f7-8b03-580e0e0b9ad9\") " pod="openshift-ingress/e2e-derive-765c35ee-openshift-default-8459fc56d-fjqd9"
Jul 28 23:52:57.040750 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:57.040538    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/813556e8-2865-43f7-8b03-580e0e0b9ad9-istio-data\") pod \"e2e-derive-765c35ee-openshift-default-8459fc56d-fjqd9\" (UID: \"813556e8-2865-43f7-8b03-580e0e0b9ad9\") " pod="openshift-ingress/e2e-derive-765c35ee-openshift-default-8459fc56d-fjqd9"
Jul 28 23:52:57.040853 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:57.040819    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/813556e8-2865-43f7-8b03-580e0e0b9ad9-credential-socket\") pod \"e2e-derive-765c35ee-openshift-default-8459fc56d-fjqd9\" (UID: \"813556e8-2865-43f7-8b03-580e0e0b9ad9\") " pod="openshift-ingress/e2e-derive-765c35ee-openshift-default-8459fc56d-fjqd9"
Jul 28 23:52:57.040939 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:57.040917    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/813556e8-2865-43f7-8b03-580e0e0b9ad9-workload-socket\") pod \"e2e-derive-765c35ee-openshift-default-8459fc56d-fjqd9\" (UID: \"813556e8-2865-43f7-8b03-580e0e0b9ad9\") " pod="openshift-ingress/e2e-derive-765c35ee-openshift-default-8459fc56d-fjqd9"
Jul 28 23:52:57.040987 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:57.040953    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/813556e8-2865-43f7-8b03-580e0e0b9ad9-istiod-ca-cert\") pod \"e2e-derive-765c35ee-openshift-default-8459fc56d-fjqd9\" (UID: \"813556e8-2865-43f7-8b03-580e0e0b9ad9\") " pod="openshift-ingress/e2e-derive-765c35ee-openshift-default-8459fc56d-fjqd9"
Jul 28 23:52:57.041049 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:57.041033    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/813556e8-2865-43f7-8b03-580e0e0b9ad9-workload-certs\") pod \"e2e-derive-765c35ee-openshift-default-8459fc56d-fjqd9\" (UID: \"813556e8-2865-43f7-8b03-580e0e0b9ad9\") " pod="openshift-ingress/e2e-derive-765c35ee-openshift-default-8459fc56d-fjqd9"
Jul 28 23:52:57.042943 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:57.042916    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/813556e8-2865-43f7-8b03-580e0e0b9ad9-istio-envoy\") pod \"e2e-derive-765c35ee-openshift-default-8459fc56d-fjqd9\" (UID: \"813556e8-2865-43f7-8b03-580e0e0b9ad9\") " pod="openshift-ingress/e2e-derive-765c35ee-openshift-default-8459fc56d-fjqd9"
Jul 28 23:52:57.043108 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:57.043092    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/813556e8-2865-43f7-8b03-580e0e0b9ad9-istio-podinfo\") pod \"e2e-derive-765c35ee-openshift-default-8459fc56d-fjqd9\" (UID: \"813556e8-2865-43f7-8b03-580e0e0b9ad9\") " pod="openshift-ingress/e2e-derive-765c35ee-openshift-default-8459fc56d-fjqd9"
Jul 28 23:52:57.047656 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:57.047614    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/813556e8-2865-43f7-8b03-580e0e0b9ad9-istio-token\") pod \"e2e-derive-765c35ee-openshift-default-8459fc56d-fjqd9\" (UID: \"813556e8-2865-43f7-8b03-580e0e0b9ad9\") " pod="openshift-ingress/e2e-derive-765c35ee-openshift-default-8459fc56d-fjqd9"
Jul 28 23:52:57.047995 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:57.047973    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pgps6\" (UniqueName: \"kubernetes.io/projected/813556e8-2865-43f7-8b03-580e0e0b9ad9-kube-api-access-pgps6\") pod \"e2e-derive-765c35ee-openshift-default-8459fc56d-fjqd9\" (UID: \"813556e8-2865-43f7-8b03-580e0e0b9ad9\") " pod="openshift-ingress/e2e-derive-765c35ee-openshift-default-8459fc56d-fjqd9"
Jul 28 23:52:57.160688 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:57.160600    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-765c35ee-openshift-default-8459fc56d-fjqd9"
Jul 28 23:52:57.295513 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:57.295485    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-765c35ee-openshift-default-8459fc56d-fjqd9"]
Jul 28 23:52:57.297116 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:52:57.297083    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod813556e8_2865_43f7_8b03_580e0e0b9ad9.slice/crio-fe0c2b22a9b4326675b5b591faefaf2f4590a6d29db44c13a114fe996f9be694 WatchSource:0}: Error finding container fe0c2b22a9b4326675b5b591faefaf2f4590a6d29db44c13a114fe996f9be694: Status 404 returned error can't find the container with id fe0c2b22a9b4326675b5b591faefaf2f4590a6d29db44c13a114fe996f9be694
Jul 28 23:52:57.299117 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:57.299075    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 23:52:57.299221 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:57.299153    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 23:52:57.299221 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:57.299183    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 23:52:57.590427 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:57.590391    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-765c35ee-openshift-default-8459fc56d-fjqd9" event={"ID":"813556e8-2865-43f7-8b03-580e0e0b9ad9","Type":"ContainerStarted","Data":"ed70466ff4491a9a70ba83f3e1540c87579cd9be989723e38fc54004df0b9d35"}
Jul 28 23:52:57.590589 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:57.590443    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-765c35ee-openshift-default-8459fc56d-fjqd9" event={"ID":"813556e8-2865-43f7-8b03-580e0e0b9ad9","Type":"ContainerStarted","Data":"fe0c2b22a9b4326675b5b591faefaf2f4590a6d29db44c13a114fe996f9be694"}
Jul 28 23:52:57.610166 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:57.610111    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-765c35ee-openshift-default-8459fc56d-fjqd9" podStartSLOduration=1.610094704 podStartE2EDuration="1.610094704s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 23:52:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 23:52:57.608421335 +0000 UTC m=+1854.301095844" watchObservedRunningTime="2026-07-28 23:52:57.610094704 +0000 UTC m=+1854.302769212"
Jul 28 23:52:58.161593 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:58.161555    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-765c35ee-openshift-default-8459fc56d-fjqd9"
Jul 28 23:52:58.166267 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:58.166242    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-765c35ee-openshift-default-8459fc56d-fjqd9"
Jul 28 23:52:58.594248 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:58.594215    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-765c35ee-openshift-default-8459fc56d-fjqd9"
Jul 28 23:52:58.595172 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:58.595152    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-765c35ee-openshift-default-8459fc56d-fjqd9"
Jul 28 23:52:59.013119 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:59.013078    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-765c35ee-5665594995-6ghz6"]
Jul 28 23:52:59.017205 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:59.017182    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-765c35ee-5665594995-6ghz6"
Jul 28 23:52:59.019408 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:59.019384    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-765c35ee-serving-cert\""
Jul 28 23:52:59.027254 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:59.027228    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-765c35ee-5665594995-6ghz6"]
Jul 28 23:52:59.058395 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:59.058358    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/57678621-bc5a-494f-ba39-11c1766a4827-openshift-service-ca\") pod \"maas-api-e2e-derive-765c35ee-5665594995-6ghz6\" (UID: \"57678621-bc5a-494f-ba39-11c1766a4827\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-765c35ee-5665594995-6ghz6"
Jul 28 23:52:59.058586 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:59.058429    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/57678621-bc5a-494f-ba39-11c1766a4827-maas-api-tls\") pod \"maas-api-e2e-derive-765c35ee-5665594995-6ghz6\" (UID: \"57678621-bc5a-494f-ba39-11c1766a4827\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-765c35ee-5665594995-6ghz6"
Jul 28 23:52:59.058586 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:59.058460    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wjzf9\" (UniqueName: \"kubernetes.io/projected/57678621-bc5a-494f-ba39-11c1766a4827-kube-api-access-wjzf9\") pod \"maas-api-e2e-derive-765c35ee-5665594995-6ghz6\" (UID: \"57678621-bc5a-494f-ba39-11c1766a4827\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-765c35ee-5665594995-6ghz6"
Jul 28 23:52:59.159686 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:59.159654    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/57678621-bc5a-494f-ba39-11c1766a4827-openshift-service-ca\") pod \"maas-api-e2e-derive-765c35ee-5665594995-6ghz6\" (UID: \"57678621-bc5a-494f-ba39-11c1766a4827\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-765c35ee-5665594995-6ghz6"
Jul 28 23:52:59.159868 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:59.159709    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/57678621-bc5a-494f-ba39-11c1766a4827-maas-api-tls\") pod \"maas-api-e2e-derive-765c35ee-5665594995-6ghz6\" (UID: \"57678621-bc5a-494f-ba39-11c1766a4827\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-765c35ee-5665594995-6ghz6"
Jul 28 23:52:59.159868 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:52:59.159800    2572 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-765c35ee-serving-cert: secret "maas-api-e2e-derive-765c35ee-serving-cert" not found
Jul 28 23:52:59.159868 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:59.159829    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wjzf9\" (UniqueName: \"kubernetes.io/projected/57678621-bc5a-494f-ba39-11c1766a4827-kube-api-access-wjzf9\") pod \"maas-api-e2e-derive-765c35ee-5665594995-6ghz6\" (UID: \"57678621-bc5a-494f-ba39-11c1766a4827\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-765c35ee-5665594995-6ghz6"
Jul 28 23:52:59.159868 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:52:59.159860    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/57678621-bc5a-494f-ba39-11c1766a4827-maas-api-tls podName:57678621-bc5a-494f-ba39-11c1766a4827 nodeName:}" failed. No retries permitted until 2026-07-28 23:52:59.659842765 +0000 UTC m=+1856.352517252 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/57678621-bc5a-494f-ba39-11c1766a4827-maas-api-tls") pod "maas-api-e2e-derive-765c35ee-5665594995-6ghz6" (UID: "57678621-bc5a-494f-ba39-11c1766a4827") : secret "maas-api-e2e-derive-765c35ee-serving-cert" not found
Jul 28 23:52:59.160457 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:59.160433    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/57678621-bc5a-494f-ba39-11c1766a4827-openshift-service-ca\") pod \"maas-api-e2e-derive-765c35ee-5665594995-6ghz6\" (UID: \"57678621-bc5a-494f-ba39-11c1766a4827\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-765c35ee-5665594995-6ghz6"
Jul 28 23:52:59.178258 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:59.178231    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wjzf9\" (UniqueName: \"kubernetes.io/projected/57678621-bc5a-494f-ba39-11c1766a4827-kube-api-access-wjzf9\") pod \"maas-api-e2e-derive-765c35ee-5665594995-6ghz6\" (UID: \"57678621-bc5a-494f-ba39-11c1766a4827\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-765c35ee-5665594995-6ghz6"
Jul 28 23:52:59.666129 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:59.666089    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/57678621-bc5a-494f-ba39-11c1766a4827-maas-api-tls\") pod \"maas-api-e2e-derive-765c35ee-5665594995-6ghz6\" (UID: \"57678621-bc5a-494f-ba39-11c1766a4827\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-765c35ee-5665594995-6ghz6"
Jul 28 23:52:59.668654 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:59.668611    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/57678621-bc5a-494f-ba39-11c1766a4827-maas-api-tls\") pod \"maas-api-e2e-derive-765c35ee-5665594995-6ghz6\" (UID: \"57678621-bc5a-494f-ba39-11c1766a4827\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-765c35ee-5665594995-6ghz6"
Jul 28 23:52:59.929657 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:52:59.929549    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-765c35ee-5665594995-6ghz6"
Jul 28 23:53:00.066344 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:53:00.066319    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-765c35ee-5665594995-6ghz6"]
Jul 28 23:53:00.068315 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:53:00.068274    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57678621_bc5a_494f_ba39_11c1766a4827.slice/crio-9dc43792bc749396580d53c527922712199e88c8231f27211a945117255b8c9c WatchSource:0}: Error finding container 9dc43792bc749396580d53c527922712199e88c8231f27211a945117255b8c9c: Status 404 returned error can't find the container with id 9dc43792bc749396580d53c527922712199e88c8231f27211a945117255b8c9c
Jul 28 23:53:00.603434 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:53:00.603397    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-765c35ee-5665594995-6ghz6" event={"ID":"57678621-bc5a-494f-ba39-11c1766a4827","Type":"ContainerStarted","Data":"183016fc938f27b7d3c8caa4b5b5f7c09a16c195cb9b0e54344c594ba78fdb07"}
Jul 28 23:53:00.603826 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:53:00.603445    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-765c35ee-5665594995-6ghz6" event={"ID":"57678621-bc5a-494f-ba39-11c1766a4827","Type":"ContainerStarted","Data":"9dc43792bc749396580d53c527922712199e88c8231f27211a945117255b8c9c"}
Jul 28 23:53:00.620194 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:53:00.620140    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-765c35ee-5665594995-6ghz6" podStartSLOduration=2.242181485 podStartE2EDuration="2.620124262s" totalImagesPullingTime="377.942777ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 23:52:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 23:53:00.619347968 +0000 UTC m=+1857.312022503" watchObservedRunningTime="2026-07-28 23:53:00.620124262 +0000 UTC m=+1857.312798768"
Jul 28 23:53:01.608302 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:53:01.608272    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-765c35ee-5665594995-6ghz6"
Jul 28 23:53:07.618828 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:53:07.618793    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-765c35ee-5665594995-6ghz6"
Jul 28 23:53:12.667831 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:53:12.667794    2572 generic.go:356] "Generic (PLEG): container finished" podID="813556e8-2865-43f7-8b03-580e0e0b9ad9" containerID="ed70466ff4491a9a70ba83f3e1540c87579cd9be989723e38fc54004df0b9d35" exitCode=0
Jul 28 23:53:12.668249 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:53:12.667849    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-765c35ee-openshift-default-8459fc56d-fjqd9" event={"ID":"813556e8-2865-43f7-8b03-580e0e0b9ad9","Type":"ContainerDied","Data":"ed70466ff4491a9a70ba83f3e1540c87579cd9be989723e38fc54004df0b9d35"}
Jul 28 23:53:12.668249 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:53:12.668213    2572 scope.go:117] "RemoveContainer" containerID="ed70466ff4491a9a70ba83f3e1540c87579cd9be989723e38fc54004df0b9d35"
Jul 28 23:53:12.669265 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:53:12.669239    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 23:53:12.669342 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:53:12.669325    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 23:53:12.669389 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:53:12.669375    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 23:53:13.594711 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:53:13.594668    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-765c35ee-openshift-default-8459fc56d-fjqd9"
Jul 28 23:53:13.679450 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:53:13.679412    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-765c35ee-openshift-default-8459fc56d-fjqd9" event={"ID":"813556e8-2865-43f7-8b03-580e0e0b9ad9","Type":"ContainerStarted","Data":"e7c3d5a3eef34f3a64b73213d8dee9fcc1d67310d48e43902b762da65b6b31f6"}
Jul 28 23:53:14.161536 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:53:14.161504    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-765c35ee-openshift-default-8459fc56d-fjqd9"
Jul 28 23:53:14.162653 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:53:14.162613    2572 patch_prober.go:28] interesting pod/e2e-derive-765c35ee-openshift-default-8459fc56d-fjqd9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body=
Jul 28 23:53:14.162755 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:53:14.162677    2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-765c35ee-openshift-default-8459fc56d-fjqd9" podUID="813556e8-2865-43f7-8b03-580e0e0b9ad9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused"
Jul 28 23:53:15.161834 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:53:15.161795    2572 patch_prober.go:28] interesting pod/e2e-derive-765c35ee-openshift-default-8459fc56d-fjqd9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body=
Jul 28 23:53:15.162220 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:53:15.161852    2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-765c35ee-openshift-default-8459fc56d-fjqd9" podUID="813556e8-2865-43f7-8b03-580e0e0b9ad9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused"
Jul 28 23:53:16.162006 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:53:16.161970    2572 patch_prober.go:28] interesting pod/e2e-derive-765c35ee-openshift-default-8459fc56d-fjqd9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body=
Jul 28 23:53:16.162387 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:53:16.162024    2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-765c35ee-openshift-default-8459fc56d-fjqd9" podUID="813556e8-2865-43f7-8b03-580e0e0b9ad9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused"
Jul 28 23:53:17.161593 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:53:17.161554    2572 patch_prober.go:28] interesting pod/e2e-derive-765c35ee-openshift-default-8459fc56d-fjqd9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body=
Jul 28 23:53:17.161782 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:53:17.161611    2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-765c35ee-openshift-default-8459fc56d-fjqd9" podUID="813556e8-2865-43f7-8b03-580e0e0b9ad9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused"
Jul 28 23:53:18.165259 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:53:18.165224    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-765c35ee-openshift-default-8459fc56d-fjqd9"
Jul 28 23:53:18.165676 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:53:18.165609    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-765c35ee-openshift-default-8459fc56d-fjqd9"
Jul 28 23:53:18.166351 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:53:18.166333    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-765c35ee-openshift-default-8459fc56d-fjqd9"
Jul 28 23:53:29.881794 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:53:29.881708    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-765c35ee-5665594995-6ghz6"]
Jul 28 23:53:29.882220 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:53:29.881916    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-765c35ee-5665594995-6ghz6" podUID="57678621-bc5a-494f-ba39-11c1766a4827" containerName="maas-api" containerID="cri-o://183016fc938f27b7d3c8caa4b5b5f7c09a16c195cb9b0e54344c594ba78fdb07" gracePeriod=30
Jul 28 23:53:30.137178 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:53:30.137106    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-765c35ee-5665594995-6ghz6"
Jul 28 23:53:30.208729 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:53:30.208693    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-765c35ee-openshift-default-8459fc56d-fjqd9"]
Jul 28 23:53:30.208965 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:53:30.208936    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-765c35ee-openshift-default-8459fc56d-fjqd9" podUID="813556e8-2865-43f7-8b03-580e0e0b9ad9" containerName="istio-proxy" containerID="cri-o://e7c3d5a3eef34f3a64b73213d8dee9fcc1d67310d48e43902b762da65b6b31f6" gracePeriod=30
Jul 28 23:53:30.236831 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:53:30.236804    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/57678621-bc5a-494f-ba39-11c1766a4827-openshift-service-ca\") pod \"57678621-bc5a-494f-ba39-11c1766a4827\" (UID: \"57678621-bc5a-494f-ba39-11c1766a4827\") "
Jul 28 23:53:30.237020 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:53:30.236867    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/57678621-bc5a-494f-ba39-11c1766a4827-maas-api-tls\") pod \"57678621-bc5a-494f-ba39-11c1766a4827\" (UID: \"57678621-bc5a-494f-ba39-11c1766a4827\") "
Jul 28 23:53:30.237020 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:53:30.236904    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wjzf9\" (UniqueName: \"kubernetes.io/projected/57678621-bc5a-494f-ba39-11c1766a4827-kube-api-access-wjzf9\") pod \"57678621-bc5a-494f-ba39-11c1766a4827\" (UID: \"57678621-bc5a-494f-ba39-11c1766a4827\") "
Jul 28 23:53:30.237326 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:53:30.237300    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/57678621-bc5a-494f-ba39-11c1766a4827-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "57678621-bc5a-494f-ba39-11c1766a4827" (UID: "57678621-bc5a-494f-ba39-11c1766a4827"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 23:53:30.239198 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:53:30.239169    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/57678621-bc5a-494f-ba39-11c1766a4827-kube-api-access-wjzf9" (OuterVolumeSpecName: "kube-api-access-wjzf9") pod "57678621-bc5a-494f-ba39-11c1766a4827" (UID: "57678621-bc5a-494f-ba39-11c1766a4827"). InnerVolumeSpecName "kube-api-access-wjzf9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 23:53:30.239321 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:53:30.239243    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/57678621-bc5a-494f-ba39-11c1766a4827-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "57678621-bc5a-494f-ba39-11c1766a4827" (UID: "57678621-bc5a-494f-ba39-11c1766a4827"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 23:53:30.338302 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:53:30.338267    2572 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/57678621-bc5a-494f-ba39-11c1766a4827-openshift-service-ca\") on node \"ip-10-0-131-112.ec2.internal\" DevicePath \"\""
Jul 28 23:53:30.338302 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:53:30.338297    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/57678621-bc5a-494f-ba39-11c1766a4827-maas-api-tls\") on node \"ip-10-0-131-112.ec2.internal\" DevicePath \"\""
Jul 28 23:53:30.338302 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:53:30.338308    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wjzf9\" (UniqueName: \"kubernetes.io/projected/57678621-bc5a-494f-ba39-11c1766a4827-kube-api-access-wjzf9\") on node \"ip-10-0-131-112.ec2.internal\" DevicePath \"\""
Jul 28 23:53:30.755673 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:53:30.755636    2572 generic.go:356] "Generic (PLEG): container finished" podID="57678621-bc5a-494f-ba39-11c1766a4827" containerID="183016fc938f27b7d3c8caa4b5b5f7c09a16c195cb9b0e54344c594ba78fdb07" exitCode=0
Jul 28 23:53:30.755673 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:53:30.755676    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-765c35ee-5665594995-6ghz6" event={"ID":"57678621-bc5a-494f-ba39-11c1766a4827","Type":"ContainerDied","Data":"183016fc938f27b7d3c8caa4b5b5f7c09a16c195cb9b0e54344c594ba78fdb07"}
Jul 28 23:53:30.755981 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:53:30.755680    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-765c35ee-5665594995-6ghz6"
Jul 28 23:53:30.755981 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:53:30.755719    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-765c35ee-5665594995-6ghz6" event={"ID":"57678621-bc5a-494f-ba39-11c1766a4827","Type":"ContainerDied","Data":"9dc43792bc749396580d53c527922712199e88c8231f27211a945117255b8c9c"}
Jul 28 23:53:30.755981 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:53:30.755740    2572 scope.go:117] "RemoveContainer" containerID="183016fc938f27b7d3c8caa4b5b5f7c09a16c195cb9b0e54344c594ba78fdb07"
Jul 28 23:53:30.764969 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:53:30.764952    2572 scope.go:117] "RemoveContainer" containerID="183016fc938f27b7d3c8caa4b5b5f7c09a16c195cb9b0e54344c594ba78fdb07"
Jul 28 23:53:30.765230 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:53:30.765210    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"183016fc938f27b7d3c8caa4b5b5f7c09a16c195cb9b0e54344c594ba78fdb07\": container with ID starting with 183016fc938f27b7d3c8caa4b5b5f7c09a16c195cb9b0e54344c594ba78fdb07 not found: ID does not exist" containerID="183016fc938f27b7d3c8caa4b5b5f7c09a16c195cb9b0e54344c594ba78fdb07"
Jul 28 23:53:30.765298 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:53:30.765239    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"183016fc938f27b7d3c8caa4b5b5f7c09a16c195cb9b0e54344c594ba78fdb07"} err="failed to get container status \"183016fc938f27b7d3c8caa4b5b5f7c09a16c195cb9b0e54344c594ba78fdb07\": rpc error: code = NotFound desc = could not find container \"183016fc938f27b7d3c8caa4b5b5f7c09a16c195cb9b0e54344c594ba78fdb07\": container with ID starting with 183016fc938f27b7d3c8caa4b5b5f7c09a16c195cb9b0e54344c594ba78fdb07 not found: ID does not exist"
Jul 28 23:53:30.778479 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:53:30.778447    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-765c35ee-5665594995-6ghz6"]
Jul 28 23:53:30.785243 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:53:30.785215    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-765c35ee-5665594995-6ghz6"]
Jul 28 23:53:31.864958 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:53:31.864927    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="57678621-bc5a-494f-ba39-11c1766a4827" path="/var/lib/kubelet/pods/57678621-bc5a-494f-ba39-11c1766a4827/volumes"
Jul 28 23:53:33.168025 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:53:33.167984    2572 patch_prober.go:28] interesting pod/e2e-derive-765c35ee-openshift-default-8459fc56d-fjqd9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 28 23:53:33.168431 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:53:33.168041    2572 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-derive-765c35ee-openshift-default-8459fc56d-fjqd9" podUID="813556e8-2865-43f7-8b03-580e0e0b9ad9" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 23:53:35.465325 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:53:35.465299    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-765c35ee-openshift-default-8459fc56d-fjqd9"
Jul 28 23:53:35.584282 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:53:35.584246    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/813556e8-2865-43f7-8b03-580e0e0b9ad9-istio-envoy\") pod \"813556e8-2865-43f7-8b03-580e0e0b9ad9\" (UID: \"813556e8-2865-43f7-8b03-580e0e0b9ad9\") "
Jul 28 23:53:35.584282 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:53:35.584286    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/813556e8-2865-43f7-8b03-580e0e0b9ad9-credential-socket\") pod \"813556e8-2865-43f7-8b03-580e0e0b9ad9\" (UID: \"813556e8-2865-43f7-8b03-580e0e0b9ad9\") "
Jul 28 23:53:35.584512 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:53:35.584312    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/813556e8-2865-43f7-8b03-580e0e0b9ad9-workload-certs\") pod \"813556e8-2865-43f7-8b03-580e0e0b9ad9\" (UID: \"813556e8-2865-43f7-8b03-580e0e0b9ad9\") "
Jul 28 23:53:35.584512 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:53:35.584354    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pgps6\" (UniqueName: \"kubernetes.io/projected/813556e8-2865-43f7-8b03-580e0e0b9ad9-kube-api-access-pgps6\") pod \"813556e8-2865-43f7-8b03-580e0e0b9ad9\" (UID: \"813556e8-2865-43f7-8b03-580e0e0b9ad9\") "
Jul 28 23:53:35.584512 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:53:35.584372    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/813556e8-2865-43f7-8b03-580e0e0b9ad9-istio-token\") pod \"813556e8-2865-43f7-8b03-580e0e0b9ad9\" (UID: \"813556e8-2865-43f7-8b03-580e0e0b9ad9\") "
Jul 28 23:53:35.584512 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:53:35.584403    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/813556e8-2865-43f7-8b03-580e0e0b9ad9-istio-podinfo\") pod \"813556e8-2865-43f7-8b03-580e0e0b9ad9\" (UID: \"813556e8-2865-43f7-8b03-580e0e0b9ad9\") "
Jul 28 23:53:35.584512 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:53:35.584436    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/813556e8-2865-43f7-8b03-580e0e0b9ad9-istio-data\") pod \"813556e8-2865-43f7-8b03-580e0e0b9ad9\" (UID: \"813556e8-2865-43f7-8b03-580e0e0b9ad9\") "
Jul 28 23:53:35.584512 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:53:35.584502    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/813556e8-2865-43f7-8b03-580e0e0b9ad9-istiod-ca-cert\") pod \"813556e8-2865-43f7-8b03-580e0e0b9ad9\" (UID: \"813556e8-2865-43f7-8b03-580e0e0b9ad9\") "
Jul 28 23:53:35.584854 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:53:35.584551    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/813556e8-2865-43f7-8b03-580e0e0b9ad9-workload-socket\") pod \"813556e8-2865-43f7-8b03-580e0e0b9ad9\" (UID: \"813556e8-2865-43f7-8b03-580e0e0b9ad9\") "
Jul 28 23:53:35.584854 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:53:35.584602    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/813556e8-2865-43f7-8b03-580e0e0b9ad9-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "813556e8-2865-43f7-8b03-580e0e0b9ad9" (UID: "813556e8-2865-43f7-8b03-580e0e0b9ad9"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 23:53:35.584854 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:53:35.584608    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/813556e8-2865-43f7-8b03-580e0e0b9ad9-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "813556e8-2865-43f7-8b03-580e0e0b9ad9" (UID: "813556e8-2865-43f7-8b03-580e0e0b9ad9"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 23:53:35.584854 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:53:35.584819    2572 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/813556e8-2865-43f7-8b03-580e0e0b9ad9-credential-socket\") on node \"ip-10-0-131-112.ec2.internal\" DevicePath \"\""
Jul 28 23:53:35.584854 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:53:35.584837    2572 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/813556e8-2865-43f7-8b03-580e0e0b9ad9-workload-certs\") on node \"ip-10-0-131-112.ec2.internal\" DevicePath \"\""
Jul 28 23:53:35.585114 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:53:35.584859    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/813556e8-2865-43f7-8b03-580e0e0b9ad9-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "813556e8-2865-43f7-8b03-580e0e0b9ad9" (UID: "813556e8-2865-43f7-8b03-580e0e0b9ad9"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 23:53:35.585114 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:53:35.584975    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/813556e8-2865-43f7-8b03-580e0e0b9ad9-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "813556e8-2865-43f7-8b03-580e0e0b9ad9" (UID: "813556e8-2865-43f7-8b03-580e0e0b9ad9"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 23:53:35.585508 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:53:35.585484    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/813556e8-2865-43f7-8b03-580e0e0b9ad9-istio-data" (OuterVolumeSpecName: "istio-data") pod "813556e8-2865-43f7-8b03-580e0e0b9ad9" (UID: "813556e8-2865-43f7-8b03-580e0e0b9ad9"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 23:53:35.586600 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:53:35.586571    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/813556e8-2865-43f7-8b03-580e0e0b9ad9-istio-token" (OuterVolumeSpecName: "istio-token") pod "813556e8-2865-43f7-8b03-580e0e0b9ad9" (UID: "813556e8-2865-43f7-8b03-580e0e0b9ad9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 23:53:35.586814 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:53:35.586794    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/813556e8-2865-43f7-8b03-580e0e0b9ad9-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "813556e8-2865-43f7-8b03-580e0e0b9ad9" (UID: "813556e8-2865-43f7-8b03-580e0e0b9ad9"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 23:53:35.586939 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:53:35.586919    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/813556e8-2865-43f7-8b03-580e0e0b9ad9-kube-api-access-pgps6" (OuterVolumeSpecName: "kube-api-access-pgps6") pod "813556e8-2865-43f7-8b03-580e0e0b9ad9" (UID: "813556e8-2865-43f7-8b03-580e0e0b9ad9"). InnerVolumeSpecName "kube-api-access-pgps6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 23:53:35.586999 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:53:35.586978    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/813556e8-2865-43f7-8b03-580e0e0b9ad9-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "813556e8-2865-43f7-8b03-580e0e0b9ad9" (UID: "813556e8-2865-43f7-8b03-580e0e0b9ad9"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 28 23:53:35.685740 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:53:35.685702    2572 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/813556e8-2865-43f7-8b03-580e0e0b9ad9-istiod-ca-cert\") on node \"ip-10-0-131-112.ec2.internal\" DevicePath \"\""
Jul 28 23:53:35.685740 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:53:35.685733    2572 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/813556e8-2865-43f7-8b03-580e0e0b9ad9-workload-socket\") on node \"ip-10-0-131-112.ec2.internal\" DevicePath \"\""
Jul 28 23:53:35.685740 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:53:35.685744    2572 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/813556e8-2865-43f7-8b03-580e0e0b9ad9-istio-envoy\") on node \"ip-10-0-131-112.ec2.internal\" DevicePath \"\""
Jul 28 23:53:35.685964 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:53:35.685753    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pgps6\" (UniqueName: \"kubernetes.io/projected/813556e8-2865-43f7-8b03-580e0e0b9ad9-kube-api-access-pgps6\") on node \"ip-10-0-131-112.ec2.internal\" DevicePath \"\""
Jul 28 23:53:35.685964 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:53:35.685763    2572 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/813556e8-2865-43f7-8b03-580e0e0b9ad9-istio-token\") on node \"ip-10-0-131-112.ec2.internal\" DevicePath \"\""
Jul 28 23:53:35.685964 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:53:35.685771    2572 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/813556e8-2865-43f7-8b03-580e0e0b9ad9-istio-podinfo\") on node \"ip-10-0-131-112.ec2.internal\" DevicePath \"\""
Jul 28 23:53:35.685964 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:53:35.685779    2572 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/813556e8-2865-43f7-8b03-580e0e0b9ad9-istio-data\") on node \"ip-10-0-131-112.ec2.internal\" DevicePath \"\""
Jul 28 23:53:35.778886 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:53:35.778845    2572 generic.go:356] "Generic (PLEG): container finished" podID="813556e8-2865-43f7-8b03-580e0e0b9ad9" containerID="e7c3d5a3eef34f3a64b73213d8dee9fcc1d67310d48e43902b762da65b6b31f6" exitCode=0
Jul 28 23:53:35.778886 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:53:35.778891    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-765c35ee-openshift-default-8459fc56d-fjqd9"
Jul 28 23:53:35.779097 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:53:35.778924    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-765c35ee-openshift-default-8459fc56d-fjqd9" event={"ID":"813556e8-2865-43f7-8b03-580e0e0b9ad9","Type":"ContainerDied","Data":"e7c3d5a3eef34f3a64b73213d8dee9fcc1d67310d48e43902b762da65b6b31f6"}
Jul 28 23:53:35.779097 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:53:35.778976    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-765c35ee-openshift-default-8459fc56d-fjqd9" event={"ID":"813556e8-2865-43f7-8b03-580e0e0b9ad9","Type":"ContainerDied","Data":"fe0c2b22a9b4326675b5b591faefaf2f4590a6d29db44c13a114fe996f9be694"}
Jul 28 23:53:35.779097 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:53:35.778998    2572 scope.go:117] "RemoveContainer" containerID="e7c3d5a3eef34f3a64b73213d8dee9fcc1d67310d48e43902b762da65b6b31f6"
Jul 28 23:53:35.787931 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:53:35.787912    2572 scope.go:117] "RemoveContainer" containerID="ed70466ff4491a9a70ba83f3e1540c87579cd9be989723e38fc54004df0b9d35"
Jul 28 23:53:35.795344 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:53:35.795328    2572 scope.go:117] "RemoveContainer" containerID="e7c3d5a3eef34f3a64b73213d8dee9fcc1d67310d48e43902b762da65b6b31f6"
Jul 28 23:53:35.795601 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:53:35.795584    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e7c3d5a3eef34f3a64b73213d8dee9fcc1d67310d48e43902b762da65b6b31f6\": container with ID starting with e7c3d5a3eef34f3a64b73213d8dee9fcc1d67310d48e43902b762da65b6b31f6 not found: ID does not exist" containerID="e7c3d5a3eef34f3a64b73213d8dee9fcc1d67310d48e43902b762da65b6b31f6"
Jul 28 23:53:35.795691 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:53:35.795609    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e7c3d5a3eef34f3a64b73213d8dee9fcc1d67310d48e43902b762da65b6b31f6"} err="failed to get container status \"e7c3d5a3eef34f3a64b73213d8dee9fcc1d67310d48e43902b762da65b6b31f6\": rpc error: code = NotFound desc = could not find container \"e7c3d5a3eef34f3a64b73213d8dee9fcc1d67310d48e43902b762da65b6b31f6\": container with ID starting with e7c3d5a3eef34f3a64b73213d8dee9fcc1d67310d48e43902b762da65b6b31f6 not found: ID does not exist"
Jul 28 23:53:35.795691 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:53:35.795641    2572 scope.go:117] "RemoveContainer" containerID="ed70466ff4491a9a70ba83f3e1540c87579cd9be989723e38fc54004df0b9d35"
Jul 28 23:53:35.795854 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:53:35.795838    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ed70466ff4491a9a70ba83f3e1540c87579cd9be989723e38fc54004df0b9d35\": container with ID starting with ed70466ff4491a9a70ba83f3e1540c87579cd9be989723e38fc54004df0b9d35 not found: ID does not exist" containerID="ed70466ff4491a9a70ba83f3e1540c87579cd9be989723e38fc54004df0b9d35"
Jul 28 23:53:35.795889 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:53:35.795860    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ed70466ff4491a9a70ba83f3e1540c87579cd9be989723e38fc54004df0b9d35"} err="failed to get container status \"ed70466ff4491a9a70ba83f3e1540c87579cd9be989723e38fc54004df0b9d35\": rpc error: code = NotFound desc = could not find container \"ed70466ff4491a9a70ba83f3e1540c87579cd9be989723e38fc54004df0b9d35\": container with ID starting with ed70466ff4491a9a70ba83f3e1540c87579cd9be989723e38fc54004df0b9d35 not found: ID does not exist"
Jul 28 23:53:35.799290 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:53:35.799265    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-765c35ee-openshift-default-8459fc56d-fjqd9"]
Jul 28 23:53:35.805142 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:53:35.805122    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-765c35ee-openshift-default-8459fc56d-fjqd9"]
Jul 28 23:53:35.865346 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:53:35.865272    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="813556e8-2865-43f7-8b03-580e0e0b9ad9" path="/var/lib/kubelet/pods/813556e8-2865-43f7-8b03-580e0e0b9ad9/volumes"
Jul 28 23:55:23.856466 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:55:23.856429    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-790b15df-openshift-default-5676f444d4-frn24"]
Jul 28 23:55:23.856930 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:55:23.856732    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="813556e8-2865-43f7-8b03-580e0e0b9ad9" containerName="istio-proxy"
Jul 28 23:55:23.856930 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:55:23.856745    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="813556e8-2865-43f7-8b03-580e0e0b9ad9" containerName="istio-proxy"
Jul 28 23:55:23.856930 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:55:23.856754    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="57678621-bc5a-494f-ba39-11c1766a4827" containerName="maas-api"
Jul 28 23:55:23.856930 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:55:23.856760    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="57678621-bc5a-494f-ba39-11c1766a4827" containerName="maas-api"
Jul 28 23:55:23.856930 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:55:23.856821    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="813556e8-2865-43f7-8b03-580e0e0b9ad9" containerName="istio-proxy"
Jul 28 23:55:23.856930 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:55:23.856830    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="57678621-bc5a-494f-ba39-11c1766a4827" containerName="maas-api"
Jul 28 23:55:23.861760 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:55:23.861737    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-790b15df-openshift-default-5676f444d4-frn24"
Jul 28 23:55:23.864335 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:55:23.864307    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-790b15df-openshift-default-dockercfg-q4x4m\""
Jul 28 23:55:23.886930 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:55:23.886902    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-790b15df-openshift-default-5676f444d4-frn24"]
Jul 28 23:55:23.983999 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:55:23.983965    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f043e63d-f2aa-40fb-93b5-d6c97ffd29ca-istio-data\") pod \"e2e-mt-790b15df-openshift-default-5676f444d4-frn24\" (UID: \"f043e63d-f2aa-40fb-93b5-d6c97ffd29ca\") " pod="openshift-ingress/e2e-mt-790b15df-openshift-default-5676f444d4-frn24"
Jul 28 23:55:23.984168 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:55:23.984007    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f043e63d-f2aa-40fb-93b5-d6c97ffd29ca-credential-socket\") pod \"e2e-mt-790b15df-openshift-default-5676f444d4-frn24\" (UID: \"f043e63d-f2aa-40fb-93b5-d6c97ffd29ca\") " pod="openshift-ingress/e2e-mt-790b15df-openshift-default-5676f444d4-frn24"
Jul 28 23:55:23.984168 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:55:23.984046    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f043e63d-f2aa-40fb-93b5-d6c97ffd29ca-workload-certs\") pod \"e2e-mt-790b15df-openshift-default-5676f444d4-frn24\" (UID: \"f043e63d-f2aa-40fb-93b5-d6c97ffd29ca\") " pod="openshift-ingress/e2e-mt-790b15df-openshift-default-5676f444d4-frn24"
Jul 28 23:55:23.984168 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:55:23.984078    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f043e63d-f2aa-40fb-93b5-d6c97ffd29ca-workload-socket\") pod \"e2e-mt-790b15df-openshift-default-5676f444d4-frn24\" (UID: \"f043e63d-f2aa-40fb-93b5-d6c97ffd29ca\") " pod="openshift-ingress/e2e-mt-790b15df-openshift-default-5676f444d4-frn24"
Jul 28 23:55:23.984168 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:55:23.984103    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-twpjn\" (UniqueName: \"kubernetes.io/projected/f043e63d-f2aa-40fb-93b5-d6c97ffd29ca-kube-api-access-twpjn\") pod \"e2e-mt-790b15df-openshift-default-5676f444d4-frn24\" (UID: \"f043e63d-f2aa-40fb-93b5-d6c97ffd29ca\") " pod="openshift-ingress/e2e-mt-790b15df-openshift-default-5676f444d4-frn24"
Jul 28 23:55:23.984326 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:55:23.984186    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f043e63d-f2aa-40fb-93b5-d6c97ffd29ca-istio-token\") pod \"e2e-mt-790b15df-openshift-default-5676f444d4-frn24\" (UID: \"f043e63d-f2aa-40fb-93b5-d6c97ffd29ca\") " pod="openshift-ingress/e2e-mt-790b15df-openshift-default-5676f444d4-frn24"
Jul 28 23:55:23.984326 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:55:23.984220    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f043e63d-f2aa-40fb-93b5-d6c97ffd29ca-istio-podinfo\") pod \"e2e-mt-790b15df-openshift-default-5676f444d4-frn24\" (UID: \"f043e63d-f2aa-40fb-93b5-d6c97ffd29ca\") " pod="openshift-ingress/e2e-mt-790b15df-openshift-default-5676f444d4-frn24"
Jul 28 23:55:23.984326 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:55:23.984243    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f043e63d-f2aa-40fb-93b5-d6c97ffd29ca-istiod-ca-cert\") pod \"e2e-mt-790b15df-openshift-default-5676f444d4-frn24\" (UID: \"f043e63d-f2aa-40fb-93b5-d6c97ffd29ca\") " pod="openshift-ingress/e2e-mt-790b15df-openshift-default-5676f444d4-frn24"
Jul 28 23:55:23.984423 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:55:23.984326    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f043e63d-f2aa-40fb-93b5-d6c97ffd29ca-istio-envoy\") pod \"e2e-mt-790b15df-openshift-default-5676f444d4-frn24\" (UID: \"f043e63d-f2aa-40fb-93b5-d6c97ffd29ca\") " pod="openshift-ingress/e2e-mt-790b15df-openshift-default-5676f444d4-frn24"
Jul 28 23:55:24.085255 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:55:24.085226    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f043e63d-f2aa-40fb-93b5-d6c97ffd29ca-istio-data\") pod \"e2e-mt-790b15df-openshift-default-5676f444d4-frn24\" (UID: \"f043e63d-f2aa-40fb-93b5-d6c97ffd29ca\") " pod="openshift-ingress/e2e-mt-790b15df-openshift-default-5676f444d4-frn24"
Jul 28 23:55:24.085395 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:55:24.085263    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f043e63d-f2aa-40fb-93b5-d6c97ffd29ca-credential-socket\") pod \"e2e-mt-790b15df-openshift-default-5676f444d4-frn24\" (UID: \"f043e63d-f2aa-40fb-93b5-d6c97ffd29ca\") " pod="openshift-ingress/e2e-mt-790b15df-openshift-default-5676f444d4-frn24"
Jul 28 23:55:24.085395 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:55:24.085311    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f043e63d-f2aa-40fb-93b5-d6c97ffd29ca-workload-certs\") pod \"e2e-mt-790b15df-openshift-default-5676f444d4-frn24\" (UID: \"f043e63d-f2aa-40fb-93b5-d6c97ffd29ca\") " pod="openshift-ingress/e2e-mt-790b15df-openshift-default-5676f444d4-frn24"
Jul 28 23:55:24.085474 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:55:24.085386    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f043e63d-f2aa-40fb-93b5-d6c97ffd29ca-workload-socket\") pod \"e2e-mt-790b15df-openshift-default-5676f444d4-frn24\" (UID: \"f043e63d-f2aa-40fb-93b5-d6c97ffd29ca\") " pod="openshift-ingress/e2e-mt-790b15df-openshift-default-5676f444d4-frn24"
Jul 28 23:55:24.085474 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:55:24.085432    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-twpjn\" (UniqueName: \"kubernetes.io/projected/f043e63d-f2aa-40fb-93b5-d6c97ffd29ca-kube-api-access-twpjn\") pod \"e2e-mt-790b15df-openshift-default-5676f444d4-frn24\" (UID: \"f043e63d-f2aa-40fb-93b5-d6c97ffd29ca\") " pod="openshift-ingress/e2e-mt-790b15df-openshift-default-5676f444d4-frn24"
Jul 28 23:55:24.085474 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:55:24.085461    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f043e63d-f2aa-40fb-93b5-d6c97ffd29ca-istio-token\") pod \"e2e-mt-790b15df-openshift-default-5676f444d4-frn24\" (UID: \"f043e63d-f2aa-40fb-93b5-d6c97ffd29ca\") " pod="openshift-ingress/e2e-mt-790b15df-openshift-default-5676f444d4-frn24"
Jul 28 23:55:24.085697 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:55:24.085667    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f043e63d-f2aa-40fb-93b5-d6c97ffd29ca-istio-podinfo\") pod \"e2e-mt-790b15df-openshift-default-5676f444d4-frn24\" (UID: \"f043e63d-f2aa-40fb-93b5-d6c97ffd29ca\") " pod="openshift-ingress/e2e-mt-790b15df-openshift-default-5676f444d4-frn24"
Jul 28 23:55:24.085782 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:55:24.085702    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f043e63d-f2aa-40fb-93b5-d6c97ffd29ca-workload-certs\") pod \"e2e-mt-790b15df-openshift-default-5676f444d4-frn24\" (UID: \"f043e63d-f2aa-40fb-93b5-d6c97ffd29ca\") " pod="openshift-ingress/e2e-mt-790b15df-openshift-default-5676f444d4-frn24"
Jul 28 23:55:24.085782 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:55:24.085717    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f043e63d-f2aa-40fb-93b5-d6c97ffd29ca-istiod-ca-cert\") pod \"e2e-mt-790b15df-openshift-default-5676f444d4-frn24\" (UID: \"f043e63d-f2aa-40fb-93b5-d6c97ffd29ca\") " pod="openshift-ingress/e2e-mt-790b15df-openshift-default-5676f444d4-frn24"
Jul 28 23:55:24.085782 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:55:24.085742    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f043e63d-f2aa-40fb-93b5-d6c97ffd29ca-istio-data\") pod \"e2e-mt-790b15df-openshift-default-5676f444d4-frn24\" (UID: \"f043e63d-f2aa-40fb-93b5-d6c97ffd29ca\") " pod="openshift-ingress/e2e-mt-790b15df-openshift-default-5676f444d4-frn24"
Jul 28 23:55:24.085782 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:55:24.085762    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f043e63d-f2aa-40fb-93b5-d6c97ffd29ca-credential-socket\") pod \"e2e-mt-790b15df-openshift-default-5676f444d4-frn24\" (UID: \"f043e63d-f2aa-40fb-93b5-d6c97ffd29ca\") " pod="openshift-ingress/e2e-mt-790b15df-openshift-default-5676f444d4-frn24"
Jul 28 23:55:24.085782 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:55:24.085774    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f043e63d-f2aa-40fb-93b5-d6c97ffd29ca-istio-envoy\") pod \"e2e-mt-790b15df-openshift-default-5676f444d4-frn24\" (UID: \"f043e63d-f2aa-40fb-93b5-d6c97ffd29ca\") " pod="openshift-ingress/e2e-mt-790b15df-openshift-default-5676f444d4-frn24"
Jul 28 23:55:24.086252 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:55:24.086223    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f043e63d-f2aa-40fb-93b5-d6c97ffd29ca-workload-socket\") pod \"e2e-mt-790b15df-openshift-default-5676f444d4-frn24\" (UID: \"f043e63d-f2aa-40fb-93b5-d6c97ffd29ca\") " pod="openshift-ingress/e2e-mt-790b15df-openshift-default-5676f444d4-frn24"
Jul 28 23:55:24.086376 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:55:24.086293    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f043e63d-f2aa-40fb-93b5-d6c97ffd29ca-istiod-ca-cert\") pod \"e2e-mt-790b15df-openshift-default-5676f444d4-frn24\" (UID: \"f043e63d-f2aa-40fb-93b5-d6c97ffd29ca\") " pod="openshift-ingress/e2e-mt-790b15df-openshift-default-5676f444d4-frn24"
Jul 28 23:55:24.088443 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:55:24.088426    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f043e63d-f2aa-40fb-93b5-d6c97ffd29ca-istio-envoy\") pod \"e2e-mt-790b15df-openshift-default-5676f444d4-frn24\" (UID: \"f043e63d-f2aa-40fb-93b5-d6c97ffd29ca\") " pod="openshift-ingress/e2e-mt-790b15df-openshift-default-5676f444d4-frn24"
Jul 28 23:55:24.088774 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:55:24.088752    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f043e63d-f2aa-40fb-93b5-d6c97ffd29ca-istio-podinfo\") pod \"e2e-mt-790b15df-openshift-default-5676f444d4-frn24\" (UID: \"f043e63d-f2aa-40fb-93b5-d6c97ffd29ca\") " pod="openshift-ingress/e2e-mt-790b15df-openshift-default-5676f444d4-frn24"
Jul 28 23:55:24.093113 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:55:24.093093    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f043e63d-f2aa-40fb-93b5-d6c97ffd29ca-istio-token\") pod \"e2e-mt-790b15df-openshift-default-5676f444d4-frn24\" (UID: \"f043e63d-f2aa-40fb-93b5-d6c97ffd29ca\") " pod="openshift-ingress/e2e-mt-790b15df-openshift-default-5676f444d4-frn24"
Jul 28 23:55:24.093263 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:55:24.093240    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-twpjn\" (UniqueName: \"kubernetes.io/projected/f043e63d-f2aa-40fb-93b5-d6c97ffd29ca-kube-api-access-twpjn\") pod \"e2e-mt-790b15df-openshift-default-5676f444d4-frn24\" (UID: \"f043e63d-f2aa-40fb-93b5-d6c97ffd29ca\") " pod="openshift-ingress/e2e-mt-790b15df-openshift-default-5676f444d4-frn24"
Jul 28 23:55:24.171817 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:55:24.171752    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-790b15df-openshift-default-5676f444d4-frn24"
Jul 28 23:55:24.310984 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:55:24.310958    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-790b15df-openshift-default-5676f444d4-frn24"]
Jul 28 23:55:24.313014 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:55:24.312976    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf043e63d_f2aa_40fb_93b5_d6c97ffd29ca.slice/crio-56c76220300bffbc424a75cfecf5f2bb3b8c3b4126ebc2f80afb8b02a116ff40 WatchSource:0}: Error finding container 56c76220300bffbc424a75cfecf5f2bb3b8c3b4126ebc2f80afb8b02a116ff40: Status 404 returned error can't find the container with id 56c76220300bffbc424a75cfecf5f2bb3b8c3b4126ebc2f80afb8b02a116ff40
Jul 28 23:55:24.315113 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:55:24.315086    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 23:55:24.315187 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:55:24.315142    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 23:55:24.315187 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:55:24.315173    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 23:55:25.252421 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:55:25.252386    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-790b15df-openshift-default-5676f444d4-frn24" event={"ID":"f043e63d-f2aa-40fb-93b5-d6c97ffd29ca","Type":"ContainerStarted","Data":"f6d923224696518821f82751b6179a25949f8441ec6797f3a5f15bd729f1ef92"}
Jul 28 23:55:25.252818 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:55:25.252443    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-790b15df-openshift-default-5676f444d4-frn24" event={"ID":"f043e63d-f2aa-40fb-93b5-d6c97ffd29ca","Type":"ContainerStarted","Data":"56c76220300bffbc424a75cfecf5f2bb3b8c3b4126ebc2f80afb8b02a116ff40"}
Jul 28 23:55:25.273273 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:55:25.273229    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-790b15df-openshift-default-5676f444d4-frn24" podStartSLOduration=2.2732151 podStartE2EDuration="2.2732151s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 23:55:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 23:55:25.270808317 +0000 UTC m=+2001.963482827" watchObservedRunningTime="2026-07-28 23:55:25.2732151 +0000 UTC m=+2001.965889608"
Jul 28 23:55:26.172114 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:55:26.172074    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-790b15df-openshift-default-5676f444d4-frn24"
Jul 28 23:55:26.176869 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:55:26.176843    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-790b15df-openshift-default-5676f444d4-frn24"
Jul 28 23:55:26.256166 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:55:26.256137    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-790b15df-openshift-default-5676f444d4-frn24"
Jul 28 23:55:26.257169 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:55:26.257146    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-790b15df-openshift-default-5676f444d4-frn24"
Jul 28 23:55:40.318315 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:55:40.318230    2572 generic.go:356] "Generic (PLEG): container finished" podID="f043e63d-f2aa-40fb-93b5-d6c97ffd29ca" containerID="f6d923224696518821f82751b6179a25949f8441ec6797f3a5f15bd729f1ef92" exitCode=0
Jul 28 23:55:40.318841 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:55:40.318304    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-790b15df-openshift-default-5676f444d4-frn24" event={"ID":"f043e63d-f2aa-40fb-93b5-d6c97ffd29ca","Type":"ContainerDied","Data":"f6d923224696518821f82751b6179a25949f8441ec6797f3a5f15bd729f1ef92"}
Jul 28 23:55:40.318841 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:55:40.318820    2572 scope.go:117] "RemoveContainer" containerID="f6d923224696518821f82751b6179a25949f8441ec6797f3a5f15bd729f1ef92"
Jul 28 23:55:40.319838 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:55:40.319813    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 23:55:40.319899 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:55:40.319874    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 23:55:40.319944 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:55:40.319903    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 23:55:41.256351 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:55:41.256310    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-790b15df-openshift-default-5676f444d4-frn24"
Jul 28 23:55:41.325589 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:55:41.325555    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-790b15df-openshift-default-5676f444d4-frn24" event={"ID":"f043e63d-f2aa-40fb-93b5-d6c97ffd29ca","Type":"ContainerStarted","Data":"8d012cdce551c05106482ffd30ee837aec9a142368da8c0e7512a8abb4b17b02"}
Jul 28 23:55:42.172890 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:55:42.172859    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-790b15df-openshift-default-5676f444d4-frn24"
Jul 28 23:55:42.173777 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:55:42.173749    2572 patch_prober.go:28] interesting pod/e2e-mt-790b15df-openshift-default-5676f444d4-frn24 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body=
Jul 28 23:55:42.173913 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:55:42.173808    2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-790b15df-openshift-default-5676f444d4-frn24" podUID="f043e63d-f2aa-40fb-93b5-d6c97ffd29ca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused"
Jul 28 23:55:43.172981 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:55:43.172937    2572 patch_prober.go:28] interesting pod/e2e-mt-790b15df-openshift-default-5676f444d4-frn24 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body=
Jul 28 23:55:43.173454 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:55:43.173001    2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-790b15df-openshift-default-5676f444d4-frn24" podUID="f043e63d-f2aa-40fb-93b5-d6c97ffd29ca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused"
Jul 28 23:55:44.173101 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:55:44.173068    2572 patch_prober.go:28] interesting pod/e2e-mt-790b15df-openshift-default-5676f444d4-frn24 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body=
Jul 28 23:55:44.173465 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:55:44.173123    2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-790b15df-openshift-default-5676f444d4-frn24" podUID="f043e63d-f2aa-40fb-93b5-d6c97ffd29ca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused"
Jul 28 23:55:45.172276 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:55:45.172237    2572 patch_prober.go:28] interesting pod/e2e-mt-790b15df-openshift-default-5676f444d4-frn24 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body=
Jul 28 23:55:45.172456 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:55:45.172290    2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-790b15df-openshift-default-5676f444d4-frn24" podUID="f043e63d-f2aa-40fb-93b5-d6c97ffd29ca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused"
Jul 28 23:55:46.176245 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:55:46.176212    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-790b15df-openshift-default-5676f444d4-frn24"
Jul 28 23:55:46.176699 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:55:46.176487    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-790b15df-openshift-default-5676f444d4-frn24"
Jul 28 23:55:46.177293 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:55:46.177275    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-790b15df-openshift-default-5676f444d4-frn24"
Jul 28 23:56:16.369784 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:56:16.369743    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-790b15df-openshift-default-5676f444d4-frn24"]
Jul 28 23:56:16.370347 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:56:16.370016    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-790b15df-openshift-default-5676f444d4-frn24" podUID="f043e63d-f2aa-40fb-93b5-d6c97ffd29ca" containerName="istio-proxy" containerID="cri-o://8d012cdce551c05106482ffd30ee837aec9a142368da8c0e7512a8abb4b17b02" gracePeriod=30
Jul 28 23:56:21.518110 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:56:21.518064    2572 generic.go:356] "Generic (PLEG): container finished" podID="f043e63d-f2aa-40fb-93b5-d6c97ffd29ca" containerID="8d012cdce551c05106482ffd30ee837aec9a142368da8c0e7512a8abb4b17b02" exitCode=0
Jul 28 23:56:21.518464 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:56:21.518114    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-790b15df-openshift-default-5676f444d4-frn24" event={"ID":"f043e63d-f2aa-40fb-93b5-d6c97ffd29ca","Type":"ContainerDied","Data":"8d012cdce551c05106482ffd30ee837aec9a142368da8c0e7512a8abb4b17b02"}
Jul 28 23:56:21.518464 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:56:21.518195    2572 scope.go:117] "RemoveContainer" containerID="f6d923224696518821f82751b6179a25949f8441ec6797f3a5f15bd729f1ef92"
Jul 28 23:56:21.624254 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:56:21.624232    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-790b15df-openshift-default-5676f444d4-frn24"
Jul 28 23:56:21.793457 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:56:21.793419    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f043e63d-f2aa-40fb-93b5-d6c97ffd29ca-workload-socket\") pod \"f043e63d-f2aa-40fb-93b5-d6c97ffd29ca\" (UID: \"f043e63d-f2aa-40fb-93b5-d6c97ffd29ca\") "
Jul 28 23:56:21.793746 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:56:21.793478    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f043e63d-f2aa-40fb-93b5-d6c97ffd29ca-istio-token\") pod \"f043e63d-f2aa-40fb-93b5-d6c97ffd29ca\" (UID: \"f043e63d-f2aa-40fb-93b5-d6c97ffd29ca\") "
Jul 28 23:56:21.793746 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:56:21.793499    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-twpjn\" (UniqueName: \"kubernetes.io/projected/f043e63d-f2aa-40fb-93b5-d6c97ffd29ca-kube-api-access-twpjn\") pod \"f043e63d-f2aa-40fb-93b5-d6c97ffd29ca\" (UID: \"f043e63d-f2aa-40fb-93b5-d6c97ffd29ca\") "
Jul 28 23:56:21.793746 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:56:21.793531    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f043e63d-f2aa-40fb-93b5-d6c97ffd29ca-istio-podinfo\") pod \"f043e63d-f2aa-40fb-93b5-d6c97ffd29ca\" (UID: \"f043e63d-f2aa-40fb-93b5-d6c97ffd29ca\") "
Jul 28 23:56:21.793746 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:56:21.793559    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f043e63d-f2aa-40fb-93b5-d6c97ffd29ca-workload-certs\") pod \"f043e63d-f2aa-40fb-93b5-d6c97ffd29ca\" (UID: \"f043e63d-f2aa-40fb-93b5-d6c97ffd29ca\") "
Jul 28 23:56:21.793746 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:56:21.793606    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f043e63d-f2aa-40fb-93b5-d6c97ffd29ca-istio-envoy\") pod \"f043e63d-f2aa-40fb-93b5-d6c97ffd29ca\" (UID: \"f043e63d-f2aa-40fb-93b5-d6c97ffd29ca\") "
Jul 28 23:56:21.793746 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:56:21.793662    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f043e63d-f2aa-40fb-93b5-d6c97ffd29ca-credential-socket\") pod \"f043e63d-f2aa-40fb-93b5-d6c97ffd29ca\" (UID: \"f043e63d-f2aa-40fb-93b5-d6c97ffd29ca\") "
Jul 28 23:56:21.793746 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:56:21.793687    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f043e63d-f2aa-40fb-93b5-d6c97ffd29ca-istio-data\") pod \"f043e63d-f2aa-40fb-93b5-d6c97ffd29ca\" (UID: \"f043e63d-f2aa-40fb-93b5-d6c97ffd29ca\") "
Jul 28 23:56:21.793746 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:56:21.793708    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f043e63d-f2aa-40fb-93b5-d6c97ffd29ca-istiod-ca-cert\") pod \"f043e63d-f2aa-40fb-93b5-d6c97ffd29ca\" (UID: \"f043e63d-f2aa-40fb-93b5-d6c97ffd29ca\") "
Jul 28 23:56:21.794172 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:56:21.793823    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f043e63d-f2aa-40fb-93b5-d6c97ffd29ca-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f043e63d-f2aa-40fb-93b5-d6c97ffd29ca" (UID: "f043e63d-f2aa-40fb-93b5-d6c97ffd29ca"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 23:56:21.794172 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:56:21.793943    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f043e63d-f2aa-40fb-93b5-d6c97ffd29ca-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f043e63d-f2aa-40fb-93b5-d6c97ffd29ca" (UID: "f043e63d-f2aa-40fb-93b5-d6c97ffd29ca"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 23:56:21.794172 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:56:21.794071    2572 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f043e63d-f2aa-40fb-93b5-d6c97ffd29ca-credential-socket\") on node \"ip-10-0-131-112.ec2.internal\" DevicePath \"\""
Jul 28 23:56:21.794172 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:56:21.794092    2572 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f043e63d-f2aa-40fb-93b5-d6c97ffd29ca-workload-socket\") on node \"ip-10-0-131-112.ec2.internal\" DevicePath \"\""
Jul 28 23:56:21.794172 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:56:21.794139    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f043e63d-f2aa-40fb-93b5-d6c97ffd29ca-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f043e63d-f2aa-40fb-93b5-d6c97ffd29ca" (UID: "f043e63d-f2aa-40fb-93b5-d6c97ffd29ca"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 23:56:21.794172 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:56:21.794139    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f043e63d-f2aa-40fb-93b5-d6c97ffd29ca-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f043e63d-f2aa-40fb-93b5-d6c97ffd29ca" (UID: "f043e63d-f2aa-40fb-93b5-d6c97ffd29ca"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 23:56:21.794938 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:56:21.794911    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f043e63d-f2aa-40fb-93b5-d6c97ffd29ca-istio-data" (OuterVolumeSpecName: "istio-data") pod "f043e63d-f2aa-40fb-93b5-d6c97ffd29ca" (UID: "f043e63d-f2aa-40fb-93b5-d6c97ffd29ca"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 23:56:21.795741 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:56:21.795718    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f043e63d-f2aa-40fb-93b5-d6c97ffd29ca-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f043e63d-f2aa-40fb-93b5-d6c97ffd29ca" (UID: "f043e63d-f2aa-40fb-93b5-d6c97ffd29ca"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 28 23:56:21.795741 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:56:21.795731    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f043e63d-f2aa-40fb-93b5-d6c97ffd29ca-kube-api-access-twpjn" (OuterVolumeSpecName: "kube-api-access-twpjn") pod "f043e63d-f2aa-40fb-93b5-d6c97ffd29ca" (UID: "f043e63d-f2aa-40fb-93b5-d6c97ffd29ca"). InnerVolumeSpecName "kube-api-access-twpjn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 23:56:21.795880 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:56:21.795829    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f043e63d-f2aa-40fb-93b5-d6c97ffd29ca-istio-token" (OuterVolumeSpecName: "istio-token") pod "f043e63d-f2aa-40fb-93b5-d6c97ffd29ca" (UID: "f043e63d-f2aa-40fb-93b5-d6c97ffd29ca"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 23:56:21.795991 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:56:21.795972    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f043e63d-f2aa-40fb-93b5-d6c97ffd29ca-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f043e63d-f2aa-40fb-93b5-d6c97ffd29ca" (UID: "f043e63d-f2aa-40fb-93b5-d6c97ffd29ca"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 23:56:21.894800 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:56:21.894775    2572 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f043e63d-f2aa-40fb-93b5-d6c97ffd29ca-istio-envoy\") on node \"ip-10-0-131-112.ec2.internal\" DevicePath \"\""
Jul 28 23:56:21.894800 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:56:21.894800    2572 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f043e63d-f2aa-40fb-93b5-d6c97ffd29ca-istio-data\") on node \"ip-10-0-131-112.ec2.internal\" DevicePath \"\""
Jul 28 23:56:21.894951 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:56:21.894813    2572 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f043e63d-f2aa-40fb-93b5-d6c97ffd29ca-istiod-ca-cert\") on node \"ip-10-0-131-112.ec2.internal\" DevicePath \"\""
Jul 28 23:56:21.894951 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:56:21.894824    2572 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f043e63d-f2aa-40fb-93b5-d6c97ffd29ca-istio-token\") on node \"ip-10-0-131-112.ec2.internal\" DevicePath \"\""
Jul 28 23:56:21.894951 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:56:21.894835    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-twpjn\" (UniqueName: \"kubernetes.io/projected/f043e63d-f2aa-40fb-93b5-d6c97ffd29ca-kube-api-access-twpjn\") on node \"ip-10-0-131-112.ec2.internal\" DevicePath \"\""
Jul 28 23:56:21.894951 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:56:21.894849    2572 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f043e63d-f2aa-40fb-93b5-d6c97ffd29ca-istio-podinfo\") on node \"ip-10-0-131-112.ec2.internal\" DevicePath \"\""
Jul 28 23:56:21.894951 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:56:21.894861    2572 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f043e63d-f2aa-40fb-93b5-d6c97ffd29ca-workload-certs\") on node \"ip-10-0-131-112.ec2.internal\" DevicePath \"\""
Jul 28 23:56:22.523163 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:56:22.523129    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-790b15df-openshift-default-5676f444d4-frn24"
Jul 28 23:56:22.523550 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:56:22.523135    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-790b15df-openshift-default-5676f444d4-frn24" event={"ID":"f043e63d-f2aa-40fb-93b5-d6c97ffd29ca","Type":"ContainerDied","Data":"56c76220300bffbc424a75cfecf5f2bb3b8c3b4126ebc2f80afb8b02a116ff40"}
Jul 28 23:56:22.523550 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:56:22.523267    2572 scope.go:117] "RemoveContainer" containerID="8d012cdce551c05106482ffd30ee837aec9a142368da8c0e7512a8abb4b17b02"
Jul 28 23:56:22.539466 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:56:22.539440    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-790b15df-openshift-default-5676f444d4-frn24"]
Jul 28 23:56:22.541759 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:56:22.541736    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-790b15df-openshift-default-5676f444d4-frn24"]
Jul 28 23:56:23.866489 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:56:23.866454    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f043e63d-f2aa-40fb-93b5-d6c97ffd29ca" path="/var/lib/kubelet/pods/f043e63d-f2aa-40fb-93b5-d6c97ffd29ca/volumes"
Jul 28 23:56:35.706161 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:56:35.706128    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-21449885-openshift-default-5f8f558744-xfsxw"]
Jul 28 23:56:35.706760 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:56:35.706464    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="813556e8-2865-43f7-8b03-580e0e0b9ad9" containerName="istio-proxy"
Jul 28 23:56:35.706760 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:56:35.706491    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="813556e8-2865-43f7-8b03-580e0e0b9ad9" containerName="istio-proxy"
Jul 28 23:56:35.706760 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:56:35.706506    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f043e63d-f2aa-40fb-93b5-d6c97ffd29ca" containerName="istio-proxy"
Jul 28 23:56:35.706760 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:56:35.706515    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="f043e63d-f2aa-40fb-93b5-d6c97ffd29ca" containerName="istio-proxy"
Jul 28 23:56:35.706760 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:56:35.706612    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="f043e63d-f2aa-40fb-93b5-d6c97ffd29ca" containerName="istio-proxy"
Jul 28 23:56:35.706760 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:56:35.706640    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="813556e8-2865-43f7-8b03-580e0e0b9ad9" containerName="istio-proxy"
Jul 28 23:56:35.706760 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:56:35.706649    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="f043e63d-f2aa-40fb-93b5-d6c97ffd29ca" containerName="istio-proxy"
Jul 28 23:56:35.710904 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:56:35.710884    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-21449885-openshift-default-5f8f558744-xfsxw"
Jul 28 23:56:35.713369 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:56:35.713351    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-21449885-openshift-default-dockercfg-tgbm4\""
Jul 28 23:56:35.723661 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:56:35.723640    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-21449885-openshift-default-5f8f558744-xfsxw"]
Jul 28 23:56:35.807078 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:56:35.807026    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b71ce34b-61f9-46c0-b3b5-e51e193f4bc4-workload-socket\") pod \"e2e-mt-21449885-openshift-default-5f8f558744-xfsxw\" (UID: \"b71ce34b-61f9-46c0-b3b5-e51e193f4bc4\") " pod="openshift-ingress/e2e-mt-21449885-openshift-default-5f8f558744-xfsxw"
Jul 28 23:56:35.807366 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:56:35.807343    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6hzft\" (UniqueName: \"kubernetes.io/projected/b71ce34b-61f9-46c0-b3b5-e51e193f4bc4-kube-api-access-6hzft\") pod \"e2e-mt-21449885-openshift-default-5f8f558744-xfsxw\" (UID: \"b71ce34b-61f9-46c0-b3b5-e51e193f4bc4\") " pod="openshift-ingress/e2e-mt-21449885-openshift-default-5f8f558744-xfsxw"
Jul 28 23:56:35.807535 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:56:35.807520    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b71ce34b-61f9-46c0-b3b5-e51e193f4bc4-istio-token\") pod \"e2e-mt-21449885-openshift-default-5f8f558744-xfsxw\" (UID: \"b71ce34b-61f9-46c0-b3b5-e51e193f4bc4\") " pod="openshift-ingress/e2e-mt-21449885-openshift-default-5f8f558744-xfsxw"
Jul 28 23:56:35.807664 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:56:35.807650    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b71ce34b-61f9-46c0-b3b5-e51e193f4bc4-istio-data\") pod \"e2e-mt-21449885-openshift-default-5f8f558744-xfsxw\" (UID: \"b71ce34b-61f9-46c0-b3b5-e51e193f4bc4\") " pod="openshift-ingress/e2e-mt-21449885-openshift-default-5f8f558744-xfsxw"
Jul 28 23:56:35.807784 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:56:35.807768    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b71ce34b-61f9-46c0-b3b5-e51e193f4bc4-istio-podinfo\") pod \"e2e-mt-21449885-openshift-default-5f8f558744-xfsxw\" (UID: \"b71ce34b-61f9-46c0-b3b5-e51e193f4bc4\") " pod="openshift-ingress/e2e-mt-21449885-openshift-default-5f8f558744-xfsxw"
Jul 28 23:56:35.807891 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:56:35.807876    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b71ce34b-61f9-46c0-b3b5-e51e193f4bc4-workload-certs\") pod \"e2e-mt-21449885-openshift-default-5f8f558744-xfsxw\" (UID: \"b71ce34b-61f9-46c0-b3b5-e51e193f4bc4\") " pod="openshift-ingress/e2e-mt-21449885-openshift-default-5f8f558744-xfsxw"
Jul 28 23:56:35.807992 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:56:35.807979    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b71ce34b-61f9-46c0-b3b5-e51e193f4bc4-istiod-ca-cert\") pod \"e2e-mt-21449885-openshift-default-5f8f558744-xfsxw\" (UID: \"b71ce34b-61f9-46c0-b3b5-e51e193f4bc4\") " pod="openshift-ingress/e2e-mt-21449885-openshift-default-5f8f558744-xfsxw"
Jul 28 23:56:35.808127 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:56:35.808110    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b71ce34b-61f9-46c0-b3b5-e51e193f4bc4-credential-socket\") pod \"e2e-mt-21449885-openshift-default-5f8f558744-xfsxw\" (UID: \"b71ce34b-61f9-46c0-b3b5-e51e193f4bc4\") " pod="openshift-ingress/e2e-mt-21449885-openshift-default-5f8f558744-xfsxw"
Jul 28 23:56:35.808224 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:56:35.808212    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b71ce34b-61f9-46c0-b3b5-e51e193f4bc4-istio-envoy\") pod \"e2e-mt-21449885-openshift-default-5f8f558744-xfsxw\" (UID: \"b71ce34b-61f9-46c0-b3b5-e51e193f4bc4\") " pod="openshift-ingress/e2e-mt-21449885-openshift-default-5f8f558744-xfsxw"
Jul 28 23:56:35.909393 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:56:35.909359    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b71ce34b-61f9-46c0-b3b5-e51e193f4bc4-credential-socket\") pod \"e2e-mt-21449885-openshift-default-5f8f558744-xfsxw\" (UID: \"b71ce34b-61f9-46c0-b3b5-e51e193f4bc4\") " pod="openshift-ingress/e2e-mt-21449885-openshift-default-5f8f558744-xfsxw"
Jul 28 23:56:35.909573 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:56:35.909406    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b71ce34b-61f9-46c0-b3b5-e51e193f4bc4-istio-envoy\") pod \"e2e-mt-21449885-openshift-default-5f8f558744-xfsxw\" (UID: \"b71ce34b-61f9-46c0-b3b5-e51e193f4bc4\") " pod="openshift-ingress/e2e-mt-21449885-openshift-default-5f8f558744-xfsxw"
Jul 28 23:56:35.909573 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:56:35.909439    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b71ce34b-61f9-46c0-b3b5-e51e193f4bc4-workload-socket\") pod \"e2e-mt-21449885-openshift-default-5f8f558744-xfsxw\" (UID: \"b71ce34b-61f9-46c0-b3b5-e51e193f4bc4\") " pod="openshift-ingress/e2e-mt-21449885-openshift-default-5f8f558744-xfsxw"
Jul 28 23:56:35.909573 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:56:35.909480    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6hzft\" (UniqueName: \"kubernetes.io/projected/b71ce34b-61f9-46c0-b3b5-e51e193f4bc4-kube-api-access-6hzft\") pod \"e2e-mt-21449885-openshift-default-5f8f558744-xfsxw\" (UID: \"b71ce34b-61f9-46c0-b3b5-e51e193f4bc4\") " pod="openshift-ingress/e2e-mt-21449885-openshift-default-5f8f558744-xfsxw"
Jul 28 23:56:35.909573 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:56:35.909540    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b71ce34b-61f9-46c0-b3b5-e51e193f4bc4-istio-token\") pod \"e2e-mt-21449885-openshift-default-5f8f558744-xfsxw\" (UID: \"b71ce34b-61f9-46c0-b3b5-e51e193f4bc4\") " pod="openshift-ingress/e2e-mt-21449885-openshift-default-5f8f558744-xfsxw"
Jul 28 23:56:35.909573 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:56:35.909567    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b71ce34b-61f9-46c0-b3b5-e51e193f4bc4-istio-data\") pod \"e2e-mt-21449885-openshift-default-5f8f558744-xfsxw\" (UID: \"b71ce34b-61f9-46c0-b3b5-e51e193f4bc4\") " pod="openshift-ingress/e2e-mt-21449885-openshift-default-5f8f558744-xfsxw"
Jul 28 23:56:35.909849 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:56:35.909601    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b71ce34b-61f9-46c0-b3b5-e51e193f4bc4-istio-podinfo\") pod \"e2e-mt-21449885-openshift-default-5f8f558744-xfsxw\" (UID: \"b71ce34b-61f9-46c0-b3b5-e51e193f4bc4\") " pod="openshift-ingress/e2e-mt-21449885-openshift-default-5f8f558744-xfsxw"
Jul 28 23:56:35.909849 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:56:35.909651    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b71ce34b-61f9-46c0-b3b5-e51e193f4bc4-workload-certs\") pod \"e2e-mt-21449885-openshift-default-5f8f558744-xfsxw\" (UID: \"b71ce34b-61f9-46c0-b3b5-e51e193f4bc4\") " pod="openshift-ingress/e2e-mt-21449885-openshift-default-5f8f558744-xfsxw"
Jul 28 23:56:35.909849 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:56:35.909678    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b71ce34b-61f9-46c0-b3b5-e51e193f4bc4-istiod-ca-cert\") pod \"e2e-mt-21449885-openshift-default-5f8f558744-xfsxw\" (UID: \"b71ce34b-61f9-46c0-b3b5-e51e193f4bc4\") " pod="openshift-ingress/e2e-mt-21449885-openshift-default-5f8f558744-xfsxw"
Jul 28 23:56:35.909849 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:56:35.909820    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b71ce34b-61f9-46c0-b3b5-e51e193f4bc4-credential-socket\") pod \"e2e-mt-21449885-openshift-default-5f8f558744-xfsxw\" (UID: \"b71ce34b-61f9-46c0-b3b5-e51e193f4bc4\") " pod="openshift-ingress/e2e-mt-21449885-openshift-default-5f8f558744-xfsxw"
Jul 28 23:56:35.909988 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:56:35.909868    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b71ce34b-61f9-46c0-b3b5-e51e193f4bc4-workload-socket\") pod \"e2e-mt-21449885-openshift-default-5f8f558744-xfsxw\" (UID: \"b71ce34b-61f9-46c0-b3b5-e51e193f4bc4\") " pod="openshift-ingress/e2e-mt-21449885-openshift-default-5f8f558744-xfsxw"
Jul 28 23:56:35.910252 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:56:35.910230    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b71ce34b-61f9-46c0-b3b5-e51e193f4bc4-workload-certs\") pod \"e2e-mt-21449885-openshift-default-5f8f558744-xfsxw\" (UID: \"b71ce34b-61f9-46c0-b3b5-e51e193f4bc4\") " pod="openshift-ingress/e2e-mt-21449885-openshift-default-5f8f558744-xfsxw"
Jul 28 23:56:35.910335 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:56:35.910278    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b71ce34b-61f9-46c0-b3b5-e51e193f4bc4-istio-data\") pod \"e2e-mt-21449885-openshift-default-5f8f558744-xfsxw\" (UID: \"b71ce34b-61f9-46c0-b3b5-e51e193f4bc4\") " pod="openshift-ingress/e2e-mt-21449885-openshift-default-5f8f558744-xfsxw"
Jul 28 23:56:35.910431 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:56:35.910412    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b71ce34b-61f9-46c0-b3b5-e51e193f4bc4-istiod-ca-cert\") pod \"e2e-mt-21449885-openshift-default-5f8f558744-xfsxw\" (UID: \"b71ce34b-61f9-46c0-b3b5-e51e193f4bc4\") " pod="openshift-ingress/e2e-mt-21449885-openshift-default-5f8f558744-xfsxw"
Jul 28 23:56:35.912013 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:56:35.911995    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b71ce34b-61f9-46c0-b3b5-e51e193f4bc4-istio-envoy\") pod \"e2e-mt-21449885-openshift-default-5f8f558744-xfsxw\" (UID: \"b71ce34b-61f9-46c0-b3b5-e51e193f4bc4\") " pod="openshift-ingress/e2e-mt-21449885-openshift-default-5f8f558744-xfsxw"
Jul 28 23:56:35.912134 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:56:35.912117    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b71ce34b-61f9-46c0-b3b5-e51e193f4bc4-istio-podinfo\") pod \"e2e-mt-21449885-openshift-default-5f8f558744-xfsxw\" (UID: \"b71ce34b-61f9-46c0-b3b5-e51e193f4bc4\") " pod="openshift-ingress/e2e-mt-21449885-openshift-default-5f8f558744-xfsxw"
Jul 28 23:56:35.917525 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:56:35.917499    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b71ce34b-61f9-46c0-b3b5-e51e193f4bc4-istio-token\") pod \"e2e-mt-21449885-openshift-default-5f8f558744-xfsxw\" (UID: \"b71ce34b-61f9-46c0-b3b5-e51e193f4bc4\") " pod="openshift-ingress/e2e-mt-21449885-openshift-default-5f8f558744-xfsxw"
Jul 28 23:56:35.917606 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:56:35.917573    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6hzft\" (UniqueName: \"kubernetes.io/projected/b71ce34b-61f9-46c0-b3b5-e51e193f4bc4-kube-api-access-6hzft\") pod \"e2e-mt-21449885-openshift-default-5f8f558744-xfsxw\" (UID: \"b71ce34b-61f9-46c0-b3b5-e51e193f4bc4\") " pod="openshift-ingress/e2e-mt-21449885-openshift-default-5f8f558744-xfsxw"
Jul 28 23:56:36.021591 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:56:36.021558    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-21449885-openshift-default-5f8f558744-xfsxw"
Jul 28 23:56:36.155746 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:56:36.155687    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-21449885-openshift-default-5f8f558744-xfsxw"]
Jul 28 23:56:36.158485 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:56:36.158447    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb71ce34b_61f9_46c0_b3b5_e51e193f4bc4.slice/crio-f251fb979c3617e02e226a7a6a1efb9b2bcdfbc61ff5de878d02467fd39a935d WatchSource:0}: Error finding container f251fb979c3617e02e226a7a6a1efb9b2bcdfbc61ff5de878d02467fd39a935d: Status 404 returned error can't find the container with id f251fb979c3617e02e226a7a6a1efb9b2bcdfbc61ff5de878d02467fd39a935d
Jul 28 23:56:36.160599 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:56:36.160572    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 23:56:36.160674 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:56:36.160657    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 23:56:36.160721 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:56:36.160689    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 23:56:36.583323 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:56:36.583283    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-21449885-openshift-default-5f8f558744-xfsxw" event={"ID":"b71ce34b-61f9-46c0-b3b5-e51e193f4bc4","Type":"ContainerStarted","Data":"1ba2e0dfd5976331b16e4284300bc3b54bb50926b12985c1a242df9dcb6a7cc8"}
Jul 28 23:56:36.583535 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:56:36.583342    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-21449885-openshift-default-5f8f558744-xfsxw" event={"ID":"b71ce34b-61f9-46c0-b3b5-e51e193f4bc4","Type":"ContainerStarted","Data":"f251fb979c3617e02e226a7a6a1efb9b2bcdfbc61ff5de878d02467fd39a935d"}
Jul 28 23:56:36.606645 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:56:36.606563    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-21449885-openshift-default-5f8f558744-xfsxw" podStartSLOduration=1.606547252 podStartE2EDuration="1.606547252s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 23:56:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 23:56:36.604400295 +0000 UTC m=+2073.297074806" watchObservedRunningTime="2026-07-28 23:56:36.606547252 +0000 UTC m=+2073.299221761"
Jul 28 23:56:37.022118 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:56:37.022076    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-21449885-openshift-default-5f8f558744-xfsxw"
Jul 28 23:56:37.027324 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:56:37.027299    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-21449885-openshift-default-5f8f558744-xfsxw"
Jul 28 23:56:37.588164 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:56:37.588128    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-21449885-openshift-default-5f8f558744-xfsxw"
Jul 28 23:56:37.589158 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:56:37.589135    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-21449885-openshift-default-5f8f558744-xfsxw"
Jul 28 23:56:38.554395 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:56:38.554366    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-21449885-669d5b6689-t9wtk"]
Jul 28 23:56:38.554859 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:56:38.554835    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f043e63d-f2aa-40fb-93b5-d6c97ffd29ca" containerName="istio-proxy"
Jul 28 23:56:38.554859 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:56:38.554854    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="f043e63d-f2aa-40fb-93b5-d6c97ffd29ca" containerName="istio-proxy"
Jul 28 23:56:38.557808 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:56:38.557782    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-21449885-669d5b6689-t9wtk"
Jul 28 23:56:38.560170 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:56:38.560150    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-21449885-serving-cert\""
Jul 28 23:56:38.569418 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:56:38.569385    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-21449885-669d5b6689-t9wtk"]
Jul 28 23:56:38.634559 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:56:38.634521    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f9cpc\" (UniqueName: \"kubernetes.io/projected/9397ce55-670f-4274-a76a-13a2117a0ba5-kube-api-access-f9cpc\") pod \"maas-api-e2e-mt-21449885-669d5b6689-t9wtk\" (UID: \"9397ce55-670f-4274-a76a-13a2117a0ba5\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-21449885-669d5b6689-t9wtk"
Jul 28 23:56:38.634782 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:56:38.634566    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9397ce55-670f-4274-a76a-13a2117a0ba5-maas-api-tls\") pod \"maas-api-e2e-mt-21449885-669d5b6689-t9wtk\" (UID: \"9397ce55-670f-4274-a76a-13a2117a0ba5\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-21449885-669d5b6689-t9wtk"
Jul 28 23:56:38.634782 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:56:38.634659    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9397ce55-670f-4274-a76a-13a2117a0ba5-openshift-service-ca\") pod \"maas-api-e2e-mt-21449885-669d5b6689-t9wtk\" (UID: \"9397ce55-670f-4274-a76a-13a2117a0ba5\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-21449885-669d5b6689-t9wtk"
Jul 28 23:56:38.735975 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:56:38.735934    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f9cpc\" (UniqueName: \"kubernetes.io/projected/9397ce55-670f-4274-a76a-13a2117a0ba5-kube-api-access-f9cpc\") pod \"maas-api-e2e-mt-21449885-669d5b6689-t9wtk\" (UID: \"9397ce55-670f-4274-a76a-13a2117a0ba5\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-21449885-669d5b6689-t9wtk"
Jul 28 23:56:38.735975 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:56:38.735979    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9397ce55-670f-4274-a76a-13a2117a0ba5-maas-api-tls\") pod \"maas-api-e2e-mt-21449885-669d5b6689-t9wtk\" (UID: \"9397ce55-670f-4274-a76a-13a2117a0ba5\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-21449885-669d5b6689-t9wtk"
Jul 28 23:56:38.736303 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:56:38.736065    2572 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-21449885-serving-cert: secret "maas-api-e2e-mt-21449885-serving-cert" not found
Jul 28 23:56:38.736303 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:56:38.736123    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9397ce55-670f-4274-a76a-13a2117a0ba5-maas-api-tls podName:9397ce55-670f-4274-a76a-13a2117a0ba5 nodeName:}" failed. No retries permitted until 2026-07-28 23:56:39.236105856 +0000 UTC m=+2075.928780344 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/9397ce55-670f-4274-a76a-13a2117a0ba5-maas-api-tls") pod "maas-api-e2e-mt-21449885-669d5b6689-t9wtk" (UID: "9397ce55-670f-4274-a76a-13a2117a0ba5") : secret "maas-api-e2e-mt-21449885-serving-cert" not found
Jul 28 23:56:38.736303 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:56:38.736157    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9397ce55-670f-4274-a76a-13a2117a0ba5-openshift-service-ca\") pod \"maas-api-e2e-mt-21449885-669d5b6689-t9wtk\" (UID: \"9397ce55-670f-4274-a76a-13a2117a0ba5\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-21449885-669d5b6689-t9wtk"
Jul 28 23:56:38.736787 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:56:38.736770    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9397ce55-670f-4274-a76a-13a2117a0ba5-openshift-service-ca\") pod \"maas-api-e2e-mt-21449885-669d5b6689-t9wtk\" (UID: \"9397ce55-670f-4274-a76a-13a2117a0ba5\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-21449885-669d5b6689-t9wtk"
Jul 28 23:56:38.745190 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:56:38.745166    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f9cpc\" (UniqueName: \"kubernetes.io/projected/9397ce55-670f-4274-a76a-13a2117a0ba5-kube-api-access-f9cpc\") pod \"maas-api-e2e-mt-21449885-669d5b6689-t9wtk\" (UID: \"9397ce55-670f-4274-a76a-13a2117a0ba5\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-21449885-669d5b6689-t9wtk"
Jul 28 23:56:39.240983 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:56:39.240949    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9397ce55-670f-4274-a76a-13a2117a0ba5-maas-api-tls\") pod \"maas-api-e2e-mt-21449885-669d5b6689-t9wtk\" (UID: \"9397ce55-670f-4274-a76a-13a2117a0ba5\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-21449885-669d5b6689-t9wtk"
Jul 28 23:56:39.243530 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:56:39.243506    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9397ce55-670f-4274-a76a-13a2117a0ba5-maas-api-tls\") pod \"maas-api-e2e-mt-21449885-669d5b6689-t9wtk\" (UID: \"9397ce55-670f-4274-a76a-13a2117a0ba5\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-21449885-669d5b6689-t9wtk"
Jul 28 23:56:39.469372 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:56:39.469333    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-21449885-669d5b6689-t9wtk"
Jul 28 23:56:39.606315 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:56:39.606290    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-21449885-669d5b6689-t9wtk"]
Jul 28 23:56:39.608381 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:56:39.608357    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9397ce55_670f_4274_a76a_13a2117a0ba5.slice/crio-238602ac580adedec5ed28d50a7c01b99751f158cc7b2be5382d05ed92f6b098 WatchSource:0}: Error finding container 238602ac580adedec5ed28d50a7c01b99751f158cc7b2be5382d05ed92f6b098: Status 404 returned error can't find the container with id 238602ac580adedec5ed28d50a7c01b99751f158cc7b2be5382d05ed92f6b098
Jul 28 23:56:39.609878 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:56:39.609858    2572 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 23:56:40.603762 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:56:40.603726    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-21449885-669d5b6689-t9wtk" event={"ID":"9397ce55-670f-4274-a76a-13a2117a0ba5","Type":"ContainerStarted","Data":"1bc29f06ad5d4cb3c6136ee8ea85683ab895e61e70827e6290fdf31659f9506c"}
Jul 28 23:56:40.603963 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:56:40.603775    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-21449885-669d5b6689-t9wtk" event={"ID":"9397ce55-670f-4274-a76a-13a2117a0ba5","Type":"ContainerStarted","Data":"238602ac580adedec5ed28d50a7c01b99751f158cc7b2be5382d05ed92f6b098"}
Jul 28 23:56:40.603963 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:56:40.603858    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-21449885-669d5b6689-t9wtk"
Jul 28 23:56:40.619601 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:56:40.619553    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-21449885-669d5b6689-t9wtk" podStartSLOduration=2.122235205 podStartE2EDuration="2.61953586s" totalImagesPullingTime="497.300655ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 23:56:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 23:56:40.619056346 +0000 UTC m=+2077.311730867" watchObservedRunningTime="2026-07-28 23:56:40.61953586 +0000 UTC m=+2077.312210368"
Jul 28 23:56:46.613640 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:56:46.613604    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-21449885-669d5b6689-t9wtk"
Jul 28 23:56:51.664158 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:56:51.664126    2572 generic.go:356] "Generic (PLEG): container finished" podID="b71ce34b-61f9-46c0-b3b5-e51e193f4bc4" containerID="1ba2e0dfd5976331b16e4284300bc3b54bb50926b12985c1a242df9dcb6a7cc8" exitCode=0
Jul 28 23:56:51.664575 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:56:51.664202    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-21449885-openshift-default-5f8f558744-xfsxw" event={"ID":"b71ce34b-61f9-46c0-b3b5-e51e193f4bc4","Type":"ContainerDied","Data":"1ba2e0dfd5976331b16e4284300bc3b54bb50926b12985c1a242df9dcb6a7cc8"}
Jul 28 23:56:51.664617 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:56:51.664581    2572 scope.go:117] "RemoveContainer" containerID="1ba2e0dfd5976331b16e4284300bc3b54bb50926b12985c1a242df9dcb6a7cc8"
Jul 28 23:56:51.665589 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:56:51.665562    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 23:56:51.665909 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:56:51.665658    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 23:56:51.665909 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:56:51.665705    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 23:56:52.588522 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:56:52.588486    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-21449885-openshift-default-5f8f558744-xfsxw"
Jul 28 23:56:52.670907 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:56:52.670862    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-21449885-openshift-default-5f8f558744-xfsxw" event={"ID":"b71ce34b-61f9-46c0-b3b5-e51e193f4bc4","Type":"ContainerStarted","Data":"f7c39b497325b8c61a80221c78e58c34eddf92c0486db7ed7be0e84dbe82c65e"}
Jul 28 23:56:53.022041 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:56:53.022012    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-21449885-openshift-default-5f8f558744-xfsxw"
Jul 28 23:56:53.023135 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:56:53.023112    2572 patch_prober.go:28] interesting pod/e2e-mt-21449885-openshift-default-5f8f558744-xfsxw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body=
Jul 28 23:56:53.023246 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:56:53.023166    2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-21449885-openshift-default-5f8f558744-xfsxw" podUID="b71ce34b-61f9-46c0-b3b5-e51e193f4bc4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused"
Jul 28 23:56:54.022499 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:56:54.022463    2572 patch_prober.go:28] interesting pod/e2e-mt-21449885-openshift-default-5f8f558744-xfsxw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body=
Jul 28 23:56:54.022926 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:56:54.022517    2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-21449885-openshift-default-5f8f558744-xfsxw" podUID="b71ce34b-61f9-46c0-b3b5-e51e193f4bc4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused"
Jul 28 23:56:55.022240 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:56:55.022202    2572 patch_prober.go:28] interesting pod/e2e-mt-21449885-openshift-default-5f8f558744-xfsxw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body=
Jul 28 23:56:55.022427 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:56:55.022270    2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-21449885-openshift-default-5f8f558744-xfsxw" podUID="b71ce34b-61f9-46c0-b3b5-e51e193f4bc4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused"
Jul 28 23:56:56.022706 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:56:56.022669    2572 patch_prober.go:28] interesting pod/e2e-mt-21449885-openshift-default-5f8f558744-xfsxw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body=
Jul 28 23:56:56.023087 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:56:56.022730    2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-21449885-openshift-default-5f8f558744-xfsxw" podUID="b71ce34b-61f9-46c0-b3b5-e51e193f4bc4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused"
Jul 28 23:56:57.026465 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:56:57.026432    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-21449885-openshift-default-5f8f558744-xfsxw"
Jul 28 23:56:57.026924 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:56:57.026720    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-21449885-openshift-default-5f8f558744-xfsxw"
Jul 28 23:56:57.027466 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:56:57.027449    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-21449885-openshift-default-5f8f558744-xfsxw"
Jul 28 23:57:53.073997 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:57:53.073959    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-21449885-669d5b6689-t9wtk"]
Jul 28 23:57:53.074489 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:57:53.074243    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-21449885-669d5b6689-t9wtk" podUID="9397ce55-670f-4274-a76a-13a2117a0ba5" containerName="maas-api" containerID="cri-o://1bc29f06ad5d4cb3c6136ee8ea85683ab895e61e70827e6290fdf31659f9506c" gracePeriod=30
Jul 28 23:57:53.325757 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:57:53.325686    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-21449885-669d5b6689-t9wtk"
Jul 28 23:57:53.463540 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:57:53.463512    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9397ce55-670f-4274-a76a-13a2117a0ba5-maas-api-tls\") pod \"9397ce55-670f-4274-a76a-13a2117a0ba5\" (UID: \"9397ce55-670f-4274-a76a-13a2117a0ba5\") "
Jul 28 23:57:53.463540 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:57:53.463545    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9397ce55-670f-4274-a76a-13a2117a0ba5-openshift-service-ca\") pod \"9397ce55-670f-4274-a76a-13a2117a0ba5\" (UID: \"9397ce55-670f-4274-a76a-13a2117a0ba5\") "
Jul 28 23:57:53.463803 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:57:53.463619    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f9cpc\" (UniqueName: \"kubernetes.io/projected/9397ce55-670f-4274-a76a-13a2117a0ba5-kube-api-access-f9cpc\") pod \"9397ce55-670f-4274-a76a-13a2117a0ba5\" (UID: \"9397ce55-670f-4274-a76a-13a2117a0ba5\") "
Jul 28 23:57:53.463948 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:57:53.463925    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9397ce55-670f-4274-a76a-13a2117a0ba5-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "9397ce55-670f-4274-a76a-13a2117a0ba5" (UID: "9397ce55-670f-4274-a76a-13a2117a0ba5"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 23:57:53.465761 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:57:53.465739    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9397ce55-670f-4274-a76a-13a2117a0ba5-kube-api-access-f9cpc" (OuterVolumeSpecName: "kube-api-access-f9cpc") pod "9397ce55-670f-4274-a76a-13a2117a0ba5" (UID: "9397ce55-670f-4274-a76a-13a2117a0ba5"). InnerVolumeSpecName "kube-api-access-f9cpc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 23:57:53.465867 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:57:53.465844    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9397ce55-670f-4274-a76a-13a2117a0ba5-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "9397ce55-670f-4274-a76a-13a2117a0ba5" (UID: "9397ce55-670f-4274-a76a-13a2117a0ba5"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 23:57:53.564884 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:57:53.564854    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9397ce55-670f-4274-a76a-13a2117a0ba5-maas-api-tls\") on node \"ip-10-0-131-112.ec2.internal\" DevicePath \"\""
Jul 28 23:57:53.564884 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:57:53.564878    2572 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9397ce55-670f-4274-a76a-13a2117a0ba5-openshift-service-ca\") on node \"ip-10-0-131-112.ec2.internal\" DevicePath \"\""
Jul 28 23:57:53.564884 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:57:53.564889    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f9cpc\" (UniqueName: \"kubernetes.io/projected/9397ce55-670f-4274-a76a-13a2117a0ba5-kube-api-access-f9cpc\") on node \"ip-10-0-131-112.ec2.internal\" DevicePath \"\""
Jul 28 23:57:53.939961 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:57:53.939929    2572 generic.go:356] "Generic (PLEG): container finished" podID="9397ce55-670f-4274-a76a-13a2117a0ba5" containerID="1bc29f06ad5d4cb3c6136ee8ea85683ab895e61e70827e6290fdf31659f9506c" exitCode=0
Jul 28 23:57:53.940127 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:57:53.939971    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-21449885-669d5b6689-t9wtk" event={"ID":"9397ce55-670f-4274-a76a-13a2117a0ba5","Type":"ContainerDied","Data":"1bc29f06ad5d4cb3c6136ee8ea85683ab895e61e70827e6290fdf31659f9506c"}
Jul 28 23:57:53.940127 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:57:53.939983    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-21449885-669d5b6689-t9wtk"
Jul 28 23:57:53.940127 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:57:53.940009    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-21449885-669d5b6689-t9wtk" event={"ID":"9397ce55-670f-4274-a76a-13a2117a0ba5","Type":"ContainerDied","Data":"238602ac580adedec5ed28d50a7c01b99751f158cc7b2be5382d05ed92f6b098"}
Jul 28 23:57:53.940127 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:57:53.940031    2572 scope.go:117] "RemoveContainer" containerID="1bc29f06ad5d4cb3c6136ee8ea85683ab895e61e70827e6290fdf31659f9506c"
Jul 28 23:57:53.948209 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:57:53.948187    2572 scope.go:117] "RemoveContainer" containerID="1bc29f06ad5d4cb3c6136ee8ea85683ab895e61e70827e6290fdf31659f9506c"
Jul 28 23:57:53.948459 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:57:53.948441    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1bc29f06ad5d4cb3c6136ee8ea85683ab895e61e70827e6290fdf31659f9506c\": container with ID starting with 1bc29f06ad5d4cb3c6136ee8ea85683ab895e61e70827e6290fdf31659f9506c not found: ID does not exist" containerID="1bc29f06ad5d4cb3c6136ee8ea85683ab895e61e70827e6290fdf31659f9506c"
Jul 28 23:57:53.948527 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:57:53.948466    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1bc29f06ad5d4cb3c6136ee8ea85683ab895e61e70827e6290fdf31659f9506c"} err="failed to get container status \"1bc29f06ad5d4cb3c6136ee8ea85683ab895e61e70827e6290fdf31659f9506c\": rpc error: code = NotFound desc = could not find container \"1bc29f06ad5d4cb3c6136ee8ea85683ab895e61e70827e6290fdf31659f9506c\": container with ID starting with 1bc29f06ad5d4cb3c6136ee8ea85683ab895e61e70827e6290fdf31659f9506c not found: ID does not exist"
Jul 28 23:57:53.955151 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:57:53.955131    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-21449885-669d5b6689-t9wtk"]
Jul 28 23:57:53.958231 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:57:53.958208    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-21449885-669d5b6689-t9wtk"]
Jul 28 23:57:55.864960 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:57:55.864924    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9397ce55-670f-4274-a76a-13a2117a0ba5" path="/var/lib/kubelet/pods/9397ce55-670f-4274-a76a-13a2117a0ba5/volumes"
Jul 28 23:58:57.963934 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:58:57.963900    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-21449885-openshift-default-5f8f558744-xfsxw"]
Jul 28 23:58:57.964436 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:58:57.964211    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-21449885-openshift-default-5f8f558744-xfsxw" podUID="b71ce34b-61f9-46c0-b3b5-e51e193f4bc4" containerName="istio-proxy" containerID="cri-o://f7c39b497325b8c61a80221c78e58c34eddf92c0486db7ed7be0e84dbe82c65e" gracePeriod=30
Jul 28 23:59:03.202948 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:59:03.202925    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-21449885-openshift-default-5f8f558744-xfsxw"
Jul 28 23:59:03.240189 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:59:03.240156    2572 generic.go:356] "Generic (PLEG): container finished" podID="b71ce34b-61f9-46c0-b3b5-e51e193f4bc4" containerID="f7c39b497325b8c61a80221c78e58c34eddf92c0486db7ed7be0e84dbe82c65e" exitCode=0
Jul 28 23:59:03.240331 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:59:03.240197    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-21449885-openshift-default-5f8f558744-xfsxw"
Jul 28 23:59:03.240331 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:59:03.240199    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-21449885-openshift-default-5f8f558744-xfsxw" event={"ID":"b71ce34b-61f9-46c0-b3b5-e51e193f4bc4","Type":"ContainerDied","Data":"f7c39b497325b8c61a80221c78e58c34eddf92c0486db7ed7be0e84dbe82c65e"}
Jul 28 23:59:03.240331 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:59:03.240318    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-21449885-openshift-default-5f8f558744-xfsxw" event={"ID":"b71ce34b-61f9-46c0-b3b5-e51e193f4bc4","Type":"ContainerDied","Data":"f251fb979c3617e02e226a7a6a1efb9b2bcdfbc61ff5de878d02467fd39a935d"}
Jul 28 23:59:03.240525 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:59:03.240340    2572 scope.go:117] "RemoveContainer" containerID="f7c39b497325b8c61a80221c78e58c34eddf92c0486db7ed7be0e84dbe82c65e"
Jul 28 23:59:03.248521 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:59:03.248493    2572 scope.go:117] "RemoveContainer" containerID="1ba2e0dfd5976331b16e4284300bc3b54bb50926b12985c1a242df9dcb6a7cc8"
Jul 28 23:59:03.255912 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:59:03.255897    2572 scope.go:117] "RemoveContainer" containerID="f7c39b497325b8c61a80221c78e58c34eddf92c0486db7ed7be0e84dbe82c65e"
Jul 28 23:59:03.256167 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:59:03.256148    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f7c39b497325b8c61a80221c78e58c34eddf92c0486db7ed7be0e84dbe82c65e\": container with ID starting with f7c39b497325b8c61a80221c78e58c34eddf92c0486db7ed7be0e84dbe82c65e not found: ID does not exist" containerID="f7c39b497325b8c61a80221c78e58c34eddf92c0486db7ed7be0e84dbe82c65e"
Jul 28 23:59:03.256213 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:59:03.256177    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f7c39b497325b8c61a80221c78e58c34eddf92c0486db7ed7be0e84dbe82c65e"} err="failed to get container status \"f7c39b497325b8c61a80221c78e58c34eddf92c0486db7ed7be0e84dbe82c65e\": rpc error: code = NotFound desc = could not find container \"f7c39b497325b8c61a80221c78e58c34eddf92c0486db7ed7be0e84dbe82c65e\": container with ID starting with f7c39b497325b8c61a80221c78e58c34eddf92c0486db7ed7be0e84dbe82c65e not found: ID does not exist"
Jul 28 23:59:03.256213 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:59:03.256195    2572 scope.go:117] "RemoveContainer" containerID="1ba2e0dfd5976331b16e4284300bc3b54bb50926b12985c1a242df9dcb6a7cc8"
Jul 28 23:59:03.256406 ip-10-0-131-112 kubenswrapper[2572]: E0728 23:59:03.256391    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1ba2e0dfd5976331b16e4284300bc3b54bb50926b12985c1a242df9dcb6a7cc8\": container with ID starting with 1ba2e0dfd5976331b16e4284300bc3b54bb50926b12985c1a242df9dcb6a7cc8 not found: ID does not exist" containerID="1ba2e0dfd5976331b16e4284300bc3b54bb50926b12985c1a242df9dcb6a7cc8"
Jul 28 23:59:03.256450 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:59:03.256409    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1ba2e0dfd5976331b16e4284300bc3b54bb50926b12985c1a242df9dcb6a7cc8"} err="failed to get container status \"1ba2e0dfd5976331b16e4284300bc3b54bb50926b12985c1a242df9dcb6a7cc8\": rpc error: code = NotFound desc = could not find container \"1ba2e0dfd5976331b16e4284300bc3b54bb50926b12985c1a242df9dcb6a7cc8\": container with ID starting with 1ba2e0dfd5976331b16e4284300bc3b54bb50926b12985c1a242df9dcb6a7cc8 not found: ID does not exist"
Jul 28 23:59:03.328399 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:59:03.328374    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b71ce34b-61f9-46c0-b3b5-e51e193f4bc4-istio-envoy\") pod \"b71ce34b-61f9-46c0-b3b5-e51e193f4bc4\" (UID: \"b71ce34b-61f9-46c0-b3b5-e51e193f4bc4\") "
Jul 28 23:59:03.328505 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:59:03.328406    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b71ce34b-61f9-46c0-b3b5-e51e193f4bc4-istiod-ca-cert\") pod \"b71ce34b-61f9-46c0-b3b5-e51e193f4bc4\" (UID: \"b71ce34b-61f9-46c0-b3b5-e51e193f4bc4\") "
Jul 28 23:59:03.328505 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:59:03.328436    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6hzft\" (UniqueName: \"kubernetes.io/projected/b71ce34b-61f9-46c0-b3b5-e51e193f4bc4-kube-api-access-6hzft\") pod \"b71ce34b-61f9-46c0-b3b5-e51e193f4bc4\" (UID: \"b71ce34b-61f9-46c0-b3b5-e51e193f4bc4\") "
Jul 28 23:59:03.328505 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:59:03.328499    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b71ce34b-61f9-46c0-b3b5-e51e193f4bc4-workload-certs\") pod \"b71ce34b-61f9-46c0-b3b5-e51e193f4bc4\" (UID: \"b71ce34b-61f9-46c0-b3b5-e51e193f4bc4\") "
Jul 28 23:59:03.328658 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:59:03.328538    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b71ce34b-61f9-46c0-b3b5-e51e193f4bc4-istio-token\") pod \"b71ce34b-61f9-46c0-b3b5-e51e193f4bc4\" (UID: \"b71ce34b-61f9-46c0-b3b5-e51e193f4bc4\") "
Jul 28 23:59:03.328658 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:59:03.328581    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b71ce34b-61f9-46c0-b3b5-e51e193f4bc4-istio-podinfo\") pod \"b71ce34b-61f9-46c0-b3b5-e51e193f4bc4\" (UID: \"b71ce34b-61f9-46c0-b3b5-e51e193f4bc4\") "
Jul 28 23:59:03.328658 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:59:03.328651    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b71ce34b-61f9-46c0-b3b5-e51e193f4bc4-workload-socket\") pod \"b71ce34b-61f9-46c0-b3b5-e51e193f4bc4\" (UID: \"b71ce34b-61f9-46c0-b3b5-e51e193f4bc4\") "
Jul 28 23:59:03.328803 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:59:03.328680    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b71ce34b-61f9-46c0-b3b5-e51e193f4bc4-istio-data\") pod \"b71ce34b-61f9-46c0-b3b5-e51e193f4bc4\" (UID: \"b71ce34b-61f9-46c0-b3b5-e51e193f4bc4\") "
Jul 28 23:59:03.328803 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:59:03.328713    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b71ce34b-61f9-46c0-b3b5-e51e193f4bc4-credential-socket\") pod \"b71ce34b-61f9-46c0-b3b5-e51e193f4bc4\" (UID: \"b71ce34b-61f9-46c0-b3b5-e51e193f4bc4\") "
Jul 28 23:59:03.328886 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:59:03.328817    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b71ce34b-61f9-46c0-b3b5-e51e193f4bc4-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b71ce34b-61f9-46c0-b3b5-e51e193f4bc4" (UID: "b71ce34b-61f9-46c0-b3b5-e51e193f4bc4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 23:59:03.328886 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:59:03.328863    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b71ce34b-61f9-46c0-b3b5-e51e193f4bc4-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b71ce34b-61f9-46c0-b3b5-e51e193f4bc4" (UID: "b71ce34b-61f9-46c0-b3b5-e51e193f4bc4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 23:59:03.328994 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:59:03.328979    2572 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b71ce34b-61f9-46c0-b3b5-e51e193f4bc4-istiod-ca-cert\") on node \"ip-10-0-131-112.ec2.internal\" DevicePath \"\""
Jul 28 23:59:03.329049 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:59:03.328997    2572 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b71ce34b-61f9-46c0-b3b5-e51e193f4bc4-workload-certs\") on node \"ip-10-0-131-112.ec2.internal\" DevicePath \"\""
Jul 28 23:59:03.329168 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:59:03.329138    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b71ce34b-61f9-46c0-b3b5-e51e193f4bc4-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b71ce34b-61f9-46c0-b3b5-e51e193f4bc4" (UID: "b71ce34b-61f9-46c0-b3b5-e51e193f4bc4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 23:59:03.329669 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:59:03.329644    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b71ce34b-61f9-46c0-b3b5-e51e193f4bc4-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b71ce34b-61f9-46c0-b3b5-e51e193f4bc4" (UID: "b71ce34b-61f9-46c0-b3b5-e51e193f4bc4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 23:59:03.329842 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:59:03.329818    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b71ce34b-61f9-46c0-b3b5-e51e193f4bc4-istio-data" (OuterVolumeSpecName: "istio-data") pod "b71ce34b-61f9-46c0-b3b5-e51e193f4bc4" (UID: "b71ce34b-61f9-46c0-b3b5-e51e193f4bc4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 23:59:03.330591 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:59:03.330567    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b71ce34b-61f9-46c0-b3b5-e51e193f4bc4-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b71ce34b-61f9-46c0-b3b5-e51e193f4bc4" (UID: "b71ce34b-61f9-46c0-b3b5-e51e193f4bc4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 23:59:03.330707 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:59:03.330687    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b71ce34b-61f9-46c0-b3b5-e51e193f4bc4-kube-api-access-6hzft" (OuterVolumeSpecName: "kube-api-access-6hzft") pod "b71ce34b-61f9-46c0-b3b5-e51e193f4bc4" (UID: "b71ce34b-61f9-46c0-b3b5-e51e193f4bc4"). InnerVolumeSpecName "kube-api-access-6hzft". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 23:59:03.330905 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:59:03.330884    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b71ce34b-61f9-46c0-b3b5-e51e193f4bc4-istio-token" (OuterVolumeSpecName: "istio-token") pod "b71ce34b-61f9-46c0-b3b5-e51e193f4bc4" (UID: "b71ce34b-61f9-46c0-b3b5-e51e193f4bc4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 23:59:03.330991 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:59:03.330911    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b71ce34b-61f9-46c0-b3b5-e51e193f4bc4-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b71ce34b-61f9-46c0-b3b5-e51e193f4bc4" (UID: "b71ce34b-61f9-46c0-b3b5-e51e193f4bc4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 28 23:59:03.430322 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:59:03.430298    2572 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b71ce34b-61f9-46c0-b3b5-e51e193f4bc4-istio-data\") on node \"ip-10-0-131-112.ec2.internal\" DevicePath \"\""
Jul 28 23:59:03.430322 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:59:03.430317    2572 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b71ce34b-61f9-46c0-b3b5-e51e193f4bc4-credential-socket\") on node \"ip-10-0-131-112.ec2.internal\" DevicePath \"\""
Jul 28 23:59:03.430449 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:59:03.430326    2572 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b71ce34b-61f9-46c0-b3b5-e51e193f4bc4-istio-envoy\") on node \"ip-10-0-131-112.ec2.internal\" DevicePath \"\""
Jul 28 23:59:03.430449 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:59:03.430338    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6hzft\" (UniqueName: \"kubernetes.io/projected/b71ce34b-61f9-46c0-b3b5-e51e193f4bc4-kube-api-access-6hzft\") on node \"ip-10-0-131-112.ec2.internal\" DevicePath \"\""
Jul 28 23:59:03.430449 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:59:03.430348    2572 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b71ce34b-61f9-46c0-b3b5-e51e193f4bc4-istio-token\") on node \"ip-10-0-131-112.ec2.internal\" DevicePath \"\""
Jul 28 23:59:03.430449 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:59:03.430355    2572 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b71ce34b-61f9-46c0-b3b5-e51e193f4bc4-istio-podinfo\") on node \"ip-10-0-131-112.ec2.internal\" DevicePath \"\""
Jul 28 23:59:03.430449 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:59:03.430363    2572 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b71ce34b-61f9-46c0-b3b5-e51e193f4bc4-workload-socket\") on node \"ip-10-0-131-112.ec2.internal\" DevicePath \"\""
Jul 28 23:59:03.565338 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:59:03.565311    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-21449885-openshift-default-5f8f558744-xfsxw"]
Jul 28 23:59:03.569152 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:59:03.569127    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-21449885-openshift-default-5f8f558744-xfsxw"]
Jul 28 23:59:03.865941 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:59:03.865870    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b71ce34b-61f9-46c0-b3b5-e51e193f4bc4" path="/var/lib/kubelet/pods/b71ce34b-61f9-46c0-b3b5-e51e193f4bc4/volumes"
Jul 28 23:59:03.938012 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:59:03.937983    2572 scope.go:117] "RemoveContainer" containerID="b2f825e49a093ccbcfdcffef3f322ff032e5e497638f243fcdde674e04a25b78"
Jul 28 23:59:52.833689 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:59:52.832891    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-c418f4-openshift-default-6b7cd5b44b-mjw86"]
Jul 28 23:59:52.833689 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:59:52.833576    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9397ce55-670f-4274-a76a-13a2117a0ba5" containerName="maas-api"
Jul 28 23:59:52.833689 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:59:52.833602    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="9397ce55-670f-4274-a76a-13a2117a0ba5" containerName="maas-api"
Jul 28 23:59:52.834740 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:59:52.833658    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b71ce34b-61f9-46c0-b3b5-e51e193f4bc4" containerName="istio-proxy"
Jul 28 23:59:52.834740 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:59:52.834331    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="b71ce34b-61f9-46c0-b3b5-e51e193f4bc4" containerName="istio-proxy"
Jul 28 23:59:52.834740 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:59:52.834353    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b71ce34b-61f9-46c0-b3b5-e51e193f4bc4" containerName="istio-proxy"
Jul 28 23:59:52.834740 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:59:52.834363    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="b71ce34b-61f9-46c0-b3b5-e51e193f4bc4" containerName="istio-proxy"
Jul 28 23:59:52.834740 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:59:52.834546    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="b71ce34b-61f9-46c0-b3b5-e51e193f4bc4" containerName="istio-proxy"
Jul 28 23:59:52.834740 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:59:52.834558    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="b71ce34b-61f9-46c0-b3b5-e51e193f4bc4" containerName="istio-proxy"
Jul 28 23:59:52.834740 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:59:52.834569    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="9397ce55-670f-4274-a76a-13a2117a0ba5" containerName="maas-api"
Jul 28 23:59:52.838729 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:59:52.838704    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-c418f4-openshift-default-6b7cd5b44b-mjw86"
Jul 28 23:59:52.842161 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:59:52.842133    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-c418f4-openshift-default-dockercfg-jp274\""
Jul 28 23:59:52.844036 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:59:52.844009    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-c418f4-openshift-default-6b7cd5b44b-mjw86"]
Jul 28 23:59:52.932403 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:59:52.932364    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff7d3e74-15f3-43e8-b5b7-f618c10d3630-workload-socket\") pod \"e2e-inf-a-c418f4-openshift-default-6b7cd5b44b-mjw86\" (UID: \"ff7d3e74-15f3-43e8-b5b7-f618c10d3630\") " pod="openshift-ingress/e2e-inf-a-c418f4-openshift-default-6b7cd5b44b-mjw86"
Jul 28 23:59:52.932403 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:59:52.932408    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ff7d3e74-15f3-43e8-b5b7-f618c10d3630-istiod-ca-cert\") pod \"e2e-inf-a-c418f4-openshift-default-6b7cd5b44b-mjw86\" (UID: \"ff7d3e74-15f3-43e8-b5b7-f618c10d3630\") " pod="openshift-ingress/e2e-inf-a-c418f4-openshift-default-6b7cd5b44b-mjw86"
Jul 28 23:59:52.932735 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:59:52.932478    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ff7d3e74-15f3-43e8-b5b7-f618c10d3630-istio-envoy\") pod \"e2e-inf-a-c418f4-openshift-default-6b7cd5b44b-mjw86\" (UID: \"ff7d3e74-15f3-43e8-b5b7-f618c10d3630\") " pod="openshift-ingress/e2e-inf-a-c418f4-openshift-default-6b7cd5b44b-mjw86"
Jul 28 23:59:52.932735 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:59:52.932527    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jks7h\" (UniqueName: \"kubernetes.io/projected/ff7d3e74-15f3-43e8-b5b7-f618c10d3630-kube-api-access-jks7h\") pod \"e2e-inf-a-c418f4-openshift-default-6b7cd5b44b-mjw86\" (UID: \"ff7d3e74-15f3-43e8-b5b7-f618c10d3630\") " pod="openshift-ingress/e2e-inf-a-c418f4-openshift-default-6b7cd5b44b-mjw86"
Jul 28 23:59:52.932735 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:59:52.932565    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ff7d3e74-15f3-43e8-b5b7-f618c10d3630-istio-token\") pod \"e2e-inf-a-c418f4-openshift-default-6b7cd5b44b-mjw86\" (UID: \"ff7d3e74-15f3-43e8-b5b7-f618c10d3630\") " pod="openshift-ingress/e2e-inf-a-c418f4-openshift-default-6b7cd5b44b-mjw86"
Jul 28 23:59:52.932735 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:59:52.932660    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ff7d3e74-15f3-43e8-b5b7-f618c10d3630-istio-podinfo\") pod \"e2e-inf-a-c418f4-openshift-default-6b7cd5b44b-mjw86\" (UID: \"ff7d3e74-15f3-43e8-b5b7-f618c10d3630\") " pod="openshift-ingress/e2e-inf-a-c418f4-openshift-default-6b7cd5b44b-mjw86"
Jul 28 23:59:52.932735 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:59:52.932691    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff7d3e74-15f3-43e8-b5b7-f618c10d3630-credential-socket\") pod \"e2e-inf-a-c418f4-openshift-default-6b7cd5b44b-mjw86\" (UID: \"ff7d3e74-15f3-43e8-b5b7-f618c10d3630\") " pod="openshift-ingress/e2e-inf-a-c418f4-openshift-default-6b7cd5b44b-mjw86"
Jul 28 23:59:52.932735 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:59:52.932728    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ff7d3e74-15f3-43e8-b5b7-f618c10d3630-workload-certs\") pod \"e2e-inf-a-c418f4-openshift-default-6b7cd5b44b-mjw86\" (UID: \"ff7d3e74-15f3-43e8-b5b7-f618c10d3630\") " pod="openshift-ingress/e2e-inf-a-c418f4-openshift-default-6b7cd5b44b-mjw86"
Jul 28 23:59:52.932949 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:59:52.932797    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ff7d3e74-15f3-43e8-b5b7-f618c10d3630-istio-data\") pod \"e2e-inf-a-c418f4-openshift-default-6b7cd5b44b-mjw86\" (UID: \"ff7d3e74-15f3-43e8-b5b7-f618c10d3630\") " pod="openshift-ingress/e2e-inf-a-c418f4-openshift-default-6b7cd5b44b-mjw86"
Jul 28 23:59:53.033995 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:59:53.033962    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ff7d3e74-15f3-43e8-b5b7-f618c10d3630-istio-data\") pod \"e2e-inf-a-c418f4-openshift-default-6b7cd5b44b-mjw86\" (UID: \"ff7d3e74-15f3-43e8-b5b7-f618c10d3630\") " pod="openshift-ingress/e2e-inf-a-c418f4-openshift-default-6b7cd5b44b-mjw86"
Jul 28 23:59:53.034178 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:59:53.034026    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff7d3e74-15f3-43e8-b5b7-f618c10d3630-workload-socket\") pod \"e2e-inf-a-c418f4-openshift-default-6b7cd5b44b-mjw86\" (UID: \"ff7d3e74-15f3-43e8-b5b7-f618c10d3630\") " pod="openshift-ingress/e2e-inf-a-c418f4-openshift-default-6b7cd5b44b-mjw86"
Jul 28 23:59:53.034178 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:59:53.034061    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ff7d3e74-15f3-43e8-b5b7-f618c10d3630-istiod-ca-cert\") pod \"e2e-inf-a-c418f4-openshift-default-6b7cd5b44b-mjw86\" (UID: \"ff7d3e74-15f3-43e8-b5b7-f618c10d3630\") " pod="openshift-ingress/e2e-inf-a-c418f4-openshift-default-6b7cd5b44b-mjw86"
Jul 28 23:59:53.034178 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:59:53.034090    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ff7d3e74-15f3-43e8-b5b7-f618c10d3630-istio-envoy\") pod \"e2e-inf-a-c418f4-openshift-default-6b7cd5b44b-mjw86\" (UID: \"ff7d3e74-15f3-43e8-b5b7-f618c10d3630\") " pod="openshift-ingress/e2e-inf-a-c418f4-openshift-default-6b7cd5b44b-mjw86"
Jul 28 23:59:53.034178 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:59:53.034117    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jks7h\" (UniqueName: \"kubernetes.io/projected/ff7d3e74-15f3-43e8-b5b7-f618c10d3630-kube-api-access-jks7h\") pod \"e2e-inf-a-c418f4-openshift-default-6b7cd5b44b-mjw86\" (UID: \"ff7d3e74-15f3-43e8-b5b7-f618c10d3630\") " pod="openshift-ingress/e2e-inf-a-c418f4-openshift-default-6b7cd5b44b-mjw86"
Jul 28 23:59:53.034178 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:59:53.034147    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ff7d3e74-15f3-43e8-b5b7-f618c10d3630-istio-token\") pod \"e2e-inf-a-c418f4-openshift-default-6b7cd5b44b-mjw86\" (UID: \"ff7d3e74-15f3-43e8-b5b7-f618c10d3630\") " pod="openshift-ingress/e2e-inf-a-c418f4-openshift-default-6b7cd5b44b-mjw86"
Jul 28 23:59:53.034455 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:59:53.034189    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ff7d3e74-15f3-43e8-b5b7-f618c10d3630-istio-podinfo\") pod \"e2e-inf-a-c418f4-openshift-default-6b7cd5b44b-mjw86\" (UID: \"ff7d3e74-15f3-43e8-b5b7-f618c10d3630\") " pod="openshift-ingress/e2e-inf-a-c418f4-openshift-default-6b7cd5b44b-mjw86"
Jul 28 23:59:53.034455 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:59:53.034218    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff7d3e74-15f3-43e8-b5b7-f618c10d3630-credential-socket\") pod \"e2e-inf-a-c418f4-openshift-default-6b7cd5b44b-mjw86\" (UID: \"ff7d3e74-15f3-43e8-b5b7-f618c10d3630\") " pod="openshift-ingress/e2e-inf-a-c418f4-openshift-default-6b7cd5b44b-mjw86"
Jul 28 23:59:53.034455 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:59:53.034260    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ff7d3e74-15f3-43e8-b5b7-f618c10d3630-workload-certs\") pod \"e2e-inf-a-c418f4-openshift-default-6b7cd5b44b-mjw86\" (UID: \"ff7d3e74-15f3-43e8-b5b7-f618c10d3630\") " pod="openshift-ingress/e2e-inf-a-c418f4-openshift-default-6b7cd5b44b-mjw86"
Jul 28 23:59:53.034616 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:59:53.034464    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ff7d3e74-15f3-43e8-b5b7-f618c10d3630-istio-data\") pod \"e2e-inf-a-c418f4-openshift-default-6b7cd5b44b-mjw86\" (UID: \"ff7d3e74-15f3-43e8-b5b7-f618c10d3630\") " pod="openshift-ingress/e2e-inf-a-c418f4-openshift-default-6b7cd5b44b-mjw86"
Jul 28 23:59:53.034867 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:59:53.034836    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff7d3e74-15f3-43e8-b5b7-f618c10d3630-credential-socket\") pod \"e2e-inf-a-c418f4-openshift-default-6b7cd5b44b-mjw86\" (UID: \"ff7d3e74-15f3-43e8-b5b7-f618c10d3630\") " pod="openshift-ingress/e2e-inf-a-c418f4-openshift-default-6b7cd5b44b-mjw86"
Jul 28 23:59:53.034984 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:59:53.034927    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ff7d3e74-15f3-43e8-b5b7-f618c10d3630-workload-certs\") pod \"e2e-inf-a-c418f4-openshift-default-6b7cd5b44b-mjw86\" (UID: \"ff7d3e74-15f3-43e8-b5b7-f618c10d3630\") " pod="openshift-ingress/e2e-inf-a-c418f4-openshift-default-6b7cd5b44b-mjw86"
Jul 28 23:59:53.034984 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:59:53.034976    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ff7d3e74-15f3-43e8-b5b7-f618c10d3630-istiod-ca-cert\") pod \"e2e-inf-a-c418f4-openshift-default-6b7cd5b44b-mjw86\" (UID: \"ff7d3e74-15f3-43e8-b5b7-f618c10d3630\") " pod="openshift-ingress/e2e-inf-a-c418f4-openshift-default-6b7cd5b44b-mjw86"
Jul 28 23:59:53.035087 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:59:53.034979    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff7d3e74-15f3-43e8-b5b7-f618c10d3630-workload-socket\") pod \"e2e-inf-a-c418f4-openshift-default-6b7cd5b44b-mjw86\" (UID: \"ff7d3e74-15f3-43e8-b5b7-f618c10d3630\") " pod="openshift-ingress/e2e-inf-a-c418f4-openshift-default-6b7cd5b44b-mjw86"
Jul 28 23:59:53.036598 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:59:53.036573    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ff7d3e74-15f3-43e8-b5b7-f618c10d3630-istio-envoy\") pod \"e2e-inf-a-c418f4-openshift-default-6b7cd5b44b-mjw86\" (UID: \"ff7d3e74-15f3-43e8-b5b7-f618c10d3630\") " pod="openshift-ingress/e2e-inf-a-c418f4-openshift-default-6b7cd5b44b-mjw86"
Jul 28 23:59:53.036707 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:59:53.036692    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ff7d3e74-15f3-43e8-b5b7-f618c10d3630-istio-podinfo\") pod \"e2e-inf-a-c418f4-openshift-default-6b7cd5b44b-mjw86\" (UID: \"ff7d3e74-15f3-43e8-b5b7-f618c10d3630\") " pod="openshift-ingress/e2e-inf-a-c418f4-openshift-default-6b7cd5b44b-mjw86"
Jul 28 23:59:53.041798 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:59:53.041772    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ff7d3e74-15f3-43e8-b5b7-f618c10d3630-istio-token\") pod \"e2e-inf-a-c418f4-openshift-default-6b7cd5b44b-mjw86\" (UID: \"ff7d3e74-15f3-43e8-b5b7-f618c10d3630\") " pod="openshift-ingress/e2e-inf-a-c418f4-openshift-default-6b7cd5b44b-mjw86"
Jul 28 23:59:53.042146 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:59:53.042127    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jks7h\" (UniqueName: \"kubernetes.io/projected/ff7d3e74-15f3-43e8-b5b7-f618c10d3630-kube-api-access-jks7h\") pod \"e2e-inf-a-c418f4-openshift-default-6b7cd5b44b-mjw86\" (UID: \"ff7d3e74-15f3-43e8-b5b7-f618c10d3630\") " pod="openshift-ingress/e2e-inf-a-c418f4-openshift-default-6b7cd5b44b-mjw86"
Jul 28 23:59:53.151114 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:59:53.151026    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-c418f4-openshift-default-6b7cd5b44b-mjw86"
Jul 28 23:59:53.289029 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:59:53.289005    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-c418f4-openshift-default-6b7cd5b44b-mjw86"]
Jul 28 23:59:53.291069 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:59:53.291033    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff7d3e74_15f3_43e8_b5b7_f618c10d3630.slice/crio-bcbfd2fb648d9e56b23c1dd9b8030e1c53f9ffd616ea6e8a7b19974f3b4d747f WatchSource:0}: Error finding container bcbfd2fb648d9e56b23c1dd9b8030e1c53f9ffd616ea6e8a7b19974f3b4d747f: Status 404 returned error can't find the container with id bcbfd2fb648d9e56b23c1dd9b8030e1c53f9ffd616ea6e8a7b19974f3b4d747f
Jul 28 23:59:53.293109 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:59:53.293080    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 23:59:53.293185 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:59:53.293143    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 23:59:53.293185 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:59:53.293169    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 28 23:59:53.464197 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:59:53.464111    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-c418f4-openshift-default-6b7cd5b44b-mjw86" event={"ID":"ff7d3e74-15f3-43e8-b5b7-f618c10d3630","Type":"ContainerStarted","Data":"fcdcbb52514f55f725b16936203c2d53b8467d3f785967eae78fb4111e7e881b"}
Jul 28 23:59:53.464197 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:59:53.464168    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-c418f4-openshift-default-6b7cd5b44b-mjw86" event={"ID":"ff7d3e74-15f3-43e8-b5b7-f618c10d3630","Type":"ContainerStarted","Data":"bcbfd2fb648d9e56b23c1dd9b8030e1c53f9ffd616ea6e8a7b19974f3b4d747f"}
Jul 28 23:59:54.151401 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:59:54.151357    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-c418f4-openshift-default-6b7cd5b44b-mjw86"
Jul 28 23:59:54.156301 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:59:54.156277    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-c418f4-openshift-default-6b7cd5b44b-mjw86"
Jul 28 23:59:54.177896 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:59:54.177846    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-c418f4-openshift-default-6b7cd5b44b-mjw86" podStartSLOduration=2.177831151 podStartE2EDuration="2.177831151s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 23:59:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 23:59:53.484280652 +0000 UTC m=+2270.176955185" watchObservedRunningTime="2026-07-28 23:59:54.177831151 +0000 UTC m=+2270.870505659"
Jul 28 23:59:54.468895 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:59:54.468801    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-c418f4-openshift-default-6b7cd5b44b-mjw86"
Jul 28 23:59:54.469834 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:59:54.469813    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-c418f4-openshift-default-6b7cd5b44b-mjw86"
Jul 28 23:59:55.591085 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:59:55.591040    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-c418f4-7b76dd6dc4-wd97m"]
Jul 28 23:59:55.594054 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:59:55.594034    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c418f4-7b76dd6dc4-wd97m"
Jul 28 23:59:55.596326 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:59:55.596307    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-c418f4-serving-cert\""
Jul 28 23:59:55.605314 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:59:55.605288    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-c418f4-7b76dd6dc4-wd97m"]
Jul 28 23:59:55.761395 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:59:55.761353    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2cjkb\" (UniqueName: \"kubernetes.io/projected/08cfcb0d-5f8a-4de1-aaa3-15b755b160aa-kube-api-access-2cjkb\") pod \"maas-api-e2e-inf-a-c418f4-7b76dd6dc4-wd97m\" (UID: \"08cfcb0d-5f8a-4de1-aaa3-15b755b160aa\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c418f4-7b76dd6dc4-wd97m"
Jul 28 23:59:55.761576 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:59:55.761403    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/08cfcb0d-5f8a-4de1-aaa3-15b755b160aa-maas-api-tls\") pod \"maas-api-e2e-inf-a-c418f4-7b76dd6dc4-wd97m\" (UID: \"08cfcb0d-5f8a-4de1-aaa3-15b755b160aa\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c418f4-7b76dd6dc4-wd97m"
Jul 28 23:59:55.761576 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:59:55.761439    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/08cfcb0d-5f8a-4de1-aaa3-15b755b160aa-openshift-service-ca\") pod \"maas-api-e2e-inf-a-c418f4-7b76dd6dc4-wd97m\" (UID: \"08cfcb0d-5f8a-4de1-aaa3-15b755b160aa\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c418f4-7b76dd6dc4-wd97m"
Jul 28 23:59:55.862824 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:59:55.862741    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2cjkb\" (UniqueName: \"kubernetes.io/projected/08cfcb0d-5f8a-4de1-aaa3-15b755b160aa-kube-api-access-2cjkb\") pod \"maas-api-e2e-inf-a-c418f4-7b76dd6dc4-wd97m\" (UID: \"08cfcb0d-5f8a-4de1-aaa3-15b755b160aa\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c418f4-7b76dd6dc4-wd97m"
Jul 28 23:59:55.862824 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:59:55.862778    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/08cfcb0d-5f8a-4de1-aaa3-15b755b160aa-maas-api-tls\") pod \"maas-api-e2e-inf-a-c418f4-7b76dd6dc4-wd97m\" (UID: \"08cfcb0d-5f8a-4de1-aaa3-15b755b160aa\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c418f4-7b76dd6dc4-wd97m"
Jul 28 23:59:55.862824 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:59:55.862803    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/08cfcb0d-5f8a-4de1-aaa3-15b755b160aa-openshift-service-ca\") pod \"maas-api-e2e-inf-a-c418f4-7b76dd6dc4-wd97m\" (UID: \"08cfcb0d-5f8a-4de1-aaa3-15b755b160aa\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c418f4-7b76dd6dc4-wd97m"
Jul 28 23:59:55.863660 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:59:55.863619    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/08cfcb0d-5f8a-4de1-aaa3-15b755b160aa-openshift-service-ca\") pod \"maas-api-e2e-inf-a-c418f4-7b76dd6dc4-wd97m\" (UID: \"08cfcb0d-5f8a-4de1-aaa3-15b755b160aa\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c418f4-7b76dd6dc4-wd97m"
Jul 28 23:59:55.865320 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:59:55.865297    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/08cfcb0d-5f8a-4de1-aaa3-15b755b160aa-maas-api-tls\") pod \"maas-api-e2e-inf-a-c418f4-7b76dd6dc4-wd97m\" (UID: \"08cfcb0d-5f8a-4de1-aaa3-15b755b160aa\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c418f4-7b76dd6dc4-wd97m"
Jul 28 23:59:55.870664 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:59:55.870643    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2cjkb\" (UniqueName: \"kubernetes.io/projected/08cfcb0d-5f8a-4de1-aaa3-15b755b160aa-kube-api-access-2cjkb\") pod \"maas-api-e2e-inf-a-c418f4-7b76dd6dc4-wd97m\" (UID: \"08cfcb0d-5f8a-4de1-aaa3-15b755b160aa\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c418f4-7b76dd6dc4-wd97m"
Jul 28 23:59:55.905601 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:59:55.905568    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c418f4-7b76dd6dc4-wd97m"
Jul 28 23:59:56.036116 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:59:56.036087    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-c418f4-7b76dd6dc4-wd97m"]
Jul 28 23:59:56.037890 ip-10-0-131-112 kubenswrapper[2572]: W0728 23:59:56.037861    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08cfcb0d_5f8a_4de1_aaa3_15b755b160aa.slice/crio-1458542604d2b8fe7753a6b7a946c1139b8d6fa21de442ec84ab689e2362f7f2 WatchSource:0}: Error finding container 1458542604d2b8fe7753a6b7a946c1139b8d6fa21de442ec84ab689e2362f7f2: Status 404 returned error can't find the container with id 1458542604d2b8fe7753a6b7a946c1139b8d6fa21de442ec84ab689e2362f7f2
Jul 28 23:59:56.479676 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:59:56.479649    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c418f4-7b76dd6dc4-wd97m" event={"ID":"08cfcb0d-5f8a-4de1-aaa3-15b755b160aa","Type":"ContainerStarted","Data":"1458542604d2b8fe7753a6b7a946c1139b8d6fa21de442ec84ab689e2362f7f2"}
Jul 28 23:59:57.485540 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:59:57.485491    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c418f4-7b76dd6dc4-wd97m" event={"ID":"08cfcb0d-5f8a-4de1-aaa3-15b755b160aa","Type":"ContainerStarted","Data":"5d14406c84e9ce0578ac4805706ce4853b805a16620911496dd5629a885290dc"}
Jul 28 23:59:57.485947 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:59:57.485603    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c418f4-7b76dd6dc4-wd97m"
Jul 28 23:59:57.501902 ip-10-0-131-112 kubenswrapper[2572]: I0728 23:59:57.501849    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c418f4-7b76dd6dc4-wd97m" podStartSLOduration=2.119750823 podStartE2EDuration="2.501833154s" totalImagesPullingTime="382.082331ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 23:59:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 23:59:57.500111826 +0000 UTC m=+2274.192786359" watchObservedRunningTime="2026-07-28 23:59:57.501833154 +0000 UTC m=+2274.194507662"
Jul 29 00:00:03.495124 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:00:03.495088    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c418f4-7b76dd6dc4-wd97m"
Jul 29 00:00:10.510828 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:00:10.510783    2572 patch_prober.go:28] interesting pod/e2e-inf-a-c418f4-openshift-default-6b7cd5b44b-mjw86 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 29 00:00:10.511239 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:00:10.510853    2572 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-c418f4-openshift-default-6b7cd5b44b-mjw86" podUID="ff7d3e74-15f3-43e8-b5b7-f618c10d3630" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 29 00:00:11.558660 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:00:11.558606    2572 generic.go:356] "Generic (PLEG): container finished" podID="ff7d3e74-15f3-43e8-b5b7-f618c10d3630" containerID="fcdcbb52514f55f725b16936203c2d53b8467d3f785967eae78fb4111e7e881b" exitCode=0
Jul 29 00:00:11.559219 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:00:11.558683    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-c418f4-openshift-default-6b7cd5b44b-mjw86" event={"ID":"ff7d3e74-15f3-43e8-b5b7-f618c10d3630","Type":"ContainerDied","Data":"fcdcbb52514f55f725b16936203c2d53b8467d3f785967eae78fb4111e7e881b"}
Jul 29 00:00:11.559219 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:00:11.559202    2572 scope.go:117] "RemoveContainer" containerID="fcdcbb52514f55f725b16936203c2d53b8467d3f785967eae78fb4111e7e881b"
Jul 29 00:00:11.560259 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:00:11.560227    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 29 00:00:11.560372 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:00:11.560294    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 29 00:00:11.560372 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:00:11.560329    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 29 00:00:12.565014 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:00:12.564980    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-c418f4-openshift-default-6b7cd5b44b-mjw86" event={"ID":"ff7d3e74-15f3-43e8-b5b7-f618c10d3630","Type":"ContainerStarted","Data":"68c2be1163f498051b4ceef721aadab7727ba4d0ddce496d046b79ef617b2c69"}
Jul 29 00:00:13.151661 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:00:13.151606    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-c418f4-openshift-default-6b7cd5b44b-mjw86"
Jul 29 00:00:13.151909 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:00:13.151886    2572 patch_prober.go:28] interesting pod/e2e-inf-a-c418f4-openshift-default-6b7cd5b44b-mjw86 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body=
Jul 29 00:00:13.151967 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:00:13.151951    2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-c418f4-openshift-default-6b7cd5b44b-mjw86" podUID="ff7d3e74-15f3-43e8-b5b7-f618c10d3630" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused"
Jul 29 00:00:14.151656 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:00:14.151599    2572 patch_prober.go:28] interesting pod/e2e-inf-a-c418f4-openshift-default-6b7cd5b44b-mjw86 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body=
Jul 29 00:00:14.152075 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:00:14.151690    2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-c418f4-openshift-default-6b7cd5b44b-mjw86" podUID="ff7d3e74-15f3-43e8-b5b7-f618c10d3630" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused"
Jul 29 00:00:15.151998 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:00:15.151955    2572 patch_prober.go:28] interesting pod/e2e-inf-a-c418f4-openshift-default-6b7cd5b44b-mjw86 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body=
Jul 29 00:00:15.152400 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:00:15.152017    2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-c418f4-openshift-default-6b7cd5b44b-mjw86" podUID="ff7d3e74-15f3-43e8-b5b7-f618c10d3630" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused"
Jul 29 00:00:16.152009 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:00:16.151968    2572 patch_prober.go:28] interesting pod/e2e-inf-a-c418f4-openshift-default-6b7cd5b44b-mjw86 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body=
Jul 29 00:00:16.152411 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:00:16.152025    2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-c418f4-openshift-default-6b7cd5b44b-mjw86" podUID="ff7d3e74-15f3-43e8-b5b7-f618c10d3630" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused"
Jul 29 00:00:17.151868 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:00:17.151829    2572 patch_prober.go:28] interesting pod/e2e-inf-a-c418f4-openshift-default-6b7cd5b44b-mjw86 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body=
Jul 29 00:00:17.152045 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:00:17.151892    2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-c418f4-openshift-default-6b7cd5b44b-mjw86" podUID="ff7d3e74-15f3-43e8-b5b7-f618c10d3630" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused"
Jul 29 00:00:18.154675 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:00:18.154645    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-c418f4-openshift-default-6b7cd5b44b-mjw86"
Jul 29 00:00:18.155085 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:00:18.155008    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-c418f4-openshift-default-6b7cd5b44b-mjw86"
Jul 29 00:00:18.155589 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:00:18.155573    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-c418f4-openshift-default-6b7cd5b44b-mjw86"
Jul 29 00:00:20.983487 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:00:20.983451    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-b178a9-openshift-default-9c5949cd8-nsrxg"]
Jul 29 00:00:20.985916 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:00:20.985896    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-b178a9-openshift-default-9c5949cd8-nsrxg"
Jul 29 00:00:20.988527 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:00:20.988504    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-b178a9-openshift-default-dockercfg-czmgs\""
Jul 29 00:00:20.999316 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:00:20.999290    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-b178a9-openshift-default-9c5949cd8-nsrxg"]
Jul 29 00:00:21.092840 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:00:21.092794    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8226d3d3-9e43-4d31-a54a-bdf4efc87a57-credential-socket\") pod \"e2e-inf-b-b178a9-openshift-default-9c5949cd8-nsrxg\" (UID: \"8226d3d3-9e43-4d31-a54a-bdf4efc87a57\") " pod="openshift-ingress/e2e-inf-b-b178a9-openshift-default-9c5949cd8-nsrxg"
Jul 29 00:00:21.093032 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:00:21.092869    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n2xck\" (UniqueName: \"kubernetes.io/projected/8226d3d3-9e43-4d31-a54a-bdf4efc87a57-kube-api-access-n2xck\") pod \"e2e-inf-b-b178a9-openshift-default-9c5949cd8-nsrxg\" (UID: \"8226d3d3-9e43-4d31-a54a-bdf4efc87a57\") " pod="openshift-ingress/e2e-inf-b-b178a9-openshift-default-9c5949cd8-nsrxg"
Jul 29 00:00:21.093032 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:00:21.092923    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8226d3d3-9e43-4d31-a54a-bdf4efc87a57-istiod-ca-cert\") pod \"e2e-inf-b-b178a9-openshift-default-9c5949cd8-nsrxg\" (UID: \"8226d3d3-9e43-4d31-a54a-bdf4efc87a57\") " pod="openshift-ingress/e2e-inf-b-b178a9-openshift-default-9c5949cd8-nsrxg"
Jul 29 00:00:21.093032 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:00:21.092952    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8226d3d3-9e43-4d31-a54a-bdf4efc87a57-istio-data\") pod \"e2e-inf-b-b178a9-openshift-default-9c5949cd8-nsrxg\" (UID: \"8226d3d3-9e43-4d31-a54a-bdf4efc87a57\") " pod="openshift-ingress/e2e-inf-b-b178a9-openshift-default-9c5949cd8-nsrxg"
Jul 29 00:00:21.093032 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:00:21.093008    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8226d3d3-9e43-4d31-a54a-bdf4efc87a57-workload-socket\") pod \"e2e-inf-b-b178a9-openshift-default-9c5949cd8-nsrxg\" (UID: \"8226d3d3-9e43-4d31-a54a-bdf4efc87a57\") " pod="openshift-ingress/e2e-inf-b-b178a9-openshift-default-9c5949cd8-nsrxg"
Jul 29 00:00:21.093250 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:00:21.093032    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8226d3d3-9e43-4d31-a54a-bdf4efc87a57-workload-certs\") pod \"e2e-inf-b-b178a9-openshift-default-9c5949cd8-nsrxg\" (UID: \"8226d3d3-9e43-4d31-a54a-bdf4efc87a57\") " pod="openshift-ingress/e2e-inf-b-b178a9-openshift-default-9c5949cd8-nsrxg"
Jul 29 00:00:21.093250 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:00:21.093061    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8226d3d3-9e43-4d31-a54a-bdf4efc87a57-istio-podinfo\") pod \"e2e-inf-b-b178a9-openshift-default-9c5949cd8-nsrxg\" (UID: \"8226d3d3-9e43-4d31-a54a-bdf4efc87a57\") " pod="openshift-ingress/e2e-inf-b-b178a9-openshift-default-9c5949cd8-nsrxg"
Jul 29 00:00:21.093250 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:00:21.093097    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8226d3d3-9e43-4d31-a54a-bdf4efc87a57-istio-token\") pod \"e2e-inf-b-b178a9-openshift-default-9c5949cd8-nsrxg\" (UID: \"8226d3d3-9e43-4d31-a54a-bdf4efc87a57\") " pod="openshift-ingress/e2e-inf-b-b178a9-openshift-default-9c5949cd8-nsrxg"
Jul 29 00:00:21.093250 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:00:21.093126    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8226d3d3-9e43-4d31-a54a-bdf4efc87a57-istio-envoy\") pod \"e2e-inf-b-b178a9-openshift-default-9c5949cd8-nsrxg\" (UID: \"8226d3d3-9e43-4d31-a54a-bdf4efc87a57\") " pod="openshift-ingress/e2e-inf-b-b178a9-openshift-default-9c5949cd8-nsrxg"
Jul 29 00:00:21.194197 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:00:21.194152    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8226d3d3-9e43-4d31-a54a-bdf4efc87a57-istio-envoy\") pod \"e2e-inf-b-b178a9-openshift-default-9c5949cd8-nsrxg\" (UID: \"8226d3d3-9e43-4d31-a54a-bdf4efc87a57\") " pod="openshift-ingress/e2e-inf-b-b178a9-openshift-default-9c5949cd8-nsrxg"
Jul 29 00:00:21.194384 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:00:21.194221    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8226d3d3-9e43-4d31-a54a-bdf4efc87a57-credential-socket\") pod \"e2e-inf-b-b178a9-openshift-default-9c5949cd8-nsrxg\" (UID: \"8226d3d3-9e43-4d31-a54a-bdf4efc87a57\") " pod="openshift-ingress/e2e-inf-b-b178a9-openshift-default-9c5949cd8-nsrxg"
Jul 29 00:00:21.194384 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:00:21.194264    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n2xck\" (UniqueName: \"kubernetes.io/projected/8226d3d3-9e43-4d31-a54a-bdf4efc87a57-kube-api-access-n2xck\") pod \"e2e-inf-b-b178a9-openshift-default-9c5949cd8-nsrxg\" (UID: \"8226d3d3-9e43-4d31-a54a-bdf4efc87a57\") " pod="openshift-ingress/e2e-inf-b-b178a9-openshift-default-9c5949cd8-nsrxg"
Jul 29 00:00:21.194384 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:00:21.194313    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8226d3d3-9e43-4d31-a54a-bdf4efc87a57-istiod-ca-cert\") pod \"e2e-inf-b-b178a9-openshift-default-9c5949cd8-nsrxg\" (UID: \"8226d3d3-9e43-4d31-a54a-bdf4efc87a57\") " pod="openshift-ingress/e2e-inf-b-b178a9-openshift-default-9c5949cd8-nsrxg"
Jul 29 00:00:21.194384 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:00:21.194343    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8226d3d3-9e43-4d31-a54a-bdf4efc87a57-istio-data\") pod \"e2e-inf-b-b178a9-openshift-default-9c5949cd8-nsrxg\" (UID: \"8226d3d3-9e43-4d31-a54a-bdf4efc87a57\") " pod="openshift-ingress/e2e-inf-b-b178a9-openshift-default-9c5949cd8-nsrxg"
Jul 29 00:00:21.194584 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:00:21.194425    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8226d3d3-9e43-4d31-a54a-bdf4efc87a57-workload-socket\") pod \"e2e-inf-b-b178a9-openshift-default-9c5949cd8-nsrxg\" (UID: \"8226d3d3-9e43-4d31-a54a-bdf4efc87a57\") " pod="openshift-ingress/e2e-inf-b-b178a9-openshift-default-9c5949cd8-nsrxg"
Jul 29 00:00:21.194584 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:00:21.194464    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8226d3d3-9e43-4d31-a54a-bdf4efc87a57-workload-certs\") pod \"e2e-inf-b-b178a9-openshift-default-9c5949cd8-nsrxg\" (UID: \"8226d3d3-9e43-4d31-a54a-bdf4efc87a57\") " pod="openshift-ingress/e2e-inf-b-b178a9-openshift-default-9c5949cd8-nsrxg"
Jul 29 00:00:21.194584 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:00:21.194513    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8226d3d3-9e43-4d31-a54a-bdf4efc87a57-istio-podinfo\") pod \"e2e-inf-b-b178a9-openshift-default-9c5949cd8-nsrxg\" (UID: \"8226d3d3-9e43-4d31-a54a-bdf4efc87a57\") " pod="openshift-ingress/e2e-inf-b-b178a9-openshift-default-9c5949cd8-nsrxg"
Jul 29 00:00:21.194584 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:00:21.194549    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8226d3d3-9e43-4d31-a54a-bdf4efc87a57-istio-token\") pod \"e2e-inf-b-b178a9-openshift-default-9c5949cd8-nsrxg\" (UID: \"8226d3d3-9e43-4d31-a54a-bdf4efc87a57\") " pod="openshift-ingress/e2e-inf-b-b178a9-openshift-default-9c5949cd8-nsrxg"
Jul 29 00:00:21.194840 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:00:21.194743    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8226d3d3-9e43-4d31-a54a-bdf4efc87a57-istio-data\") pod \"e2e-inf-b-b178a9-openshift-default-9c5949cd8-nsrxg\" (UID: \"8226d3d3-9e43-4d31-a54a-bdf4efc87a57\") " pod="openshift-ingress/e2e-inf-b-b178a9-openshift-default-9c5949cd8-nsrxg"
Jul 29 00:00:21.194905 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:00:21.194839    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8226d3d3-9e43-4d31-a54a-bdf4efc87a57-workload-socket\") pod \"e2e-inf-b-b178a9-openshift-default-9c5949cd8-nsrxg\" (UID: \"8226d3d3-9e43-4d31-a54a-bdf4efc87a57\") " pod="openshift-ingress/e2e-inf-b-b178a9-openshift-default-9c5949cd8-nsrxg"
Jul 29 00:00:21.195014 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:00:21.194993    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8226d3d3-9e43-4d31-a54a-bdf4efc87a57-credential-socket\") pod \"e2e-inf-b-b178a9-openshift-default-9c5949cd8-nsrxg\" (UID: \"8226d3d3-9e43-4d31-a54a-bdf4efc87a57\") " pod="openshift-ingress/e2e-inf-b-b178a9-openshift-default-9c5949cd8-nsrxg"
Jul 29 00:00:21.195293 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:00:21.195267    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8226d3d3-9e43-4d31-a54a-bdf4efc87a57-workload-certs\") pod \"e2e-inf-b-b178a9-openshift-default-9c5949cd8-nsrxg\" (UID: \"8226d3d3-9e43-4d31-a54a-bdf4efc87a57\") " pod="openshift-ingress/e2e-inf-b-b178a9-openshift-default-9c5949cd8-nsrxg"
Jul 29 00:00:21.195403 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:00:21.195310    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8226d3d3-9e43-4d31-a54a-bdf4efc87a57-istiod-ca-cert\") pod \"e2e-inf-b-b178a9-openshift-default-9c5949cd8-nsrxg\" (UID: \"8226d3d3-9e43-4d31-a54a-bdf4efc87a57\") " pod="openshift-ingress/e2e-inf-b-b178a9-openshift-default-9c5949cd8-nsrxg"
Jul 29 00:00:21.196597 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:00:21.196575    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8226d3d3-9e43-4d31-a54a-bdf4efc87a57-istio-envoy\") pod \"e2e-inf-b-b178a9-openshift-default-9c5949cd8-nsrxg\" (UID: \"8226d3d3-9e43-4d31-a54a-bdf4efc87a57\") " pod="openshift-ingress/e2e-inf-b-b178a9-openshift-default-9c5949cd8-nsrxg"
Jul 29 00:00:21.197202 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:00:21.197180    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8226d3d3-9e43-4d31-a54a-bdf4efc87a57-istio-podinfo\") pod \"e2e-inf-b-b178a9-openshift-default-9c5949cd8-nsrxg\" (UID: \"8226d3d3-9e43-4d31-a54a-bdf4efc87a57\") " pod="openshift-ingress/e2e-inf-b-b178a9-openshift-default-9c5949cd8-nsrxg"
Jul 29 00:00:21.201530 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:00:21.201501    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8226d3d3-9e43-4d31-a54a-bdf4efc87a57-istio-token\") pod \"e2e-inf-b-b178a9-openshift-default-9c5949cd8-nsrxg\" (UID: \"8226d3d3-9e43-4d31-a54a-bdf4efc87a57\") " pod="openshift-ingress/e2e-inf-b-b178a9-openshift-default-9c5949cd8-nsrxg"
Jul 29 00:00:21.201972 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:00:21.201955    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n2xck\" (UniqueName: \"kubernetes.io/projected/8226d3d3-9e43-4d31-a54a-bdf4efc87a57-kube-api-access-n2xck\") pod \"e2e-inf-b-b178a9-openshift-default-9c5949cd8-nsrxg\" (UID: \"8226d3d3-9e43-4d31-a54a-bdf4efc87a57\") " pod="openshift-ingress/e2e-inf-b-b178a9-openshift-default-9c5949cd8-nsrxg"
Jul 29 00:00:21.296683 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:00:21.296577    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-b178a9-openshift-default-9c5949cd8-nsrxg"
Jul 29 00:00:21.435851 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:00:21.435826    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-b178a9-openshift-default-9c5949cd8-nsrxg"]
Jul 29 00:00:21.437737 ip-10-0-131-112 kubenswrapper[2572]: W0729 00:00:21.437708    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8226d3d3_9e43_4d31_a54a_bdf4efc87a57.slice/crio-41a7f771e1c45eaa1d55b00b5ea661d6ce8617af82e7ba2a352783e318954ef6 WatchSource:0}: Error finding container 41a7f771e1c45eaa1d55b00b5ea661d6ce8617af82e7ba2a352783e318954ef6: Status 404 returned error can't find the container with id 41a7f771e1c45eaa1d55b00b5ea661d6ce8617af82e7ba2a352783e318954ef6
Jul 29 00:00:21.439956 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:00:21.439925    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 29 00:00:21.440037 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:00:21.439993    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 29 00:00:21.440037 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:00:21.440026    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 29 00:00:21.609693 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:00:21.609564    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-b178a9-openshift-default-9c5949cd8-nsrxg" event={"ID":"8226d3d3-9e43-4d31-a54a-bdf4efc87a57","Type":"ContainerStarted","Data":"520cae0c1a094bc903ec14ab477677e681ecfbd3a554d32c0880ab43a147b8c4"}
Jul 29 00:00:21.609693 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:00:21.609660    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-b178a9-openshift-default-9c5949cd8-nsrxg" event={"ID":"8226d3d3-9e43-4d31-a54a-bdf4efc87a57","Type":"ContainerStarted","Data":"41a7f771e1c45eaa1d55b00b5ea661d6ce8617af82e7ba2a352783e318954ef6"}
Jul 29 00:00:21.629818 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:00:21.629765    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-b178a9-openshift-default-9c5949cd8-nsrxg" podStartSLOduration=1.629743252 podStartE2EDuration="1.629743252s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 00:00:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:00:21.628496101 +0000 UTC m=+2298.321170611" watchObservedRunningTime="2026-07-29 00:00:21.629743252 +0000 UTC m=+2298.322417761"
Jul 29 00:00:22.297463 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:00:22.297431    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-b178a9-openshift-default-9c5949cd8-nsrxg"
Jul 29 00:00:22.302131 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:00:22.302104    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-b178a9-openshift-default-9c5949cd8-nsrxg"
Jul 29 00:00:22.614188 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:00:22.614100    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-b178a9-openshift-default-9c5949cd8-nsrxg"
Jul 29 00:00:22.615229 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:00:22.615210    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-b178a9-openshift-default-9c5949cd8-nsrxg"
Jul 29 00:00:38.657787 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:00:38.657747    2572 patch_prober.go:28] interesting pod/e2e-inf-b-b178a9-openshift-default-9c5949cd8-nsrxg container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 29 00:00:38.658163 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:00:38.657806    2572 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-b178a9-openshift-default-9c5949cd8-nsrxg" podUID="8226d3d3-9e43-4d31-a54a-bdf4efc87a57" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 29 00:00:40.692585 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:00:40.692552    2572 generic.go:356] "Generic (PLEG): container finished" podID="8226d3d3-9e43-4d31-a54a-bdf4efc87a57" containerID="520cae0c1a094bc903ec14ab477677e681ecfbd3a554d32c0880ab43a147b8c4" exitCode=0
Jul 29 00:00:40.693016 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:00:40.692641    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-b178a9-openshift-default-9c5949cd8-nsrxg" event={"ID":"8226d3d3-9e43-4d31-a54a-bdf4efc87a57","Type":"ContainerDied","Data":"520cae0c1a094bc903ec14ab477677e681ecfbd3a554d32c0880ab43a147b8c4"}
Jul 29 00:00:40.693016 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:00:40.693001    2572 scope.go:117] "RemoveContainer" containerID="520cae0c1a094bc903ec14ab477677e681ecfbd3a554d32c0880ab43a147b8c4"
Jul 29 00:00:40.694066 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:00:40.694040    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 29 00:00:40.694124 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:00:40.694104    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 29 00:00:40.694165 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:00:40.694133    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 29 00:00:41.698790 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:00:41.698756    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-b178a9-openshift-default-9c5949cd8-nsrxg" event={"ID":"8226d3d3-9e43-4d31-a54a-bdf4efc87a57","Type":"ContainerStarted","Data":"e101a490fa79b3d17905f6ef9cd739e1f95a1b1b1db61a1bffdbfcd471aa7c2c"}
Jul 29 00:00:42.296785 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:00:42.296749    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-b178a9-openshift-default-9c5949cd8-nsrxg"
Jul 29 00:00:42.296964 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:00:42.296923    2572 patch_prober.go:28] interesting pod/e2e-inf-b-b178a9-openshift-default-9c5949cd8-nsrxg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body=
Jul 29 00:00:42.297016 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:00:42.296983    2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-b178a9-openshift-default-9c5949cd8-nsrxg" podUID="8226d3d3-9e43-4d31-a54a-bdf4efc87a57" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused"
Jul 29 00:00:43.297724 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:00:43.297684    2572 patch_prober.go:28] interesting pod/e2e-inf-b-b178a9-openshift-default-9c5949cd8-nsrxg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body=
Jul 29 00:00:43.298103 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:00:43.297756    2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-b178a9-openshift-default-9c5949cd8-nsrxg" podUID="8226d3d3-9e43-4d31-a54a-bdf4efc87a57" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused"
Jul 29 00:00:44.297637 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:00:44.297577    2572 patch_prober.go:28] interesting pod/e2e-inf-b-b178a9-openshift-default-9c5949cd8-nsrxg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body=
Jul 29 00:00:44.297816 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:00:44.297664    2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-b178a9-openshift-default-9c5949cd8-nsrxg" podUID="8226d3d3-9e43-4d31-a54a-bdf4efc87a57" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused"
Jul 29 00:00:45.297009 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:00:45.296965    2572 patch_prober.go:28] interesting pod/e2e-inf-b-b178a9-openshift-default-9c5949cd8-nsrxg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body=
Jul 29 00:00:45.297220 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:00:45.297017    2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-b178a9-openshift-default-9c5949cd8-nsrxg" podUID="8226d3d3-9e43-4d31-a54a-bdf4efc87a57" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused"
Jul 29 00:00:46.300836 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:00:46.300805    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-b178a9-openshift-default-9c5949cd8-nsrxg"
Jul 29 00:00:46.301248 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:00:46.301184    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-b178a9-openshift-default-9c5949cd8-nsrxg"
Jul 29 00:00:46.301868 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:00:46.301850    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-b178a9-openshift-default-9c5949cd8-nsrxg"
Jul 29 00:02:24.587687 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:02:24.587653    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-c418f4-7b76dd6dc4-wd97m"]
Jul 29 00:02:24.588296 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:02:24.587908    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c418f4-7b76dd6dc4-wd97m" podUID="08cfcb0d-5f8a-4de1-aaa3-15b755b160aa" containerName="maas-api" containerID="cri-o://5d14406c84e9ce0578ac4805706ce4853b805a16620911496dd5629a885290dc" gracePeriod=30
Jul 29 00:02:25.039690 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:02:25.039649    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c418f4-7b76dd6dc4-wd97m"
Jul 29 00:02:25.101032 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:02:25.101004    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/08cfcb0d-5f8a-4de1-aaa3-15b755b160aa-maas-api-tls\") pod \"08cfcb0d-5f8a-4de1-aaa3-15b755b160aa\" (UID: \"08cfcb0d-5f8a-4de1-aaa3-15b755b160aa\") "
Jul 29 00:02:25.101182 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:02:25.101061    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2cjkb\" (UniqueName: \"kubernetes.io/projected/08cfcb0d-5f8a-4de1-aaa3-15b755b160aa-kube-api-access-2cjkb\") pod \"08cfcb0d-5f8a-4de1-aaa3-15b755b160aa\" (UID: \"08cfcb0d-5f8a-4de1-aaa3-15b755b160aa\") "
Jul 29 00:02:25.101182 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:02:25.101081    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/08cfcb0d-5f8a-4de1-aaa3-15b755b160aa-openshift-service-ca\") pod \"08cfcb0d-5f8a-4de1-aaa3-15b755b160aa\" (UID: \"08cfcb0d-5f8a-4de1-aaa3-15b755b160aa\") "
Jul 29 00:02:25.101515 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:02:25.101488    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/08cfcb0d-5f8a-4de1-aaa3-15b755b160aa-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "08cfcb0d-5f8a-4de1-aaa3-15b755b160aa" (UID: "08cfcb0d-5f8a-4de1-aaa3-15b755b160aa"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 00:02:25.103051 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:02:25.103020    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/08cfcb0d-5f8a-4de1-aaa3-15b755b160aa-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "08cfcb0d-5f8a-4de1-aaa3-15b755b160aa" (UID: "08cfcb0d-5f8a-4de1-aaa3-15b755b160aa"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 00:02:25.103151 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:02:25.103116    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/08cfcb0d-5f8a-4de1-aaa3-15b755b160aa-kube-api-access-2cjkb" (OuterVolumeSpecName: "kube-api-access-2cjkb") pod "08cfcb0d-5f8a-4de1-aaa3-15b755b160aa" (UID: "08cfcb0d-5f8a-4de1-aaa3-15b755b160aa"). InnerVolumeSpecName "kube-api-access-2cjkb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 00:02:25.156287 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:02:25.156226    2572 generic.go:356] "Generic (PLEG): container finished" podID="08cfcb0d-5f8a-4de1-aaa3-15b755b160aa" containerID="5d14406c84e9ce0578ac4805706ce4853b805a16620911496dd5629a885290dc" exitCode=0
Jul 29 00:02:25.156287 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:02:25.156267    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c418f4-7b76dd6dc4-wd97m"
Jul 29 00:02:25.156413 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:02:25.156310    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c418f4-7b76dd6dc4-wd97m" event={"ID":"08cfcb0d-5f8a-4de1-aaa3-15b755b160aa","Type":"ContainerDied","Data":"5d14406c84e9ce0578ac4805706ce4853b805a16620911496dd5629a885290dc"}
Jul 29 00:02:25.156413 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:02:25.156366    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c418f4-7b76dd6dc4-wd97m" event={"ID":"08cfcb0d-5f8a-4de1-aaa3-15b755b160aa","Type":"ContainerDied","Data":"1458542604d2b8fe7753a6b7a946c1139b8d6fa21de442ec84ab689e2362f7f2"}
Jul 29 00:02:25.156413 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:02:25.156387    2572 scope.go:117] "RemoveContainer" containerID="5d14406c84e9ce0578ac4805706ce4853b805a16620911496dd5629a885290dc"
Jul 29 00:02:25.167893 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:02:25.167861    2572 scope.go:117] "RemoveContainer" containerID="5d14406c84e9ce0578ac4805706ce4853b805a16620911496dd5629a885290dc"
Jul 29 00:02:25.168185 ip-10-0-131-112 kubenswrapper[2572]: E0729 00:02:25.168166    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5d14406c84e9ce0578ac4805706ce4853b805a16620911496dd5629a885290dc\": container with ID starting with 5d14406c84e9ce0578ac4805706ce4853b805a16620911496dd5629a885290dc not found: ID does not exist" containerID="5d14406c84e9ce0578ac4805706ce4853b805a16620911496dd5629a885290dc"
Jul 29 00:02:25.168258 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:02:25.168194    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5d14406c84e9ce0578ac4805706ce4853b805a16620911496dd5629a885290dc"} err="failed to get container status \"5d14406c84e9ce0578ac4805706ce4853b805a16620911496dd5629a885290dc\": rpc error: code = NotFound desc = could not find container \"5d14406c84e9ce0578ac4805706ce4853b805a16620911496dd5629a885290dc\": container with ID starting with 5d14406c84e9ce0578ac4805706ce4853b805a16620911496dd5629a885290dc not found: ID does not exist"
Jul 29 00:02:25.184181 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:02:25.184157    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-c418f4-7b76dd6dc4-wd97m"]
Jul 29 00:02:25.188853 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:02:25.188832    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-c418f4-7b76dd6dc4-wd97m"]
Jul 29 00:02:25.201843 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:02:25.201819    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/08cfcb0d-5f8a-4de1-aaa3-15b755b160aa-maas-api-tls\") on node \"ip-10-0-131-112.ec2.internal\" DevicePath \"\""
Jul 29 00:02:25.201843 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:02:25.201841    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2cjkb\" (UniqueName: \"kubernetes.io/projected/08cfcb0d-5f8a-4de1-aaa3-15b755b160aa-kube-api-access-2cjkb\") on node \"ip-10-0-131-112.ec2.internal\" DevicePath \"\""
Jul 29 00:02:25.201957 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:02:25.201866    2572 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/08cfcb0d-5f8a-4de1-aaa3-15b755b160aa-openshift-service-ca\") on node \"ip-10-0-131-112.ec2.internal\" DevicePath \"\""
Jul 29 00:02:25.865709 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:02:25.865680    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="08cfcb0d-5f8a-4de1-aaa3-15b755b160aa" path="/var/lib/kubelet/pods/08cfcb0d-5f8a-4de1-aaa3-15b755b160aa/volumes"
Jul 29 00:02:31.233895 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:02:31.233858    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-c418f4-openshift-default-6b7cd5b44b-mjw86"]
Jul 29 00:02:31.234313 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:02:31.234119    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-c418f4-openshift-default-6b7cd5b44b-mjw86" podUID="ff7d3e74-15f3-43e8-b5b7-f618c10d3630" containerName="istio-proxy" containerID="cri-o://68c2be1163f498051b4ceef721aadab7727ba4d0ddce496d046b79ef617b2c69" gracePeriod=30
Jul 29 00:02:33.157357 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:02:33.157319    2572 patch_prober.go:28] interesting pod/e2e-inf-a-c418f4-openshift-default-6b7cd5b44b-mjw86 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 29 00:02:33.157821 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:02:33.157395    2572 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-c418f4-openshift-default-6b7cd5b44b-mjw86" podUID="ff7d3e74-15f3-43e8-b5b7-f618c10d3630" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 29 00:02:36.479330 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:02:36.479303    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-c418f4-openshift-default-6b7cd5b44b-mjw86"
Jul 29 00:02:36.604461 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:02:36.604375    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff7d3e74-15f3-43e8-b5b7-f618c10d3630-workload-socket\") pod \"ff7d3e74-15f3-43e8-b5b7-f618c10d3630\" (UID: \"ff7d3e74-15f3-43e8-b5b7-f618c10d3630\") "
Jul 29 00:02:36.604461 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:02:36.604418    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ff7d3e74-15f3-43e8-b5b7-f618c10d3630-istio-envoy\") pod \"ff7d3e74-15f3-43e8-b5b7-f618c10d3630\" (UID: \"ff7d3e74-15f3-43e8-b5b7-f618c10d3630\") "
Jul 29 00:02:36.604461 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:02:36.604457    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ff7d3e74-15f3-43e8-b5b7-f618c10d3630-istio-token\") pod \"ff7d3e74-15f3-43e8-b5b7-f618c10d3630\" (UID: \"ff7d3e74-15f3-43e8-b5b7-f618c10d3630\") "
Jul 29 00:02:36.604753 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:02:36.604474    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ff7d3e74-15f3-43e8-b5b7-f618c10d3630-istio-podinfo\") pod \"ff7d3e74-15f3-43e8-b5b7-f618c10d3630\" (UID: \"ff7d3e74-15f3-43e8-b5b7-f618c10d3630\") "
Jul 29 00:02:36.604753 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:02:36.604509    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jks7h\" (UniqueName: \"kubernetes.io/projected/ff7d3e74-15f3-43e8-b5b7-f618c10d3630-kube-api-access-jks7h\") pod \"ff7d3e74-15f3-43e8-b5b7-f618c10d3630\" (UID: \"ff7d3e74-15f3-43e8-b5b7-f618c10d3630\") "
Jul 29 00:02:36.604753 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:02:36.604532    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ff7d3e74-15f3-43e8-b5b7-f618c10d3630-workload-certs\") pod \"ff7d3e74-15f3-43e8-b5b7-f618c10d3630\" (UID: \"ff7d3e74-15f3-43e8-b5b7-f618c10d3630\") "
Jul 29 00:02:36.604753 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:02:36.604546    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff7d3e74-15f3-43e8-b5b7-f618c10d3630-credential-socket\") pod \"ff7d3e74-15f3-43e8-b5b7-f618c10d3630\" (UID: \"ff7d3e74-15f3-43e8-b5b7-f618c10d3630\") "
Jul 29 00:02:36.604753 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:02:36.604581    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ff7d3e74-15f3-43e8-b5b7-f618c10d3630-istiod-ca-cert\") pod \"ff7d3e74-15f3-43e8-b5b7-f618c10d3630\" (UID: \"ff7d3e74-15f3-43e8-b5b7-f618c10d3630\") "
Jul 29 00:02:36.604753 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:02:36.604685    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ff7d3e74-15f3-43e8-b5b7-f618c10d3630-istio-data\") pod \"ff7d3e74-15f3-43e8-b5b7-f618c10d3630\" (UID: \"ff7d3e74-15f3-43e8-b5b7-f618c10d3630\") "
Jul 29 00:02:36.604753 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:02:36.604703    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff7d3e74-15f3-43e8-b5b7-f618c10d3630-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ff7d3e74-15f3-43e8-b5b7-f618c10d3630" (UID: "ff7d3e74-15f3-43e8-b5b7-f618c10d3630"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 00:02:36.605126 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:02:36.604931    2572 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff7d3e74-15f3-43e8-b5b7-f618c10d3630-workload-socket\") on node \"ip-10-0-131-112.ec2.internal\" DevicePath \"\""
Jul 29 00:02:36.605126 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:02:36.604913    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff7d3e74-15f3-43e8-b5b7-f618c10d3630-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ff7d3e74-15f3-43e8-b5b7-f618c10d3630" (UID: "ff7d3e74-15f3-43e8-b5b7-f618c10d3630"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 00:02:36.605126 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:02:36.604963    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff7d3e74-15f3-43e8-b5b7-f618c10d3630-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ff7d3e74-15f3-43e8-b5b7-f618c10d3630" (UID: "ff7d3e74-15f3-43e8-b5b7-f618c10d3630"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 00:02:36.605281 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:02:36.605218    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ff7d3e74-15f3-43e8-b5b7-f618c10d3630-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ff7d3e74-15f3-43e8-b5b7-f618c10d3630" (UID: "ff7d3e74-15f3-43e8-b5b7-f618c10d3630"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 00:02:36.605701 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:02:36.605681    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff7d3e74-15f3-43e8-b5b7-f618c10d3630-istio-data" (OuterVolumeSpecName: "istio-data") pod "ff7d3e74-15f3-43e8-b5b7-f618c10d3630" (UID: "ff7d3e74-15f3-43e8-b5b7-f618c10d3630"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 00:02:36.606680 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:02:36.606658    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ff7d3e74-15f3-43e8-b5b7-f618c10d3630-istio-token" (OuterVolumeSpecName: "istio-token") pod "ff7d3e74-15f3-43e8-b5b7-f618c10d3630" (UID: "ff7d3e74-15f3-43e8-b5b7-f618c10d3630"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 00:02:36.606776 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:02:36.606763    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ff7d3e74-15f3-43e8-b5b7-f618c10d3630-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ff7d3e74-15f3-43e8-b5b7-f618c10d3630" (UID: "ff7d3e74-15f3-43e8-b5b7-f618c10d3630"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 29 00:02:36.607164 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:02:36.607142    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff7d3e74-15f3-43e8-b5b7-f618c10d3630-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ff7d3e74-15f3-43e8-b5b7-f618c10d3630" (UID: "ff7d3e74-15f3-43e8-b5b7-f618c10d3630"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 00:02:36.607222 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:02:36.607162    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ff7d3e74-15f3-43e8-b5b7-f618c10d3630-kube-api-access-jks7h" (OuterVolumeSpecName: "kube-api-access-jks7h") pod "ff7d3e74-15f3-43e8-b5b7-f618c10d3630" (UID: "ff7d3e74-15f3-43e8-b5b7-f618c10d3630"). InnerVolumeSpecName "kube-api-access-jks7h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 00:02:36.706032 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:02:36.705994    2572 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ff7d3e74-15f3-43e8-b5b7-f618c10d3630-istio-envoy\") on node \"ip-10-0-131-112.ec2.internal\" DevicePath \"\""
Jul 29 00:02:36.706032 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:02:36.706026    2572 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ff7d3e74-15f3-43e8-b5b7-f618c10d3630-istio-token\") on node \"ip-10-0-131-112.ec2.internal\" DevicePath \"\""
Jul 29 00:02:36.706032 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:02:36.706039    2572 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ff7d3e74-15f3-43e8-b5b7-f618c10d3630-istio-podinfo\") on node \"ip-10-0-131-112.ec2.internal\" DevicePath \"\""
Jul 29 00:02:36.706260 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:02:36.706052    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jks7h\" (UniqueName: \"kubernetes.io/projected/ff7d3e74-15f3-43e8-b5b7-f618c10d3630-kube-api-access-jks7h\") on node \"ip-10-0-131-112.ec2.internal\" DevicePath \"\""
Jul 29 00:02:36.706260 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:02:36.706066    2572 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ff7d3e74-15f3-43e8-b5b7-f618c10d3630-workload-certs\") on node \"ip-10-0-131-112.ec2.internal\" DevicePath \"\""
Jul 29 00:02:36.706260 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:02:36.706077    2572 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff7d3e74-15f3-43e8-b5b7-f618c10d3630-credential-socket\") on node \"ip-10-0-131-112.ec2.internal\" DevicePath \"\""
Jul 29 00:02:36.706260 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:02:36.706089    2572 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ff7d3e74-15f3-43e8-b5b7-f618c10d3630-istiod-ca-cert\") on node \"ip-10-0-131-112.ec2.internal\" DevicePath \"\""
Jul 29 00:02:36.706260 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:02:36.706101    2572 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ff7d3e74-15f3-43e8-b5b7-f618c10d3630-istio-data\") on node \"ip-10-0-131-112.ec2.internal\" DevicePath \"\""
Jul 29 00:02:37.210274 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:02:37.210237    2572 generic.go:356] "Generic (PLEG): container finished" podID="ff7d3e74-15f3-43e8-b5b7-f618c10d3630" containerID="68c2be1163f498051b4ceef721aadab7727ba4d0ddce496d046b79ef617b2c69" exitCode=0
Jul 29 00:02:37.210274 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:02:37.210277    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-c418f4-openshift-default-6b7cd5b44b-mjw86"
Jul 29 00:02:37.210499 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:02:37.210308    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-c418f4-openshift-default-6b7cd5b44b-mjw86" event={"ID":"ff7d3e74-15f3-43e8-b5b7-f618c10d3630","Type":"ContainerDied","Data":"68c2be1163f498051b4ceef721aadab7727ba4d0ddce496d046b79ef617b2c69"}
Jul 29 00:02:37.210499 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:02:37.210359    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-c418f4-openshift-default-6b7cd5b44b-mjw86" event={"ID":"ff7d3e74-15f3-43e8-b5b7-f618c10d3630","Type":"ContainerDied","Data":"bcbfd2fb648d9e56b23c1dd9b8030e1c53f9ffd616ea6e8a7b19974f3b4d747f"}
Jul 29 00:02:37.210499 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:02:37.210381    2572 scope.go:117] "RemoveContainer" containerID="68c2be1163f498051b4ceef721aadab7727ba4d0ddce496d046b79ef617b2c69"
Jul 29 00:02:37.219612 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:02:37.219593    2572 scope.go:117] "RemoveContainer" containerID="fcdcbb52514f55f725b16936203c2d53b8467d3f785967eae78fb4111e7e881b"
Jul 29 00:02:37.226920 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:02:37.226904    2572 scope.go:117] "RemoveContainer" containerID="68c2be1163f498051b4ceef721aadab7727ba4d0ddce496d046b79ef617b2c69"
Jul 29 00:02:37.227150 ip-10-0-131-112 kubenswrapper[2572]: E0729 00:02:37.227133    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"68c2be1163f498051b4ceef721aadab7727ba4d0ddce496d046b79ef617b2c69\": container with ID starting with 68c2be1163f498051b4ceef721aadab7727ba4d0ddce496d046b79ef617b2c69 not found: ID does not exist" containerID="68c2be1163f498051b4ceef721aadab7727ba4d0ddce496d046b79ef617b2c69"
Jul 29 00:02:37.227201 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:02:37.227156    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"68c2be1163f498051b4ceef721aadab7727ba4d0ddce496d046b79ef617b2c69"} err="failed to get container status \"68c2be1163f498051b4ceef721aadab7727ba4d0ddce496d046b79ef617b2c69\": rpc error: code = NotFound desc = could not find container \"68c2be1163f498051b4ceef721aadab7727ba4d0ddce496d046b79ef617b2c69\": container with ID starting with 68c2be1163f498051b4ceef721aadab7727ba4d0ddce496d046b79ef617b2c69 not found: ID does not exist"
Jul 29 00:02:37.227201 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:02:37.227171    2572 scope.go:117] "RemoveContainer" containerID="fcdcbb52514f55f725b16936203c2d53b8467d3f785967eae78fb4111e7e881b"
Jul 29 00:02:37.227371 ip-10-0-131-112 kubenswrapper[2572]: E0729 00:02:37.227357    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fcdcbb52514f55f725b16936203c2d53b8467d3f785967eae78fb4111e7e881b\": container with ID starting with fcdcbb52514f55f725b16936203c2d53b8467d3f785967eae78fb4111e7e881b not found: ID does not exist" containerID="fcdcbb52514f55f725b16936203c2d53b8467d3f785967eae78fb4111e7e881b"
Jul 29 00:02:37.227409 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:02:37.227374    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fcdcbb52514f55f725b16936203c2d53b8467d3f785967eae78fb4111e7e881b"} err="failed to get container status \"fcdcbb52514f55f725b16936203c2d53b8467d3f785967eae78fb4111e7e881b\": rpc error: code = NotFound desc = could not find container \"fcdcbb52514f55f725b16936203c2d53b8467d3f785967eae78fb4111e7e881b\": container with ID starting with fcdcbb52514f55f725b16936203c2d53b8467d3f785967eae78fb4111e7e881b not found: ID does not exist"
Jul 29 00:02:37.232536 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:02:37.232513    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-c418f4-openshift-default-6b7cd5b44b-mjw86"]
Jul 29 00:02:37.236555 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:02:37.236535    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-c418f4-openshift-default-6b7cd5b44b-mjw86"]
Jul 29 00:02:37.865440 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:02:37.865399    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ff7d3e74-15f3-43e8-b5b7-f618c10d3630" path="/var/lib/kubelet/pods/ff7d3e74-15f3-43e8-b5b7-f618c10d3630/volumes"
Jul 29 00:02:48.655794 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:02:48.655758    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-b178a9-openshift-default-9c5949cd8-nsrxg"]
Jul 29 00:02:48.656252 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:02:48.656045    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-b178a9-openshift-default-9c5949cd8-nsrxg" podUID="8226d3d3-9e43-4d31-a54a-bdf4efc87a57" containerName="istio-proxy" containerID="cri-o://e101a490fa79b3d17905f6ef9cd739e1f95a1b1b1db61a1bffdbfcd471aa7c2c" gracePeriod=30
Jul 29 00:02:52.691645 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:02:52.689175    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-519e1e-f9f5df48f-99z52"]
Jul 29 00:02:52.691645 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:02:52.690003    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ff7d3e74-15f3-43e8-b5b7-f618c10d3630" containerName="istio-proxy"
Jul 29 00:02:52.691645 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:02:52.690027    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="ff7d3e74-15f3-43e8-b5b7-f618c10d3630" containerName="istio-proxy"
Jul 29 00:02:52.691645 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:02:52.690057    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="08cfcb0d-5f8a-4de1-aaa3-15b755b160aa" containerName="maas-api"
Jul 29 00:02:52.691645 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:02:52.690066    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="08cfcb0d-5f8a-4de1-aaa3-15b755b160aa" containerName="maas-api"
Jul 29 00:02:52.691645 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:02:52.690229    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="ff7d3e74-15f3-43e8-b5b7-f618c10d3630" containerName="istio-proxy"
Jul 29 00:02:52.691645 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:02:52.690243    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="ff7d3e74-15f3-43e8-b5b7-f618c10d3630" containerName="istio-proxy"
Jul 29 00:02:52.691645 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:02:52.690260    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="08cfcb0d-5f8a-4de1-aaa3-15b755b160aa" containerName="maas-api"
Jul 29 00:02:52.696005 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:02:52.695973    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-519e1e-f9f5df48f-99z52"
Jul 29 00:02:52.698799 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:02:52.698761    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-519e1e-serving-cert\""
Jul 29 00:02:52.702591 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:02:52.702567    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-519e1e-f9f5df48f-99z52"]
Jul 29 00:02:52.848101 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:02:52.848065    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wjkh2\" (UniqueName: \"kubernetes.io/projected/64fbfebf-560d-43cf-98b7-4c3c6288c376-kube-api-access-wjkh2\") pod \"maas-api-e2e-api-a-519e1e-f9f5df48f-99z52\" (UID: \"64fbfebf-560d-43cf-98b7-4c3c6288c376\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-519e1e-f9f5df48f-99z52"
Jul 29 00:02:52.848276 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:02:52.848113    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/64fbfebf-560d-43cf-98b7-4c3c6288c376-maas-api-tls\") pod \"maas-api-e2e-api-a-519e1e-f9f5df48f-99z52\" (UID: \"64fbfebf-560d-43cf-98b7-4c3c6288c376\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-519e1e-f9f5df48f-99z52"
Jul 29 00:02:52.848276 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:02:52.848222    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/64fbfebf-560d-43cf-98b7-4c3c6288c376-openshift-service-ca\") pod \"maas-api-e2e-api-a-519e1e-f9f5df48f-99z52\" (UID: \"64fbfebf-560d-43cf-98b7-4c3c6288c376\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-519e1e-f9f5df48f-99z52"
Jul 29 00:02:52.949592 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:02:52.949505    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/64fbfebf-560d-43cf-98b7-4c3c6288c376-maas-api-tls\") pod \"maas-api-e2e-api-a-519e1e-f9f5df48f-99z52\" (UID: \"64fbfebf-560d-43cf-98b7-4c3c6288c376\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-519e1e-f9f5df48f-99z52"
Jul 29 00:02:52.949592 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:02:52.949571    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/64fbfebf-560d-43cf-98b7-4c3c6288c376-openshift-service-ca\") pod \"maas-api-e2e-api-a-519e1e-f9f5df48f-99z52\" (UID: \"64fbfebf-560d-43cf-98b7-4c3c6288c376\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-519e1e-f9f5df48f-99z52"
Jul 29 00:02:52.949826 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:02:52.949661    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wjkh2\" (UniqueName: \"kubernetes.io/projected/64fbfebf-560d-43cf-98b7-4c3c6288c376-kube-api-access-wjkh2\") pod \"maas-api-e2e-api-a-519e1e-f9f5df48f-99z52\" (UID: \"64fbfebf-560d-43cf-98b7-4c3c6288c376\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-519e1e-f9f5df48f-99z52"
Jul 29 00:02:52.949826 ip-10-0-131-112 kubenswrapper[2572]: E0729 00:02:52.949684    2572 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-519e1e-serving-cert: secret "maas-api-e2e-api-a-519e1e-serving-cert" not found
Jul 29 00:02:52.949826 ip-10-0-131-112 kubenswrapper[2572]: E0729 00:02:52.949753    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/64fbfebf-560d-43cf-98b7-4c3c6288c376-maas-api-tls podName:64fbfebf-560d-43cf-98b7-4c3c6288c376 nodeName:}" failed. No retries permitted until 2026-07-29 00:02:53.449734398 +0000 UTC m=+2450.142408886 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/64fbfebf-560d-43cf-98b7-4c3c6288c376-maas-api-tls") pod "maas-api-e2e-api-a-519e1e-f9f5df48f-99z52" (UID: "64fbfebf-560d-43cf-98b7-4c3c6288c376") : secret "maas-api-e2e-api-a-519e1e-serving-cert" not found
Jul 29 00:02:52.950356 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:02:52.950333    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/64fbfebf-560d-43cf-98b7-4c3c6288c376-openshift-service-ca\") pod \"maas-api-e2e-api-a-519e1e-f9f5df48f-99z52\" (UID: \"64fbfebf-560d-43cf-98b7-4c3c6288c376\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-519e1e-f9f5df48f-99z52"
Jul 29 00:02:52.958670 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:02:52.958646    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wjkh2\" (UniqueName: \"kubernetes.io/projected/64fbfebf-560d-43cf-98b7-4c3c6288c376-kube-api-access-wjkh2\") pod \"maas-api-e2e-api-a-519e1e-f9f5df48f-99z52\" (UID: \"64fbfebf-560d-43cf-98b7-4c3c6288c376\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-519e1e-f9f5df48f-99z52"
Jul 29 00:02:53.453822 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:02:53.453790    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/64fbfebf-560d-43cf-98b7-4c3c6288c376-maas-api-tls\") pod \"maas-api-e2e-api-a-519e1e-f9f5df48f-99z52\" (UID: \"64fbfebf-560d-43cf-98b7-4c3c6288c376\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-519e1e-f9f5df48f-99z52"
Jul 29 00:02:53.456392 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:02:53.456363    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/64fbfebf-560d-43cf-98b7-4c3c6288c376-maas-api-tls\") pod \"maas-api-e2e-api-a-519e1e-f9f5df48f-99z52\" (UID: \"64fbfebf-560d-43cf-98b7-4c3c6288c376\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-519e1e-f9f5df48f-99z52"
Jul 29 00:02:53.609511 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:02:53.609481    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-519e1e-f9f5df48f-99z52"
Jul 29 00:02:53.743635 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:02:53.743600    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-519e1e-f9f5df48f-99z52"]
Jul 29 00:02:53.745792 ip-10-0-131-112 kubenswrapper[2572]: W0729 00:02:53.745769    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod64fbfebf_560d_43cf_98b7_4c3c6288c376.slice/crio-b071bd5c0d55ff5c82781a495c135d3d8a2eea7a02f5f987b2abaa16216c0966 WatchSource:0}: Error finding container b071bd5c0d55ff5c82781a495c135d3d8a2eea7a02f5f987b2abaa16216c0966: Status 404 returned error can't find the container with id b071bd5c0d55ff5c82781a495c135d3d8a2eea7a02f5f987b2abaa16216c0966
Jul 29 00:02:53.761138 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:02:53.761113    2572 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 29 00:02:53.900753 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:02:53.900726    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-b178a9-openshift-default-9c5949cd8-nsrxg"
Jul 29 00:02:54.060267 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:02:54.060185    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8226d3d3-9e43-4d31-a54a-bdf4efc87a57-istio-data\") pod \"8226d3d3-9e43-4d31-a54a-bdf4efc87a57\" (UID: \"8226d3d3-9e43-4d31-a54a-bdf4efc87a57\") "
Jul 29 00:02:54.060267 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:02:54.060227    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8226d3d3-9e43-4d31-a54a-bdf4efc87a57-istiod-ca-cert\") pod \"8226d3d3-9e43-4d31-a54a-bdf4efc87a57\" (UID: \"8226d3d3-9e43-4d31-a54a-bdf4efc87a57\") "
Jul 29 00:02:54.060478 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:02:54.060296    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n2xck\" (UniqueName: \"kubernetes.io/projected/8226d3d3-9e43-4d31-a54a-bdf4efc87a57-kube-api-access-n2xck\") pod \"8226d3d3-9e43-4d31-a54a-bdf4efc87a57\" (UID: \"8226d3d3-9e43-4d31-a54a-bdf4efc87a57\") "
Jul 29 00:02:54.060478 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:02:54.060320    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8226d3d3-9e43-4d31-a54a-bdf4efc87a57-workload-socket\") pod \"8226d3d3-9e43-4d31-a54a-bdf4efc87a57\" (UID: \"8226d3d3-9e43-4d31-a54a-bdf4efc87a57\") "
Jul 29 00:02:54.060478 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:02:54.060347    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8226d3d3-9e43-4d31-a54a-bdf4efc87a57-istio-envoy\") pod \"8226d3d3-9e43-4d31-a54a-bdf4efc87a57\" (UID: \"8226d3d3-9e43-4d31-a54a-bdf4efc87a57\") "
Jul 29 00:02:54.060478 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:02:54.060371    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8226d3d3-9e43-4d31-a54a-bdf4efc87a57-workload-certs\") pod \"8226d3d3-9e43-4d31-a54a-bdf4efc87a57\" (UID: \"8226d3d3-9e43-4d31-a54a-bdf4efc87a57\") "
Jul 29 00:02:54.060478 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:02:54.060397    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8226d3d3-9e43-4d31-a54a-bdf4efc87a57-credential-socket\") pod \"8226d3d3-9e43-4d31-a54a-bdf4efc87a57\" (UID: \"8226d3d3-9e43-4d31-a54a-bdf4efc87a57\") "
Jul 29 00:02:54.060478 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:02:54.060422    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8226d3d3-9e43-4d31-a54a-bdf4efc87a57-istio-podinfo\") pod \"8226d3d3-9e43-4d31-a54a-bdf4efc87a57\" (UID: \"8226d3d3-9e43-4d31-a54a-bdf4efc87a57\") "
Jul 29 00:02:54.060478 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:02:54.060464    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8226d3d3-9e43-4d31-a54a-bdf4efc87a57-istio-token\") pod \"8226d3d3-9e43-4d31-a54a-bdf4efc87a57\" (UID: \"8226d3d3-9e43-4d31-a54a-bdf4efc87a57\") "
Jul 29 00:02:54.060857 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:02:54.060656    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8226d3d3-9e43-4d31-a54a-bdf4efc87a57-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "8226d3d3-9e43-4d31-a54a-bdf4efc87a57" (UID: "8226d3d3-9e43-4d31-a54a-bdf4efc87a57"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 00:02:54.060857 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:02:54.060699    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8226d3d3-9e43-4d31-a54a-bdf4efc87a57-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "8226d3d3-9e43-4d31-a54a-bdf4efc87a57" (UID: "8226d3d3-9e43-4d31-a54a-bdf4efc87a57"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 00:02:54.060857 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:02:54.060711    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8226d3d3-9e43-4d31-a54a-bdf4efc87a57-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "8226d3d3-9e43-4d31-a54a-bdf4efc87a57" (UID: "8226d3d3-9e43-4d31-a54a-bdf4efc87a57"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 00:02:54.061028 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:02:54.060902    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8226d3d3-9e43-4d31-a54a-bdf4efc87a57-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "8226d3d3-9e43-4d31-a54a-bdf4efc87a57" (UID: "8226d3d3-9e43-4d31-a54a-bdf4efc87a57"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 00:02:54.061028 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:02:54.060922    2572 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8226d3d3-9e43-4d31-a54a-bdf4efc87a57-workload-socket\") on node \"ip-10-0-131-112.ec2.internal\" DevicePath \"\""
Jul 29 00:02:54.061028 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:02:54.060942    2572 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8226d3d3-9e43-4d31-a54a-bdf4efc87a57-workload-certs\") on node \"ip-10-0-131-112.ec2.internal\" DevicePath \"\""
Jul 29 00:02:54.061028 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:02:54.060956    2572 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8226d3d3-9e43-4d31-a54a-bdf4efc87a57-istiod-ca-cert\") on node \"ip-10-0-131-112.ec2.internal\" DevicePath \"\""
Jul 29 00:02:54.061236 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:02:54.061195    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8226d3d3-9e43-4d31-a54a-bdf4efc87a57-istio-data" (OuterVolumeSpecName: "istio-data") pod "8226d3d3-9e43-4d31-a54a-bdf4efc87a57" (UID: "8226d3d3-9e43-4d31-a54a-bdf4efc87a57"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 00:02:54.062587 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:02:54.062553    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8226d3d3-9e43-4d31-a54a-bdf4efc87a57-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "8226d3d3-9e43-4d31-a54a-bdf4efc87a57" (UID: "8226d3d3-9e43-4d31-a54a-bdf4efc87a57"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 00:02:54.062728 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:02:54.062619    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8226d3d3-9e43-4d31-a54a-bdf4efc87a57-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "8226d3d3-9e43-4d31-a54a-bdf4efc87a57" (UID: "8226d3d3-9e43-4d31-a54a-bdf4efc87a57"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 29 00:02:54.062728 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:02:54.062635    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8226d3d3-9e43-4d31-a54a-bdf4efc87a57-kube-api-access-n2xck" (OuterVolumeSpecName: "kube-api-access-n2xck") pod "8226d3d3-9e43-4d31-a54a-bdf4efc87a57" (UID: "8226d3d3-9e43-4d31-a54a-bdf4efc87a57"). InnerVolumeSpecName "kube-api-access-n2xck". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 00:02:54.062728 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:02:54.062709    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8226d3d3-9e43-4d31-a54a-bdf4efc87a57-istio-token" (OuterVolumeSpecName: "istio-token") pod "8226d3d3-9e43-4d31-a54a-bdf4efc87a57" (UID: "8226d3d3-9e43-4d31-a54a-bdf4efc87a57"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 00:02:54.161701 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:02:54.161670    2572 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8226d3d3-9e43-4d31-a54a-bdf4efc87a57-istio-token\") on node \"ip-10-0-131-112.ec2.internal\" DevicePath \"\""
Jul 29 00:02:54.161701 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:02:54.161696    2572 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8226d3d3-9e43-4d31-a54a-bdf4efc87a57-istio-data\") on node \"ip-10-0-131-112.ec2.internal\" DevicePath \"\""
Jul 29 00:02:54.161884 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:02:54.161710    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n2xck\" (UniqueName: \"kubernetes.io/projected/8226d3d3-9e43-4d31-a54a-bdf4efc87a57-kube-api-access-n2xck\") on node \"ip-10-0-131-112.ec2.internal\" DevicePath \"\""
Jul 29 00:02:54.161884 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:02:54.161723    2572 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8226d3d3-9e43-4d31-a54a-bdf4efc87a57-istio-envoy\") on node \"ip-10-0-131-112.ec2.internal\" DevicePath \"\""
Jul 29 00:02:54.161884 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:02:54.161734    2572 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8226d3d3-9e43-4d31-a54a-bdf4efc87a57-credential-socket\") on node \"ip-10-0-131-112.ec2.internal\" DevicePath \"\""
Jul 29 00:02:54.161884 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:02:54.161744    2572 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8226d3d3-9e43-4d31-a54a-bdf4efc87a57-istio-podinfo\") on node \"ip-10-0-131-112.ec2.internal\" DevicePath \"\""
Jul 29 00:02:54.288069 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:02:54.288031    2572 generic.go:356] "Generic (PLEG): container finished" podID="8226d3d3-9e43-4d31-a54a-bdf4efc87a57" containerID="e101a490fa79b3d17905f6ef9cd739e1f95a1b1b1db61a1bffdbfcd471aa7c2c" exitCode=0
Jul 29 00:02:54.288200 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:02:54.288079    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-b178a9-openshift-default-9c5949cd8-nsrxg"
Jul 29 00:02:54.288200 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:02:54.288104    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-b178a9-openshift-default-9c5949cd8-nsrxg" event={"ID":"8226d3d3-9e43-4d31-a54a-bdf4efc87a57","Type":"ContainerDied","Data":"e101a490fa79b3d17905f6ef9cd739e1f95a1b1b1db61a1bffdbfcd471aa7c2c"}
Jul 29 00:02:54.288200 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:02:54.288159    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-b178a9-openshift-default-9c5949cd8-nsrxg" event={"ID":"8226d3d3-9e43-4d31-a54a-bdf4efc87a57","Type":"ContainerDied","Data":"41a7f771e1c45eaa1d55b00b5ea661d6ce8617af82e7ba2a352783e318954ef6"}
Jul 29 00:02:54.288200 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:02:54.288182    2572 scope.go:117] "RemoveContainer" containerID="e101a490fa79b3d17905f6ef9cd739e1f95a1b1b1db61a1bffdbfcd471aa7c2c"
Jul 29 00:02:54.289437 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:02:54.289416    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-519e1e-f9f5df48f-99z52" event={"ID":"64fbfebf-560d-43cf-98b7-4c3c6288c376","Type":"ContainerStarted","Data":"b071bd5c0d55ff5c82781a495c135d3d8a2eea7a02f5f987b2abaa16216c0966"}
Jul 29 00:02:54.297195 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:02:54.297177    2572 scope.go:117] "RemoveContainer" containerID="520cae0c1a094bc903ec14ab477677e681ecfbd3a554d32c0880ab43a147b8c4"
Jul 29 00:02:54.304648 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:02:54.304614    2572 scope.go:117] "RemoveContainer" containerID="e101a490fa79b3d17905f6ef9cd739e1f95a1b1b1db61a1bffdbfcd471aa7c2c"
Jul 29 00:02:54.304887 ip-10-0-131-112 kubenswrapper[2572]: E0729 00:02:54.304868    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e101a490fa79b3d17905f6ef9cd739e1f95a1b1b1db61a1bffdbfcd471aa7c2c\": container with ID starting with e101a490fa79b3d17905f6ef9cd739e1f95a1b1b1db61a1bffdbfcd471aa7c2c not found: ID does not exist" containerID="e101a490fa79b3d17905f6ef9cd739e1f95a1b1b1db61a1bffdbfcd471aa7c2c"
Jul 29 00:02:54.304933 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:02:54.304897    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e101a490fa79b3d17905f6ef9cd739e1f95a1b1b1db61a1bffdbfcd471aa7c2c"} err="failed to get container status \"e101a490fa79b3d17905f6ef9cd739e1f95a1b1b1db61a1bffdbfcd471aa7c2c\": rpc error: code = NotFound desc = could not find container \"e101a490fa79b3d17905f6ef9cd739e1f95a1b1b1db61a1bffdbfcd471aa7c2c\": container with ID starting with e101a490fa79b3d17905f6ef9cd739e1f95a1b1b1db61a1bffdbfcd471aa7c2c not found: ID does not exist"
Jul 29 00:02:54.304933 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:02:54.304913    2572 scope.go:117] "RemoveContainer" containerID="520cae0c1a094bc903ec14ab477677e681ecfbd3a554d32c0880ab43a147b8c4"
Jul 29 00:02:54.305160 ip-10-0-131-112 kubenswrapper[2572]: E0729 00:02:54.305141    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"520cae0c1a094bc903ec14ab477677e681ecfbd3a554d32c0880ab43a147b8c4\": container with ID starting with 520cae0c1a094bc903ec14ab477677e681ecfbd3a554d32c0880ab43a147b8c4 not found: ID does not exist" containerID="520cae0c1a094bc903ec14ab477677e681ecfbd3a554d32c0880ab43a147b8c4"
Jul 29 00:02:54.305213 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:02:54.305165    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"520cae0c1a094bc903ec14ab477677e681ecfbd3a554d32c0880ab43a147b8c4"} err="failed to get container status \"520cae0c1a094bc903ec14ab477677e681ecfbd3a554d32c0880ab43a147b8c4\": rpc error: code = NotFound desc = could not find container \"520cae0c1a094bc903ec14ab477677e681ecfbd3a554d32c0880ab43a147b8c4\": container with ID starting with 520cae0c1a094bc903ec14ab477677e681ecfbd3a554d32c0880ab43a147b8c4 not found: ID does not exist"
Jul 29 00:02:54.309325 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:02:54.309304    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-b178a9-openshift-default-9c5949cd8-nsrxg"]
Jul 29 00:02:54.313270 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:02:54.313221    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-b178a9-openshift-default-9c5949cd8-nsrxg"]
Jul 29 00:02:55.295689 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:02:55.295643    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-519e1e-f9f5df48f-99z52" event={"ID":"64fbfebf-560d-43cf-98b7-4c3c6288c376","Type":"ContainerStarted","Data":"6cb71fc6381a0d5840940183c5230d2c11ebdb65de44e95dee40e5f8d975ecd0"}
Jul 29 00:02:55.296207 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:02:55.295709    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-519e1e-f9f5df48f-99z52"
Jul 29 00:02:55.317386 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:02:55.317335    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-519e1e-f9f5df48f-99z52" podStartSLOduration=2.73247255 podStartE2EDuration="3.317318934s" totalImagesPullingTime="584.846384ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 00:02:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:02:55.315105773 +0000 UTC m=+2452.007780290" watchObservedRunningTime="2026-07-29 00:02:55.317318934 +0000 UTC m=+2452.009993442"
Jul 29 00:02:55.865875 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:02:55.865842    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8226d3d3-9e43-4d31-a54a-bdf4efc87a57" path="/var/lib/kubelet/pods/8226d3d3-9e43-4d31-a54a-bdf4efc87a57/volumes"
Jul 29 00:03:01.306119 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:03:01.306086    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-519e1e-f9f5df48f-99z52"
Jul 29 00:03:18.232988 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:03:18.232952    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-48f20e-openshift-default-7bd9675b4d-b9dpq"]
Jul 29 00:03:18.233417 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:03:18.233397    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8226d3d3-9e43-4d31-a54a-bdf4efc87a57" containerName="istio-proxy"
Jul 29 00:03:18.233461 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:03:18.233417    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="8226d3d3-9e43-4d31-a54a-bdf4efc87a57" containerName="istio-proxy"
Jul 29 00:03:18.233461 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:03:18.233456    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ff7d3e74-15f3-43e8-b5b7-f618c10d3630" containerName="istio-proxy"
Jul 29 00:03:18.233525 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:03:18.233465    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="ff7d3e74-15f3-43e8-b5b7-f618c10d3630" containerName="istio-proxy"
Jul 29 00:03:18.233570 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:03:18.233544    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="8226d3d3-9e43-4d31-a54a-bdf4efc87a57" containerName="istio-proxy"
Jul 29 00:03:18.233570 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:03:18.233559    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="8226d3d3-9e43-4d31-a54a-bdf4efc87a57" containerName="istio-proxy"
Jul 29 00:03:18.236800 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:03:18.236781    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-48f20e-openshift-default-7bd9675b4d-b9dpq"
Jul 29 00:03:18.239114 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:03:18.239092    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-48f20e-openshift-default-dockercfg-cg245\""
Jul 29 00:03:18.246350 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:03:18.246324    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-48f20e-openshift-default-7bd9675b4d-b9dpq"]
Jul 29 00:03:18.370757 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:03:18.370718    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/33fe4e42-3adf-4ebc-8c1f-43646ee346b5-istio-envoy\") pod \"e2e-api-b-48f20e-openshift-default-7bd9675b4d-b9dpq\" (UID: \"33fe4e42-3adf-4ebc-8c1f-43646ee346b5\") " pod="openshift-ingress/e2e-api-b-48f20e-openshift-default-7bd9675b4d-b9dpq"
Jul 29 00:03:18.370757 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:03:18.370754    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/33fe4e42-3adf-4ebc-8c1f-43646ee346b5-istio-data\") pod \"e2e-api-b-48f20e-openshift-default-7bd9675b4d-b9dpq\" (UID: \"33fe4e42-3adf-4ebc-8c1f-43646ee346b5\") " pod="openshift-ingress/e2e-api-b-48f20e-openshift-default-7bd9675b4d-b9dpq"
Jul 29 00:03:18.371004 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:03:18.370779    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/33fe4e42-3adf-4ebc-8c1f-43646ee346b5-credential-socket\") pod \"e2e-api-b-48f20e-openshift-default-7bd9675b4d-b9dpq\" (UID: \"33fe4e42-3adf-4ebc-8c1f-43646ee346b5\") " pod="openshift-ingress/e2e-api-b-48f20e-openshift-default-7bd9675b4d-b9dpq"
Jul 29 00:03:18.371004 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:03:18.370843    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/33fe4e42-3adf-4ebc-8c1f-43646ee346b5-istio-podinfo\") pod \"e2e-api-b-48f20e-openshift-default-7bd9675b4d-b9dpq\" (UID: \"33fe4e42-3adf-4ebc-8c1f-43646ee346b5\") " pod="openshift-ingress/e2e-api-b-48f20e-openshift-default-7bd9675b4d-b9dpq"
Jul 29 00:03:18.371004 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:03:18.370889    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/33fe4e42-3adf-4ebc-8c1f-43646ee346b5-workload-certs\") pod \"e2e-api-b-48f20e-openshift-default-7bd9675b4d-b9dpq\" (UID: \"33fe4e42-3adf-4ebc-8c1f-43646ee346b5\") " pod="openshift-ingress/e2e-api-b-48f20e-openshift-default-7bd9675b4d-b9dpq"
Jul 29 00:03:18.371004 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:03:18.370912    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/33fe4e42-3adf-4ebc-8c1f-43646ee346b5-workload-socket\") pod \"e2e-api-b-48f20e-openshift-default-7bd9675b4d-b9dpq\" (UID: \"33fe4e42-3adf-4ebc-8c1f-43646ee346b5\") " pod="openshift-ingress/e2e-api-b-48f20e-openshift-default-7bd9675b4d-b9dpq"
Jul 29 00:03:18.371004 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:03:18.370944    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gttpf\" (UniqueName: \"kubernetes.io/projected/33fe4e42-3adf-4ebc-8c1f-43646ee346b5-kube-api-access-gttpf\") pod \"e2e-api-b-48f20e-openshift-default-7bd9675b4d-b9dpq\" (UID: \"33fe4e42-3adf-4ebc-8c1f-43646ee346b5\") " pod="openshift-ingress/e2e-api-b-48f20e-openshift-default-7bd9675b4d-b9dpq"
Jul 29 00:03:18.371004 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:03:18.370983    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/33fe4e42-3adf-4ebc-8c1f-43646ee346b5-istiod-ca-cert\") pod \"e2e-api-b-48f20e-openshift-default-7bd9675b4d-b9dpq\" (UID: \"33fe4e42-3adf-4ebc-8c1f-43646ee346b5\") " pod="openshift-ingress/e2e-api-b-48f20e-openshift-default-7bd9675b4d-b9dpq"
Jul 29 00:03:18.371317 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:03:18.371025    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/33fe4e42-3adf-4ebc-8c1f-43646ee346b5-istio-token\") pod \"e2e-api-b-48f20e-openshift-default-7bd9675b4d-b9dpq\" (UID: \"33fe4e42-3adf-4ebc-8c1f-43646ee346b5\") " pod="openshift-ingress/e2e-api-b-48f20e-openshift-default-7bd9675b4d-b9dpq"
Jul 29 00:03:18.471873 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:03:18.471841    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/33fe4e42-3adf-4ebc-8c1f-43646ee346b5-istio-envoy\") pod \"e2e-api-b-48f20e-openshift-default-7bd9675b4d-b9dpq\" (UID: \"33fe4e42-3adf-4ebc-8c1f-43646ee346b5\") " pod="openshift-ingress/e2e-api-b-48f20e-openshift-default-7bd9675b4d-b9dpq"
Jul 29 00:03:18.471873 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:03:18.471875    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/33fe4e42-3adf-4ebc-8c1f-43646ee346b5-istio-data\") pod \"e2e-api-b-48f20e-openshift-default-7bd9675b4d-b9dpq\" (UID: \"33fe4e42-3adf-4ebc-8c1f-43646ee346b5\") " pod="openshift-ingress/e2e-api-b-48f20e-openshift-default-7bd9675b4d-b9dpq"
Jul 29 00:03:18.472096 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:03:18.471897    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/33fe4e42-3adf-4ebc-8c1f-43646ee346b5-credential-socket\") pod \"e2e-api-b-48f20e-openshift-default-7bd9675b4d-b9dpq\" (UID: \"33fe4e42-3adf-4ebc-8c1f-43646ee346b5\") " pod="openshift-ingress/e2e-api-b-48f20e-openshift-default-7bd9675b4d-b9dpq"
Jul 29 00:03:18.472096 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:03:18.471920    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/33fe4e42-3adf-4ebc-8c1f-43646ee346b5-istio-podinfo\") pod \"e2e-api-b-48f20e-openshift-default-7bd9675b4d-b9dpq\" (UID: \"33fe4e42-3adf-4ebc-8c1f-43646ee346b5\") " pod="openshift-ingress/e2e-api-b-48f20e-openshift-default-7bd9675b4d-b9dpq"
Jul 29 00:03:18.472096 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:03:18.471957    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/33fe4e42-3adf-4ebc-8c1f-43646ee346b5-workload-certs\") pod \"e2e-api-b-48f20e-openshift-default-7bd9675b4d-b9dpq\" (UID: \"33fe4e42-3adf-4ebc-8c1f-43646ee346b5\") " pod="openshift-ingress/e2e-api-b-48f20e-openshift-default-7bd9675b4d-b9dpq"
Jul 29 00:03:18.472096 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:03:18.471978    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/33fe4e42-3adf-4ebc-8c1f-43646ee346b5-workload-socket\") pod \"e2e-api-b-48f20e-openshift-default-7bd9675b4d-b9dpq\" (UID: \"33fe4e42-3adf-4ebc-8c1f-43646ee346b5\") " pod="openshift-ingress/e2e-api-b-48f20e-openshift-default-7bd9675b4d-b9dpq"
Jul 29 00:03:18.472096 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:03:18.472005    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gttpf\" (UniqueName: \"kubernetes.io/projected/33fe4e42-3adf-4ebc-8c1f-43646ee346b5-kube-api-access-gttpf\") pod \"e2e-api-b-48f20e-openshift-default-7bd9675b4d-b9dpq\" (UID: \"33fe4e42-3adf-4ebc-8c1f-43646ee346b5\") " pod="openshift-ingress/e2e-api-b-48f20e-openshift-default-7bd9675b4d-b9dpq"
Jul 29 00:03:18.472096 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:03:18.472057    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/33fe4e42-3adf-4ebc-8c1f-43646ee346b5-istiod-ca-cert\") pod \"e2e-api-b-48f20e-openshift-default-7bd9675b4d-b9dpq\" (UID: \"33fe4e42-3adf-4ebc-8c1f-43646ee346b5\") " pod="openshift-ingress/e2e-api-b-48f20e-openshift-default-7bd9675b4d-b9dpq"
Jul 29 00:03:18.472428 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:03:18.472105    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/33fe4e42-3adf-4ebc-8c1f-43646ee346b5-istio-token\") pod \"e2e-api-b-48f20e-openshift-default-7bd9675b4d-b9dpq\" (UID: \"33fe4e42-3adf-4ebc-8c1f-43646ee346b5\") " pod="openshift-ingress/e2e-api-b-48f20e-openshift-default-7bd9675b4d-b9dpq"
Jul 29 00:03:18.472428 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:03:18.472308    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/33fe4e42-3adf-4ebc-8c1f-43646ee346b5-istio-data\") pod \"e2e-api-b-48f20e-openshift-default-7bd9675b4d-b9dpq\" (UID: \"33fe4e42-3adf-4ebc-8c1f-43646ee346b5\") " pod="openshift-ingress/e2e-api-b-48f20e-openshift-default-7bd9675b4d-b9dpq"
Jul 29 00:03:18.472526 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:03:18.472441    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/33fe4e42-3adf-4ebc-8c1f-43646ee346b5-workload-socket\") pod \"e2e-api-b-48f20e-openshift-default-7bd9675b4d-b9dpq\" (UID: \"33fe4e42-3adf-4ebc-8c1f-43646ee346b5\") " pod="openshift-ingress/e2e-api-b-48f20e-openshift-default-7bd9675b4d-b9dpq"
Jul 29 00:03:18.472562 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:03:18.472542    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/33fe4e42-3adf-4ebc-8c1f-43646ee346b5-workload-certs\") pod \"e2e-api-b-48f20e-openshift-default-7bd9675b4d-b9dpq\" (UID: \"33fe4e42-3adf-4ebc-8c1f-43646ee346b5\") " pod="openshift-ingress/e2e-api-b-48f20e-openshift-default-7bd9675b4d-b9dpq"
Jul 29 00:03:18.472707 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:03:18.472688    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/33fe4e42-3adf-4ebc-8c1f-43646ee346b5-credential-socket\") pod \"e2e-api-b-48f20e-openshift-default-7bd9675b4d-b9dpq\" (UID: \"33fe4e42-3adf-4ebc-8c1f-43646ee346b5\") " pod="openshift-ingress/e2e-api-b-48f20e-openshift-default-7bd9675b4d-b9dpq"
Jul 29 00:03:18.472981 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:03:18.472963    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/33fe4e42-3adf-4ebc-8c1f-43646ee346b5-istiod-ca-cert\") pod \"e2e-api-b-48f20e-openshift-default-7bd9675b4d-b9dpq\" (UID: \"33fe4e42-3adf-4ebc-8c1f-43646ee346b5\") " pod="openshift-ingress/e2e-api-b-48f20e-openshift-default-7bd9675b4d-b9dpq"
Jul 29 00:03:18.474298 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:03:18.474272    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/33fe4e42-3adf-4ebc-8c1f-43646ee346b5-istio-envoy\") pod \"e2e-api-b-48f20e-openshift-default-7bd9675b4d-b9dpq\" (UID: \"33fe4e42-3adf-4ebc-8c1f-43646ee346b5\") " pod="openshift-ingress/e2e-api-b-48f20e-openshift-default-7bd9675b4d-b9dpq"
Jul 29 00:03:18.474530 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:03:18.474509    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/33fe4e42-3adf-4ebc-8c1f-43646ee346b5-istio-podinfo\") pod \"e2e-api-b-48f20e-openshift-default-7bd9675b4d-b9dpq\" (UID: \"33fe4e42-3adf-4ebc-8c1f-43646ee346b5\") " pod="openshift-ingress/e2e-api-b-48f20e-openshift-default-7bd9675b4d-b9dpq"
Jul 29 00:03:18.479310 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:03:18.479284    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/33fe4e42-3adf-4ebc-8c1f-43646ee346b5-istio-token\") pod \"e2e-api-b-48f20e-openshift-default-7bd9675b4d-b9dpq\" (UID: \"33fe4e42-3adf-4ebc-8c1f-43646ee346b5\") " pod="openshift-ingress/e2e-api-b-48f20e-openshift-default-7bd9675b4d-b9dpq"
Jul 29 00:03:18.479764 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:03:18.479742    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gttpf\" (UniqueName: \"kubernetes.io/projected/33fe4e42-3adf-4ebc-8c1f-43646ee346b5-kube-api-access-gttpf\") pod \"e2e-api-b-48f20e-openshift-default-7bd9675b4d-b9dpq\" (UID: \"33fe4e42-3adf-4ebc-8c1f-43646ee346b5\") " pod="openshift-ingress/e2e-api-b-48f20e-openshift-default-7bd9675b4d-b9dpq"
Jul 29 00:03:18.550451 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:03:18.550384    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-48f20e-openshift-default-7bd9675b4d-b9dpq"
Jul 29 00:03:18.680667 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:03:18.680640    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-48f20e-openshift-default-7bd9675b4d-b9dpq"]
Jul 29 00:03:18.682411 ip-10-0-131-112 kubenswrapper[2572]: W0729 00:03:18.682390    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33fe4e42_3adf_4ebc_8c1f_43646ee346b5.slice/crio-0bc5fa9936b91659a001cfcfc53434b4d2a52242f842f94d6b7a916e699a7e53 WatchSource:0}: Error finding container 0bc5fa9936b91659a001cfcfc53434b4d2a52242f842f94d6b7a916e699a7e53: Status 404 returned error can't find the container with id 0bc5fa9936b91659a001cfcfc53434b4d2a52242f842f94d6b7a916e699a7e53
Jul 29 00:03:18.685025 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:03:18.684986    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 29 00:03:18.685125 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:03:18.685062    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 29 00:03:18.685125 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:03:18.685098    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 29 00:03:19.399103 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:03:19.399066    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-48f20e-openshift-default-7bd9675b4d-b9dpq" event={"ID":"33fe4e42-3adf-4ebc-8c1f-43646ee346b5","Type":"ContainerStarted","Data":"2a2476b0dd87fc57351f97abd4b89452932ba0c2bef0ce12dd92b24bdae187fa"}
Jul 29 00:03:19.399459 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:03:19.399118    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-48f20e-openshift-default-7bd9675b4d-b9dpq" event={"ID":"33fe4e42-3adf-4ebc-8c1f-43646ee346b5","Type":"ContainerStarted","Data":"0bc5fa9936b91659a001cfcfc53434b4d2a52242f842f94d6b7a916e699a7e53"}
Jul 29 00:03:19.418885 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:03:19.418840    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-48f20e-openshift-default-7bd9675b4d-b9dpq" podStartSLOduration=1.418824007 podStartE2EDuration="1.418824007s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 00:03:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:03:19.417287605 +0000 UTC m=+2476.109962114" watchObservedRunningTime="2026-07-29 00:03:19.418824007 +0000 UTC m=+2476.111498515"
Jul 29 00:03:19.551298 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:03:19.551262    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-48f20e-openshift-default-7bd9675b4d-b9dpq"
Jul 29 00:03:19.556143 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:03:19.556119    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-48f20e-openshift-default-7bd9675b4d-b9dpq"
Jul 29 00:03:20.403879 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:03:20.403848    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-48f20e-openshift-default-7bd9675b4d-b9dpq"
Jul 29 00:03:20.404568 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:03:20.404549    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-48f20e-openshift-default-7bd9675b4d-b9dpq"
Jul 29 00:03:36.443903 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:03:36.443613    2572 patch_prober.go:28] interesting pod/e2e-api-b-48f20e-openshift-default-7bd9675b4d-b9dpq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 29 00:03:36.443903 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:03:36.443766    2572 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-48f20e-openshift-default-7bd9675b4d-b9dpq" podUID="33fe4e42-3adf-4ebc-8c1f-43646ee346b5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 29 00:03:36.477483 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:03:36.477452    2572 generic.go:356] "Generic (PLEG): container finished" podID="33fe4e42-3adf-4ebc-8c1f-43646ee346b5" containerID="2a2476b0dd87fc57351f97abd4b89452932ba0c2bef0ce12dd92b24bdae187fa" exitCode=0
Jul 29 00:03:36.477615 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:03:36.477506    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-48f20e-openshift-default-7bd9675b4d-b9dpq" event={"ID":"33fe4e42-3adf-4ebc-8c1f-43646ee346b5","Type":"ContainerDied","Data":"2a2476b0dd87fc57351f97abd4b89452932ba0c2bef0ce12dd92b24bdae187fa"}
Jul 29 00:03:36.477946 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:03:36.477928    2572 scope.go:117] "RemoveContainer" containerID="2a2476b0dd87fc57351f97abd4b89452932ba0c2bef0ce12dd92b24bdae187fa"
Jul 29 00:03:36.478979 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:03:36.478947    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 29 00:03:36.479102 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:03:36.479019    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 29 00:03:36.479102 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:03:36.479058    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 29 00:03:37.484105 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:03:37.484071    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-48f20e-openshift-default-7bd9675b4d-b9dpq" event={"ID":"33fe4e42-3adf-4ebc-8c1f-43646ee346b5","Type":"ContainerStarted","Data":"db2fa34fab0b951a0f4970ae46cb111b57727bc4b41912b3f6b476de2f614061"}
Jul 29 00:03:37.550788 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:03:37.550751    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-48f20e-openshift-default-7bd9675b4d-b9dpq"
Jul 29 00:03:37.550983 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:03:37.550961    2572 patch_prober.go:28] interesting pod/e2e-api-b-48f20e-openshift-default-7bd9675b4d-b9dpq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Jul 29 00:03:37.551030 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:03:37.551007    2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-48f20e-openshift-default-7bd9675b4d-b9dpq" podUID="33fe4e42-3adf-4ebc-8c1f-43646ee346b5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Jul 29 00:03:38.551580 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:03:38.551538    2572 patch_prober.go:28] interesting pod/e2e-api-b-48f20e-openshift-default-7bd9675b4d-b9dpq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Jul 29 00:03:38.551994 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:03:38.551612    2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-48f20e-openshift-default-7bd9675b4d-b9dpq" podUID="33fe4e42-3adf-4ebc-8c1f-43646ee346b5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Jul 29 00:03:39.551097 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:03:39.551045    2572 patch_prober.go:28] interesting pod/e2e-api-b-48f20e-openshift-default-7bd9675b4d-b9dpq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Jul 29 00:03:39.551296 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:03:39.551135    2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-48f20e-openshift-default-7bd9675b4d-b9dpq" podUID="33fe4e42-3adf-4ebc-8c1f-43646ee346b5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Jul 29 00:03:40.551435 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:03:40.551393    2572 patch_prober.go:28] interesting pod/e2e-api-b-48f20e-openshift-default-7bd9675b4d-b9dpq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Jul 29 00:03:40.551864 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:03:40.551460    2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-48f20e-openshift-default-7bd9675b4d-b9dpq" podUID="33fe4e42-3adf-4ebc-8c1f-43646ee346b5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Jul 29 00:03:41.551174 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:03:41.551131    2572 patch_prober.go:28] interesting pod/e2e-api-b-48f20e-openshift-default-7bd9675b4d-b9dpq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Jul 29 00:03:41.551362 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:03:41.551203    2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-48f20e-openshift-default-7bd9675b4d-b9dpq" podUID="33fe4e42-3adf-4ebc-8c1f-43646ee346b5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Jul 29 00:03:42.554605 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:03:42.554574    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-48f20e-openshift-default-7bd9675b4d-b9dpq"
Jul 29 00:03:42.555068 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:03:42.554939    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-48f20e-openshift-default-7bd9675b4d-b9dpq"
Jul 29 00:03:42.555796 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:03:42.555773    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-48f20e-openshift-default-7bd9675b4d-b9dpq"
Jul 29 00:04:04.693915 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:04:04.693883    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-519e1e-f9f5df48f-99z52"]
Jul 29 00:04:04.694385 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:04:04.694117    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-519e1e-f9f5df48f-99z52" podUID="64fbfebf-560d-43cf-98b7-4c3c6288c376" containerName="maas-api" containerID="cri-o://6cb71fc6381a0d5840940183c5230d2c11ebdb65de44e95dee40e5f8d975ecd0" gracePeriod=30
Jul 29 00:04:04.938980 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:04:04.938953    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-519e1e-f9f5df48f-99z52"
Jul 29 00:04:05.086077 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:04:05.086050    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/64fbfebf-560d-43cf-98b7-4c3c6288c376-maas-api-tls\") pod \"64fbfebf-560d-43cf-98b7-4c3c6288c376\" (UID: \"64fbfebf-560d-43cf-98b7-4c3c6288c376\") "
Jul 29 00:04:05.086273 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:04:05.086096    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/64fbfebf-560d-43cf-98b7-4c3c6288c376-openshift-service-ca\") pod \"64fbfebf-560d-43cf-98b7-4c3c6288c376\" (UID: \"64fbfebf-560d-43cf-98b7-4c3c6288c376\") "
Jul 29 00:04:05.086273 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:04:05.086148    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wjkh2\" (UniqueName: \"kubernetes.io/projected/64fbfebf-560d-43cf-98b7-4c3c6288c376-kube-api-access-wjkh2\") pod \"64fbfebf-560d-43cf-98b7-4c3c6288c376\" (UID: \"64fbfebf-560d-43cf-98b7-4c3c6288c376\") "
Jul 29 00:04:05.086533 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:04:05.086500    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/64fbfebf-560d-43cf-98b7-4c3c6288c376-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "64fbfebf-560d-43cf-98b7-4c3c6288c376" (UID: "64fbfebf-560d-43cf-98b7-4c3c6288c376"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 00:04:05.088367 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:04:05.088335    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/64fbfebf-560d-43cf-98b7-4c3c6288c376-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "64fbfebf-560d-43cf-98b7-4c3c6288c376" (UID: "64fbfebf-560d-43cf-98b7-4c3c6288c376"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 00:04:05.088367 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:04:05.088346    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/64fbfebf-560d-43cf-98b7-4c3c6288c376-kube-api-access-wjkh2" (OuterVolumeSpecName: "kube-api-access-wjkh2") pod "64fbfebf-560d-43cf-98b7-4c3c6288c376" (UID: "64fbfebf-560d-43cf-98b7-4c3c6288c376"). InnerVolumeSpecName "kube-api-access-wjkh2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 00:04:05.187366 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:04:05.187328    2572 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/64fbfebf-560d-43cf-98b7-4c3c6288c376-openshift-service-ca\") on node \"ip-10-0-131-112.ec2.internal\" DevicePath \"\""
Jul 29 00:04:05.187366 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:04:05.187359    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wjkh2\" (UniqueName: \"kubernetes.io/projected/64fbfebf-560d-43cf-98b7-4c3c6288c376-kube-api-access-wjkh2\") on node \"ip-10-0-131-112.ec2.internal\" DevicePath \"\""
Jul 29 00:04:05.187366 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:04:05.187370    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/64fbfebf-560d-43cf-98b7-4c3c6288c376-maas-api-tls\") on node \"ip-10-0-131-112.ec2.internal\" DevicePath \"\""
Jul 29 00:04:05.607695 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:04:05.607663    2572 generic.go:356] "Generic (PLEG): container finished" podID="64fbfebf-560d-43cf-98b7-4c3c6288c376" containerID="6cb71fc6381a0d5840940183c5230d2c11ebdb65de44e95dee40e5f8d975ecd0" exitCode=0
Jul 29 00:04:05.607869 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:04:05.607703    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-519e1e-f9f5df48f-99z52"
Jul 29 00:04:05.607869 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:04:05.607710    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-519e1e-f9f5df48f-99z52" event={"ID":"64fbfebf-560d-43cf-98b7-4c3c6288c376","Type":"ContainerDied","Data":"6cb71fc6381a0d5840940183c5230d2c11ebdb65de44e95dee40e5f8d975ecd0"}
Jul 29 00:04:05.607869 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:04:05.607756    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-519e1e-f9f5df48f-99z52" event={"ID":"64fbfebf-560d-43cf-98b7-4c3c6288c376","Type":"ContainerDied","Data":"b071bd5c0d55ff5c82781a495c135d3d8a2eea7a02f5f987b2abaa16216c0966"}
Jul 29 00:04:05.607869 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:04:05.607782    2572 scope.go:117] "RemoveContainer" containerID="6cb71fc6381a0d5840940183c5230d2c11ebdb65de44e95dee40e5f8d975ecd0"
Jul 29 00:04:05.616976 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:04:05.616954    2572 scope.go:117] "RemoveContainer" containerID="6cb71fc6381a0d5840940183c5230d2c11ebdb65de44e95dee40e5f8d975ecd0"
Jul 29 00:04:05.617228 ip-10-0-131-112 kubenswrapper[2572]: E0729 00:04:05.617206    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6cb71fc6381a0d5840940183c5230d2c11ebdb65de44e95dee40e5f8d975ecd0\": container with ID starting with 6cb71fc6381a0d5840940183c5230d2c11ebdb65de44e95dee40e5f8d975ecd0 not found: ID does not exist" containerID="6cb71fc6381a0d5840940183c5230d2c11ebdb65de44e95dee40e5f8d975ecd0"
Jul 29 00:04:05.617276 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:04:05.617237    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6cb71fc6381a0d5840940183c5230d2c11ebdb65de44e95dee40e5f8d975ecd0"} err="failed to get container status \"6cb71fc6381a0d5840940183c5230d2c11ebdb65de44e95dee40e5f8d975ecd0\": rpc error: code = NotFound desc = could not find container \"6cb71fc6381a0d5840940183c5230d2c11ebdb65de44e95dee40e5f8d975ecd0\": container with ID starting with 6cb71fc6381a0d5840940183c5230d2c11ebdb65de44e95dee40e5f8d975ecd0 not found: ID does not exist"
Jul 29 00:04:05.628283 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:04:05.628261    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-519e1e-f9f5df48f-99z52"]
Jul 29 00:04:05.634812 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:04:05.634793    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-519e1e-f9f5df48f-99z52"]
Jul 29 00:04:05.865720 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:04:05.865641    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="64fbfebf-560d-43cf-98b7-4c3c6288c376" path="/var/lib/kubelet/pods/64fbfebf-560d-43cf-98b7-4c3c6288c376/volumes"
Jul 29 00:04:33.708447 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:04:33.708416    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-48f20e-openshift-default-7bd9675b4d-b9dpq"]
Jul 29 00:04:33.708927 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:04:33.708722    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-48f20e-openshift-default-7bd9675b4d-b9dpq" podUID="33fe4e42-3adf-4ebc-8c1f-43646ee346b5" containerName="istio-proxy" containerID="cri-o://db2fa34fab0b951a0f4970ae46cb111b57727bc4b41912b3f6b476de2f614061" gracePeriod=30
Jul 29 00:04:37.286991 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:04:37.286952    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-daedc5-649c5c6ff8-vvd4h"]
Jul 29 00:04:37.287470 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:04:37.287392    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="64fbfebf-560d-43cf-98b7-4c3c6288c376" containerName="maas-api"
Jul 29 00:04:37.287470 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:04:37.287406    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="64fbfebf-560d-43cf-98b7-4c3c6288c376" containerName="maas-api"
Jul 29 00:04:37.287470 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:04:37.287433    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8226d3d3-9e43-4d31-a54a-bdf4efc87a57" containerName="istio-proxy"
Jul 29 00:04:37.287470 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:04:37.287442    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="8226d3d3-9e43-4d31-a54a-bdf4efc87a57" containerName="istio-proxy"
Jul 29 00:04:37.287605 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:04:37.287506    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="64fbfebf-560d-43cf-98b7-4c3c6288c376" containerName="maas-api"
Jul 29 00:04:37.290714 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:04:37.290695    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-daedc5-649c5c6ff8-vvd4h"
Jul 29 00:04:37.293023 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:04:37.292995    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-daedc5-serving-cert\""
Jul 29 00:04:37.304055 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:04:37.304030    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-daedc5-649c5c6ff8-vvd4h"]
Jul 29 00:04:37.373144 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:04:37.373107    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/26c0d3a1-95f7-45eb-bc66-b81b60b4260d-maas-api-tls\") pod \"maas-api-e2e-shared-a-daedc5-649c5c6ff8-vvd4h\" (UID: \"26c0d3a1-95f7-45eb-bc66-b81b60b4260d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-daedc5-649c5c6ff8-vvd4h"
Jul 29 00:04:37.373342 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:04:37.373159    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/26c0d3a1-95f7-45eb-bc66-b81b60b4260d-openshift-service-ca\") pod \"maas-api-e2e-shared-a-daedc5-649c5c6ff8-vvd4h\" (UID: \"26c0d3a1-95f7-45eb-bc66-b81b60b4260d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-daedc5-649c5c6ff8-vvd4h"
Jul 29 00:04:37.373342 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:04:37.373224    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j5m8w\" (UniqueName: \"kubernetes.io/projected/26c0d3a1-95f7-45eb-bc66-b81b60b4260d-kube-api-access-j5m8w\") pod \"maas-api-e2e-shared-a-daedc5-649c5c6ff8-vvd4h\" (UID: \"26c0d3a1-95f7-45eb-bc66-b81b60b4260d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-daedc5-649c5c6ff8-vvd4h"
Jul 29 00:04:37.474443 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:04:37.474403    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/26c0d3a1-95f7-45eb-bc66-b81b60b4260d-maas-api-tls\") pod \"maas-api-e2e-shared-a-daedc5-649c5c6ff8-vvd4h\" (UID: \"26c0d3a1-95f7-45eb-bc66-b81b60b4260d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-daedc5-649c5c6ff8-vvd4h"
Jul 29 00:04:37.474676 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:04:37.474459    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/26c0d3a1-95f7-45eb-bc66-b81b60b4260d-openshift-service-ca\") pod \"maas-api-e2e-shared-a-daedc5-649c5c6ff8-vvd4h\" (UID: \"26c0d3a1-95f7-45eb-bc66-b81b60b4260d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-daedc5-649c5c6ff8-vvd4h"
Jul 29 00:04:37.474676 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:04:37.474491    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j5m8w\" (UniqueName: \"kubernetes.io/projected/26c0d3a1-95f7-45eb-bc66-b81b60b4260d-kube-api-access-j5m8w\") pod \"maas-api-e2e-shared-a-daedc5-649c5c6ff8-vvd4h\" (UID: \"26c0d3a1-95f7-45eb-bc66-b81b60b4260d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-daedc5-649c5c6ff8-vvd4h"
Jul 29 00:04:37.474676 ip-10-0-131-112 kubenswrapper[2572]: E0729 00:04:37.474584    2572 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-daedc5-serving-cert: secret "maas-api-e2e-shared-a-daedc5-serving-cert" not found
Jul 29 00:04:37.474860 ip-10-0-131-112 kubenswrapper[2572]: E0729 00:04:37.474692    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/26c0d3a1-95f7-45eb-bc66-b81b60b4260d-maas-api-tls podName:26c0d3a1-95f7-45eb-bc66-b81b60b4260d nodeName:}" failed. No retries permitted until 2026-07-29 00:04:37.974668101 +0000 UTC m=+2554.667342588 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/26c0d3a1-95f7-45eb-bc66-b81b60b4260d-maas-api-tls") pod "maas-api-e2e-shared-a-daedc5-649c5c6ff8-vvd4h" (UID: "26c0d3a1-95f7-45eb-bc66-b81b60b4260d") : secret "maas-api-e2e-shared-a-daedc5-serving-cert" not found
Jul 29 00:04:37.475228 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:04:37.475208    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/26c0d3a1-95f7-45eb-bc66-b81b60b4260d-openshift-service-ca\") pod \"maas-api-e2e-shared-a-daedc5-649c5c6ff8-vvd4h\" (UID: \"26c0d3a1-95f7-45eb-bc66-b81b60b4260d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-daedc5-649c5c6ff8-vvd4h"
Jul 29 00:04:37.492687 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:04:37.492658    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j5m8w\" (UniqueName: \"kubernetes.io/projected/26c0d3a1-95f7-45eb-bc66-b81b60b4260d-kube-api-access-j5m8w\") pod \"maas-api-e2e-shared-a-daedc5-649c5c6ff8-vvd4h\" (UID: \"26c0d3a1-95f7-45eb-bc66-b81b60b4260d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-daedc5-649c5c6ff8-vvd4h"
Jul 29 00:04:37.979718 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:04:37.979673    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/26c0d3a1-95f7-45eb-bc66-b81b60b4260d-maas-api-tls\") pod \"maas-api-e2e-shared-a-daedc5-649c5c6ff8-vvd4h\" (UID: \"26c0d3a1-95f7-45eb-bc66-b81b60b4260d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-daedc5-649c5c6ff8-vvd4h"
Jul 29 00:04:37.982270 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:04:37.982237    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/26c0d3a1-95f7-45eb-bc66-b81b60b4260d-maas-api-tls\") pod \"maas-api-e2e-shared-a-daedc5-649c5c6ff8-vvd4h\" (UID: \"26c0d3a1-95f7-45eb-bc66-b81b60b4260d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-daedc5-649c5c6ff8-vvd4h"
Jul 29 00:04:38.201611 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:04:38.201568    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-daedc5-649c5c6ff8-vvd4h"
Jul 29 00:04:38.336095 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:04:38.336066    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-daedc5-649c5c6ff8-vvd4h"]
Jul 29 00:04:38.338747 ip-10-0-131-112 kubenswrapper[2572]: W0729 00:04:38.338712    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26c0d3a1_95f7_45eb_bc66_b81b60b4260d.slice/crio-bc52a909a5755cfc5c09e36eac89b9dd372ddc44666147e02be4bffc341ad073 WatchSource:0}: Error finding container bc52a909a5755cfc5c09e36eac89b9dd372ddc44666147e02be4bffc341ad073: Status 404 returned error can't find the container with id bc52a909a5755cfc5c09e36eac89b9dd372ddc44666147e02be4bffc341ad073
Jul 29 00:04:38.760067 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:04:38.760029    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-daedc5-649c5c6ff8-vvd4h" event={"ID":"26c0d3a1-95f7-45eb-bc66-b81b60b4260d","Type":"ContainerStarted","Data":"bc52a909a5755cfc5c09e36eac89b9dd372ddc44666147e02be4bffc341ad073"}
Jul 29 00:04:38.959573 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:04:38.959509    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-48f20e-openshift-default-7bd9675b4d-b9dpq"
Jul 29 00:04:39.090256 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:04:39.090222    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/33fe4e42-3adf-4ebc-8c1f-43646ee346b5-workload-certs\") pod \"33fe4e42-3adf-4ebc-8c1f-43646ee346b5\" (UID: \"33fe4e42-3adf-4ebc-8c1f-43646ee346b5\") "
Jul 29 00:04:39.090448 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:04:39.090266    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/33fe4e42-3adf-4ebc-8c1f-43646ee346b5-istio-data\") pod \"33fe4e42-3adf-4ebc-8c1f-43646ee346b5\" (UID: \"33fe4e42-3adf-4ebc-8c1f-43646ee346b5\") "
Jul 29 00:04:39.090448 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:04:39.090288    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/33fe4e42-3adf-4ebc-8c1f-43646ee346b5-istiod-ca-cert\") pod \"33fe4e42-3adf-4ebc-8c1f-43646ee346b5\" (UID: \"33fe4e42-3adf-4ebc-8c1f-43646ee346b5\") "
Jul 29 00:04:39.090448 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:04:39.090332    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/33fe4e42-3adf-4ebc-8c1f-43646ee346b5-workload-socket\") pod \"33fe4e42-3adf-4ebc-8c1f-43646ee346b5\" (UID: \"33fe4e42-3adf-4ebc-8c1f-43646ee346b5\") "
Jul 29 00:04:39.090448 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:04:39.090354    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gttpf\" (UniqueName: \"kubernetes.io/projected/33fe4e42-3adf-4ebc-8c1f-43646ee346b5-kube-api-access-gttpf\") pod \"33fe4e42-3adf-4ebc-8c1f-43646ee346b5\" (UID: \"33fe4e42-3adf-4ebc-8c1f-43646ee346b5\") "
Jul 29 00:04:39.090448 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:04:39.090409    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/33fe4e42-3adf-4ebc-8c1f-43646ee346b5-istio-token\") pod \"33fe4e42-3adf-4ebc-8c1f-43646ee346b5\" (UID: \"33fe4e42-3adf-4ebc-8c1f-43646ee346b5\") "
Jul 29 00:04:39.090448 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:04:39.090441    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/33fe4e42-3adf-4ebc-8c1f-43646ee346b5-credential-socket\") pod \"33fe4e42-3adf-4ebc-8c1f-43646ee346b5\" (UID: \"33fe4e42-3adf-4ebc-8c1f-43646ee346b5\") "
Jul 29 00:04:39.090828 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:04:39.090475    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/33fe4e42-3adf-4ebc-8c1f-43646ee346b5-istio-podinfo\") pod \"33fe4e42-3adf-4ebc-8c1f-43646ee346b5\" (UID: \"33fe4e42-3adf-4ebc-8c1f-43646ee346b5\") "
Jul 29 00:04:39.090828 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:04:39.090508    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/33fe4e42-3adf-4ebc-8c1f-43646ee346b5-istio-envoy\") pod \"33fe4e42-3adf-4ebc-8c1f-43646ee346b5\" (UID: \"33fe4e42-3adf-4ebc-8c1f-43646ee346b5\") "
Jul 29 00:04:39.090828 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:04:39.090550    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33fe4e42-3adf-4ebc-8c1f-43646ee346b5-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "33fe4e42-3adf-4ebc-8c1f-43646ee346b5" (UID: "33fe4e42-3adf-4ebc-8c1f-43646ee346b5"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 00:04:39.090828 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:04:39.090661    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33fe4e42-3adf-4ebc-8c1f-43646ee346b5-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "33fe4e42-3adf-4ebc-8c1f-43646ee346b5" (UID: "33fe4e42-3adf-4ebc-8c1f-43646ee346b5"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 00:04:39.090828 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:04:39.090798    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/33fe4e42-3adf-4ebc-8c1f-43646ee346b5-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "33fe4e42-3adf-4ebc-8c1f-43646ee346b5" (UID: "33fe4e42-3adf-4ebc-8c1f-43646ee346b5"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 00:04:39.091121 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:04:39.090925    2572 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/33fe4e42-3adf-4ebc-8c1f-43646ee346b5-workload-certs\") on node \"ip-10-0-131-112.ec2.internal\" DevicePath \"\""
Jul 29 00:04:39.091121 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:04:39.090951    2572 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/33fe4e42-3adf-4ebc-8c1f-43646ee346b5-istiod-ca-cert\") on node \"ip-10-0-131-112.ec2.internal\" DevicePath \"\""
Jul 29 00:04:39.091121 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:04:39.090966    2572 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/33fe4e42-3adf-4ebc-8c1f-43646ee346b5-workload-socket\") on node \"ip-10-0-131-112.ec2.internal\" DevicePath \"\""
Jul 29 00:04:39.091121 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:04:39.091008    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33fe4e42-3adf-4ebc-8c1f-43646ee346b5-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "33fe4e42-3adf-4ebc-8c1f-43646ee346b5" (UID: "33fe4e42-3adf-4ebc-8c1f-43646ee346b5"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 00:04:39.091327 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:04:39.091111    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33fe4e42-3adf-4ebc-8c1f-43646ee346b5-istio-data" (OuterVolumeSpecName: "istio-data") pod "33fe4e42-3adf-4ebc-8c1f-43646ee346b5" (UID: "33fe4e42-3adf-4ebc-8c1f-43646ee346b5"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 00:04:39.092823 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:04:39.092793    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/33fe4e42-3adf-4ebc-8c1f-43646ee346b5-kube-api-access-gttpf" (OuterVolumeSpecName: "kube-api-access-gttpf") pod "33fe4e42-3adf-4ebc-8c1f-43646ee346b5" (UID: "33fe4e42-3adf-4ebc-8c1f-43646ee346b5"). InnerVolumeSpecName "kube-api-access-gttpf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 00:04:39.092940 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:04:39.092793    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/33fe4e42-3adf-4ebc-8c1f-43646ee346b5-istio-token" (OuterVolumeSpecName: "istio-token") pod "33fe4e42-3adf-4ebc-8c1f-43646ee346b5" (UID: "33fe4e42-3adf-4ebc-8c1f-43646ee346b5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 00:04:39.092940 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:04:39.092853    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/33fe4e42-3adf-4ebc-8c1f-43646ee346b5-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "33fe4e42-3adf-4ebc-8c1f-43646ee346b5" (UID: "33fe4e42-3adf-4ebc-8c1f-43646ee346b5"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 29 00:04:39.092940 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:04:39.092911    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33fe4e42-3adf-4ebc-8c1f-43646ee346b5-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "33fe4e42-3adf-4ebc-8c1f-43646ee346b5" (UID: "33fe4e42-3adf-4ebc-8c1f-43646ee346b5"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 00:04:39.191838 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:04:39.191803    2572 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/33fe4e42-3adf-4ebc-8c1f-43646ee346b5-istio-data\") on node \"ip-10-0-131-112.ec2.internal\" DevicePath \"\""
Jul 29 00:04:39.191838 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:04:39.191832    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gttpf\" (UniqueName: \"kubernetes.io/projected/33fe4e42-3adf-4ebc-8c1f-43646ee346b5-kube-api-access-gttpf\") on node \"ip-10-0-131-112.ec2.internal\" DevicePath \"\""
Jul 29 00:04:39.191838 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:04:39.191843    2572 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/33fe4e42-3adf-4ebc-8c1f-43646ee346b5-istio-token\") on node \"ip-10-0-131-112.ec2.internal\" DevicePath \"\""
Jul 29 00:04:39.192073 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:04:39.191852    2572 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/33fe4e42-3adf-4ebc-8c1f-43646ee346b5-credential-socket\") on node \"ip-10-0-131-112.ec2.internal\" DevicePath \"\""
Jul 29 00:04:39.192073 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:04:39.191861    2572 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/33fe4e42-3adf-4ebc-8c1f-43646ee346b5-istio-podinfo\") on node \"ip-10-0-131-112.ec2.internal\" DevicePath \"\""
Jul 29 00:04:39.192073 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:04:39.191869    2572 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/33fe4e42-3adf-4ebc-8c1f-43646ee346b5-istio-envoy\") on node \"ip-10-0-131-112.ec2.internal\" DevicePath \"\""
Jul 29 00:04:39.765880 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:04:39.765843    2572 generic.go:356] "Generic (PLEG): container finished" podID="33fe4e42-3adf-4ebc-8c1f-43646ee346b5" containerID="db2fa34fab0b951a0f4970ae46cb111b57727bc4b41912b3f6b476de2f614061" exitCode=0
Jul 29 00:04:39.766354 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:04:39.765889    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-48f20e-openshift-default-7bd9675b4d-b9dpq"
Jul 29 00:04:39.766354 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:04:39.765929    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-48f20e-openshift-default-7bd9675b4d-b9dpq" event={"ID":"33fe4e42-3adf-4ebc-8c1f-43646ee346b5","Type":"ContainerDied","Data":"db2fa34fab0b951a0f4970ae46cb111b57727bc4b41912b3f6b476de2f614061"}
Jul 29 00:04:39.766354 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:04:39.765977    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-48f20e-openshift-default-7bd9675b4d-b9dpq" event={"ID":"33fe4e42-3adf-4ebc-8c1f-43646ee346b5","Type":"ContainerDied","Data":"0bc5fa9936b91659a001cfcfc53434b4d2a52242f842f94d6b7a916e699a7e53"}
Jul 29 00:04:39.766354 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:04:39.765999    2572 scope.go:117] "RemoveContainer" containerID="db2fa34fab0b951a0f4970ae46cb111b57727bc4b41912b3f6b476de2f614061"
Jul 29 00:04:39.767608 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:04:39.767586    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-daedc5-649c5c6ff8-vvd4h" event={"ID":"26c0d3a1-95f7-45eb-bc66-b81b60b4260d","Type":"ContainerStarted","Data":"bc9a3d0f81f6a06264f17e1f316e6b5acb16f8ba2263aeafa3a7996016f72cb6"}
Jul 29 00:04:39.767729 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:04:39.767707    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-daedc5-649c5c6ff8-vvd4h"
Jul 29 00:04:39.775199 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:04:39.775182    2572 scope.go:117] "RemoveContainer" containerID="2a2476b0dd87fc57351f97abd4b89452932ba0c2bef0ce12dd92b24bdae187fa"
Jul 29 00:04:39.782650 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:04:39.782616    2572 scope.go:117] "RemoveContainer" containerID="db2fa34fab0b951a0f4970ae46cb111b57727bc4b41912b3f6b476de2f614061"
Jul 29 00:04:39.782880 ip-10-0-131-112 kubenswrapper[2572]: E0729 00:04:39.782860    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"db2fa34fab0b951a0f4970ae46cb111b57727bc4b41912b3f6b476de2f614061\": container with ID starting with db2fa34fab0b951a0f4970ae46cb111b57727bc4b41912b3f6b476de2f614061 not found: ID does not exist" containerID="db2fa34fab0b951a0f4970ae46cb111b57727bc4b41912b3f6b476de2f614061"
Jul 29 00:04:39.782927 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:04:39.782888    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"db2fa34fab0b951a0f4970ae46cb111b57727bc4b41912b3f6b476de2f614061"} err="failed to get container status \"db2fa34fab0b951a0f4970ae46cb111b57727bc4b41912b3f6b476de2f614061\": rpc error: code = NotFound desc = could not find container \"db2fa34fab0b951a0f4970ae46cb111b57727bc4b41912b3f6b476de2f614061\": container with ID starting with db2fa34fab0b951a0f4970ae46cb111b57727bc4b41912b3f6b476de2f614061 not found: ID does not exist"
Jul 29 00:04:39.782927 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:04:39.782910    2572 scope.go:117] "RemoveContainer" containerID="2a2476b0dd87fc57351f97abd4b89452932ba0c2bef0ce12dd92b24bdae187fa"
Jul 29 00:04:39.783128 ip-10-0-131-112 kubenswrapper[2572]: E0729 00:04:39.783107    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2a2476b0dd87fc57351f97abd4b89452932ba0c2bef0ce12dd92b24bdae187fa\": container with ID starting with 2a2476b0dd87fc57351f97abd4b89452932ba0c2bef0ce12dd92b24bdae187fa not found: ID does not exist" containerID="2a2476b0dd87fc57351f97abd4b89452932ba0c2bef0ce12dd92b24bdae187fa"
Jul 29 00:04:39.783170 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:04:39.783134    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2a2476b0dd87fc57351f97abd4b89452932ba0c2bef0ce12dd92b24bdae187fa"} err="failed to get container status \"2a2476b0dd87fc57351f97abd4b89452932ba0c2bef0ce12dd92b24bdae187fa\": rpc error: code = NotFound desc = could not find container \"2a2476b0dd87fc57351f97abd4b89452932ba0c2bef0ce12dd92b24bdae187fa\": container with ID starting with 2a2476b0dd87fc57351f97abd4b89452932ba0c2bef0ce12dd92b24bdae187fa not found: ID does not exist"
Jul 29 00:04:39.786822 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:04:39.786779    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-daedc5-649c5c6ff8-vvd4h" podStartSLOduration=2.425206147 podStartE2EDuration="2.786766419s" totalImagesPullingTime="361.560272ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 00:04:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:04:39.78468344 +0000 UTC m=+2556.477357952" watchObservedRunningTime="2026-07-29 00:04:39.786766419 +0000 UTC m=+2556.479440978"
Jul 29 00:04:39.798442 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:04:39.798420    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-48f20e-openshift-default-7bd9675b4d-b9dpq"]
Jul 29 00:04:39.802451 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:04:39.802428    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-48f20e-openshift-default-7bd9675b4d-b9dpq"]
Jul 29 00:04:39.865970 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:04:39.865935    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="33fe4e42-3adf-4ebc-8c1f-43646ee346b5" path="/var/lib/kubelet/pods/33fe4e42-3adf-4ebc-8c1f-43646ee346b5/volumes"
Jul 29 00:04:45.778074 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:04:45.778045    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-daedc5-649c5c6ff8-vvd4h"
Jul 29 00:05:02.825538 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:05:02.825505    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-fa6130-openshift-default-86f8b5f84-7nzs8"]
Jul 29 00:05:02.825941 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:05:02.825846    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="33fe4e42-3adf-4ebc-8c1f-43646ee346b5" containerName="istio-proxy"
Jul 29 00:05:02.825941 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:05:02.825859    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="33fe4e42-3adf-4ebc-8c1f-43646ee346b5" containerName="istio-proxy"
Jul 29 00:05:02.825941 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:05:02.825919    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="33fe4e42-3adf-4ebc-8c1f-43646ee346b5" containerName="istio-proxy"
Jul 29 00:05:02.825941 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:05:02.825927    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="33fe4e42-3adf-4ebc-8c1f-43646ee346b5" containerName="istio-proxy"
Jul 29 00:05:02.828989 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:05:02.828968    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-fa6130-openshift-default-86f8b5f84-7nzs8"
Jul 29 00:05:02.831287 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:05:02.831266    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-fa6130-openshift-default-dockercfg-blv75\""
Jul 29 00:05:02.840421 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:05:02.840395    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-fa6130-openshift-default-86f8b5f84-7nzs8"]
Jul 29 00:05:03.004703 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:05:03.004661    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cf43995d-f389-47b4-992f-4df567ed653c-istio-envoy\") pod \"e2e-shared-b-fa6130-openshift-default-86f8b5f84-7nzs8\" (UID: \"cf43995d-f389-47b4-992f-4df567ed653c\") " pod="openshift-ingress/e2e-shared-b-fa6130-openshift-default-86f8b5f84-7nzs8"
Jul 29 00:05:03.004703 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:05:03.004706    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf43995d-f389-47b4-992f-4df567ed653c-credential-socket\") pod \"e2e-shared-b-fa6130-openshift-default-86f8b5f84-7nzs8\" (UID: \"cf43995d-f389-47b4-992f-4df567ed653c\") " pod="openshift-ingress/e2e-shared-b-fa6130-openshift-default-86f8b5f84-7nzs8"
Jul 29 00:05:03.004973 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:05:03.004829    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cf43995d-f389-47b4-992f-4df567ed653c-istio-podinfo\") pod \"e2e-shared-b-fa6130-openshift-default-86f8b5f84-7nzs8\" (UID: \"cf43995d-f389-47b4-992f-4df567ed653c\") " pod="openshift-ingress/e2e-shared-b-fa6130-openshift-default-86f8b5f84-7nzs8"
Jul 29 00:05:03.004973 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:05:03.004865    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf43995d-f389-47b4-992f-4df567ed653c-workload-socket\") pod \"e2e-shared-b-fa6130-openshift-default-86f8b5f84-7nzs8\" (UID: \"cf43995d-f389-47b4-992f-4df567ed653c\") " pod="openshift-ingress/e2e-shared-b-fa6130-openshift-default-86f8b5f84-7nzs8"
Jul 29 00:05:03.004973 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:05:03.004889    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cf43995d-f389-47b4-992f-4df567ed653c-istiod-ca-cert\") pod \"e2e-shared-b-fa6130-openshift-default-86f8b5f84-7nzs8\" (UID: \"cf43995d-f389-47b4-992f-4df567ed653c\") " pod="openshift-ingress/e2e-shared-b-fa6130-openshift-default-86f8b5f84-7nzs8"
Jul 29 00:05:03.004973 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:05:03.004920    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cf43995d-f389-47b4-992f-4df567ed653c-workload-certs\") pod \"e2e-shared-b-fa6130-openshift-default-86f8b5f84-7nzs8\" (UID: \"cf43995d-f389-47b4-992f-4df567ed653c\") " pod="openshift-ingress/e2e-shared-b-fa6130-openshift-default-86f8b5f84-7nzs8"
Jul 29 00:05:03.004973 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:05:03.004956    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dvjsd\" (UniqueName: \"kubernetes.io/projected/cf43995d-f389-47b4-992f-4df567ed653c-kube-api-access-dvjsd\") pod \"e2e-shared-b-fa6130-openshift-default-86f8b5f84-7nzs8\" (UID: \"cf43995d-f389-47b4-992f-4df567ed653c\") " pod="openshift-ingress/e2e-shared-b-fa6130-openshift-default-86f8b5f84-7nzs8"
Jul 29 00:05:03.005147 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:05:03.004993    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cf43995d-f389-47b4-992f-4df567ed653c-istio-token\") pod \"e2e-shared-b-fa6130-openshift-default-86f8b5f84-7nzs8\" (UID: \"cf43995d-f389-47b4-992f-4df567ed653c\") " pod="openshift-ingress/e2e-shared-b-fa6130-openshift-default-86f8b5f84-7nzs8"
Jul 29 00:05:03.005147 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:05:03.005023    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cf43995d-f389-47b4-992f-4df567ed653c-istio-data\") pod \"e2e-shared-b-fa6130-openshift-default-86f8b5f84-7nzs8\" (UID: \"cf43995d-f389-47b4-992f-4df567ed653c\") " pod="openshift-ingress/e2e-shared-b-fa6130-openshift-default-86f8b5f84-7nzs8"
Jul 29 00:05:03.105833 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:05:03.105739    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cf43995d-f389-47b4-992f-4df567ed653c-istio-data\") pod \"e2e-shared-b-fa6130-openshift-default-86f8b5f84-7nzs8\" (UID: \"cf43995d-f389-47b4-992f-4df567ed653c\") " pod="openshift-ingress/e2e-shared-b-fa6130-openshift-default-86f8b5f84-7nzs8"
Jul 29 00:05:03.105833 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:05:03.105808    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cf43995d-f389-47b4-992f-4df567ed653c-istio-envoy\") pod \"e2e-shared-b-fa6130-openshift-default-86f8b5f84-7nzs8\" (UID: \"cf43995d-f389-47b4-992f-4df567ed653c\") " pod="openshift-ingress/e2e-shared-b-fa6130-openshift-default-86f8b5f84-7nzs8"
Jul 29 00:05:03.106036 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:05:03.105842    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf43995d-f389-47b4-992f-4df567ed653c-credential-socket\") pod \"e2e-shared-b-fa6130-openshift-default-86f8b5f84-7nzs8\" (UID: \"cf43995d-f389-47b4-992f-4df567ed653c\") " pod="openshift-ingress/e2e-shared-b-fa6130-openshift-default-86f8b5f84-7nzs8"
Jul 29 00:05:03.106036 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:05:03.105905    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cf43995d-f389-47b4-992f-4df567ed653c-istio-podinfo\") pod \"e2e-shared-b-fa6130-openshift-default-86f8b5f84-7nzs8\" (UID: \"cf43995d-f389-47b4-992f-4df567ed653c\") " pod="openshift-ingress/e2e-shared-b-fa6130-openshift-default-86f8b5f84-7nzs8"
Jul 29 00:05:03.106036 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:05:03.105931    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf43995d-f389-47b4-992f-4df567ed653c-workload-socket\") pod \"e2e-shared-b-fa6130-openshift-default-86f8b5f84-7nzs8\" (UID: \"cf43995d-f389-47b4-992f-4df567ed653c\") " pod="openshift-ingress/e2e-shared-b-fa6130-openshift-default-86f8b5f84-7nzs8"
Jul 29 00:05:03.106036 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:05:03.105952    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cf43995d-f389-47b4-992f-4df567ed653c-istiod-ca-cert\") pod \"e2e-shared-b-fa6130-openshift-default-86f8b5f84-7nzs8\" (UID: \"cf43995d-f389-47b4-992f-4df567ed653c\") " pod="openshift-ingress/e2e-shared-b-fa6130-openshift-default-86f8b5f84-7nzs8"
Jul 29 00:05:03.106036 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:05:03.105980    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cf43995d-f389-47b4-992f-4df567ed653c-workload-certs\") pod \"e2e-shared-b-fa6130-openshift-default-86f8b5f84-7nzs8\" (UID: \"cf43995d-f389-47b4-992f-4df567ed653c\") " pod="openshift-ingress/e2e-shared-b-fa6130-openshift-default-86f8b5f84-7nzs8"
Jul 29 00:05:03.106036 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:05:03.106009    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dvjsd\" (UniqueName: \"kubernetes.io/projected/cf43995d-f389-47b4-992f-4df567ed653c-kube-api-access-dvjsd\") pod \"e2e-shared-b-fa6130-openshift-default-86f8b5f84-7nzs8\" (UID: \"cf43995d-f389-47b4-992f-4df567ed653c\") " pod="openshift-ingress/e2e-shared-b-fa6130-openshift-default-86f8b5f84-7nzs8"
Jul 29 00:05:03.106394 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:05:03.106038    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cf43995d-f389-47b4-992f-4df567ed653c-istio-token\") pod \"e2e-shared-b-fa6130-openshift-default-86f8b5f84-7nzs8\" (UID: \"cf43995d-f389-47b4-992f-4df567ed653c\") " pod="openshift-ingress/e2e-shared-b-fa6130-openshift-default-86f8b5f84-7nzs8"
Jul 29 00:05:03.106394 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:05:03.106155    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cf43995d-f389-47b4-992f-4df567ed653c-istio-data\") pod \"e2e-shared-b-fa6130-openshift-default-86f8b5f84-7nzs8\" (UID: \"cf43995d-f389-47b4-992f-4df567ed653c\") " pod="openshift-ingress/e2e-shared-b-fa6130-openshift-default-86f8b5f84-7nzs8"
Jul 29 00:05:03.106490 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:05:03.106439    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf43995d-f389-47b4-992f-4df567ed653c-workload-socket\") pod \"e2e-shared-b-fa6130-openshift-default-86f8b5f84-7nzs8\" (UID: \"cf43995d-f389-47b4-992f-4df567ed653c\") " pod="openshift-ingress/e2e-shared-b-fa6130-openshift-default-86f8b5f84-7nzs8"
Jul 29 00:05:03.106719 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:05:03.106696    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cf43995d-f389-47b4-992f-4df567ed653c-istiod-ca-cert\") pod \"e2e-shared-b-fa6130-openshift-default-86f8b5f84-7nzs8\" (UID: \"cf43995d-f389-47b4-992f-4df567ed653c\") " pod="openshift-ingress/e2e-shared-b-fa6130-openshift-default-86f8b5f84-7nzs8"
Jul 29 00:05:03.106768 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:05:03.106738    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf43995d-f389-47b4-992f-4df567ed653c-credential-socket\") pod \"e2e-shared-b-fa6130-openshift-default-86f8b5f84-7nzs8\" (UID: \"cf43995d-f389-47b4-992f-4df567ed653c\") " pod="openshift-ingress/e2e-shared-b-fa6130-openshift-default-86f8b5f84-7nzs8"
Jul 29 00:05:03.106768 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:05:03.106755    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cf43995d-f389-47b4-992f-4df567ed653c-workload-certs\") pod \"e2e-shared-b-fa6130-openshift-default-86f8b5f84-7nzs8\" (UID: \"cf43995d-f389-47b4-992f-4df567ed653c\") " pod="openshift-ingress/e2e-shared-b-fa6130-openshift-default-86f8b5f84-7nzs8"
Jul 29 00:05:03.108570 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:05:03.108550    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cf43995d-f389-47b4-992f-4df567ed653c-istio-envoy\") pod \"e2e-shared-b-fa6130-openshift-default-86f8b5f84-7nzs8\" (UID: \"cf43995d-f389-47b4-992f-4df567ed653c\") " pod="openshift-ingress/e2e-shared-b-fa6130-openshift-default-86f8b5f84-7nzs8"
Jul 29 00:05:03.108803 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:05:03.108785    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cf43995d-f389-47b4-992f-4df567ed653c-istio-podinfo\") pod \"e2e-shared-b-fa6130-openshift-default-86f8b5f84-7nzs8\" (UID: \"cf43995d-f389-47b4-992f-4df567ed653c\") " pod="openshift-ingress/e2e-shared-b-fa6130-openshift-default-86f8b5f84-7nzs8"
Jul 29 00:05:03.113801 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:05:03.113774    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cf43995d-f389-47b4-992f-4df567ed653c-istio-token\") pod \"e2e-shared-b-fa6130-openshift-default-86f8b5f84-7nzs8\" (UID: \"cf43995d-f389-47b4-992f-4df567ed653c\") " pod="openshift-ingress/e2e-shared-b-fa6130-openshift-default-86f8b5f84-7nzs8"
Jul 29 00:05:03.114199 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:05:03.114178    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dvjsd\" (UniqueName: \"kubernetes.io/projected/cf43995d-f389-47b4-992f-4df567ed653c-kube-api-access-dvjsd\") pod \"e2e-shared-b-fa6130-openshift-default-86f8b5f84-7nzs8\" (UID: \"cf43995d-f389-47b4-992f-4df567ed653c\") " pod="openshift-ingress/e2e-shared-b-fa6130-openshift-default-86f8b5f84-7nzs8"
Jul 29 00:05:03.140116 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:05:03.140091    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-fa6130-openshift-default-86f8b5f84-7nzs8"
Jul 29 00:05:03.272953 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:05:03.272923    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-fa6130-openshift-default-86f8b5f84-7nzs8"]
Jul 29 00:05:03.274907 ip-10-0-131-112 kubenswrapper[2572]: W0729 00:05:03.274880    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf43995d_f389_47b4_992f_4df567ed653c.slice/crio-2d6d3bfe9a75400903c32b09560a61e469b147a42fc9b123ab67af4ce39cc070 WatchSource:0}: Error finding container 2d6d3bfe9a75400903c32b09560a61e469b147a42fc9b123ab67af4ce39cc070: Status 404 returned error can't find the container with id 2d6d3bfe9a75400903c32b09560a61e469b147a42fc9b123ab67af4ce39cc070
Jul 29 00:05:03.277156 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:05:03.277119    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 29 00:05:03.277250 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:05:03.277202    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 29 00:05:03.277250 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:05:03.277242    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 29 00:05:03.869478 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:05:03.869442    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-fa6130-openshift-default-86f8b5f84-7nzs8" event={"ID":"cf43995d-f389-47b4-992f-4df567ed653c","Type":"ContainerStarted","Data":"d439127331bda97a8104e9851dfd95a758dc0e68af8128e2788f65c61e2c202f"}
Jul 29 00:05:03.869962 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:05:03.869496    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-fa6130-openshift-default-86f8b5f84-7nzs8" event={"ID":"cf43995d-f389-47b4-992f-4df567ed653c","Type":"ContainerStarted","Data":"2d6d3bfe9a75400903c32b09560a61e469b147a42fc9b123ab67af4ce39cc070"}
Jul 29 00:05:03.893613 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:05:03.893564    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-fa6130-openshift-default-86f8b5f84-7nzs8" podStartSLOduration=1.893549014 podStartE2EDuration="1.893549014s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 00:05:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:05:03.890704742 +0000 UTC m=+2580.583379251" watchObservedRunningTime="2026-07-29 00:05:03.893549014 +0000 UTC m=+2580.586223523"
Jul 29 00:05:04.140388 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:05:04.140306    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-fa6130-openshift-default-86f8b5f84-7nzs8"
Jul 29 00:05:04.145326 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:05:04.145292    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-fa6130-openshift-default-86f8b5f84-7nzs8"
Jul 29 00:05:04.874900 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:05:04.874870    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-fa6130-openshift-default-86f8b5f84-7nzs8"
Jul 29 00:05:04.875965 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:05:04.875941    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-fa6130-openshift-default-86f8b5f84-7nzs8"
Jul 29 00:05:05.886933 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:05:05.886896    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-fa6130-78b7c5ddfb-rt8hz"]
Jul 29 00:05:05.887513 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:05:05.887356    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="33fe4e42-3adf-4ebc-8c1f-43646ee346b5" containerName="istio-proxy"
Jul 29 00:05:05.887513 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:05:05.887376    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="33fe4e42-3adf-4ebc-8c1f-43646ee346b5" containerName="istio-proxy"
Jul 29 00:05:05.891815 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:05:05.891789    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-fa6130-78b7c5ddfb-rt8hz"
Jul 29 00:05:05.894855 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:05:05.894833    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-fa6130-serving-cert\""
Jul 29 00:05:05.901255 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:05:05.901224    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-fa6130-78b7c5ddfb-rt8hz"]
Jul 29 00:05:06.033223 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:05:06.033184    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pbdwn\" (UniqueName: \"kubernetes.io/projected/38dcdca6-ce6a-449c-9c75-abc89f5fcec1-kube-api-access-pbdwn\") pod \"maas-api-e2e-shared-b-fa6130-78b7c5ddfb-rt8hz\" (UID: \"38dcdca6-ce6a-449c-9c75-abc89f5fcec1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-fa6130-78b7c5ddfb-rt8hz"
Jul 29 00:05:06.033401 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:05:06.033360    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/38dcdca6-ce6a-449c-9c75-abc89f5fcec1-maas-api-tls\") pod \"maas-api-e2e-shared-b-fa6130-78b7c5ddfb-rt8hz\" (UID: \"38dcdca6-ce6a-449c-9c75-abc89f5fcec1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-fa6130-78b7c5ddfb-rt8hz"
Jul 29 00:05:06.033450 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:05:06.033438    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/38dcdca6-ce6a-449c-9c75-abc89f5fcec1-openshift-service-ca\") pod \"maas-api-e2e-shared-b-fa6130-78b7c5ddfb-rt8hz\" (UID: \"38dcdca6-ce6a-449c-9c75-abc89f5fcec1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-fa6130-78b7c5ddfb-rt8hz"
Jul 29 00:05:06.134604 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:05:06.134562    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/38dcdca6-ce6a-449c-9c75-abc89f5fcec1-maas-api-tls\") pod \"maas-api-e2e-shared-b-fa6130-78b7c5ddfb-rt8hz\" (UID: \"38dcdca6-ce6a-449c-9c75-abc89f5fcec1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-fa6130-78b7c5ddfb-rt8hz"
Jul 29 00:05:06.134824 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:05:06.134617    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/38dcdca6-ce6a-449c-9c75-abc89f5fcec1-openshift-service-ca\") pod \"maas-api-e2e-shared-b-fa6130-78b7c5ddfb-rt8hz\" (UID: \"38dcdca6-ce6a-449c-9c75-abc89f5fcec1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-fa6130-78b7c5ddfb-rt8hz"
Jul 29 00:05:06.134824 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:05:06.134682    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pbdwn\" (UniqueName: \"kubernetes.io/projected/38dcdca6-ce6a-449c-9c75-abc89f5fcec1-kube-api-access-pbdwn\") pod \"maas-api-e2e-shared-b-fa6130-78b7c5ddfb-rt8hz\" (UID: \"38dcdca6-ce6a-449c-9c75-abc89f5fcec1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-fa6130-78b7c5ddfb-rt8hz"
Jul 29 00:05:06.135400 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:05:06.135381    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/38dcdca6-ce6a-449c-9c75-abc89f5fcec1-openshift-service-ca\") pod \"maas-api-e2e-shared-b-fa6130-78b7c5ddfb-rt8hz\" (UID: \"38dcdca6-ce6a-449c-9c75-abc89f5fcec1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-fa6130-78b7c5ddfb-rt8hz"
Jul 29 00:05:06.137411 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:05:06.137352    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/38dcdca6-ce6a-449c-9c75-abc89f5fcec1-maas-api-tls\") pod \"maas-api-e2e-shared-b-fa6130-78b7c5ddfb-rt8hz\" (UID: \"38dcdca6-ce6a-449c-9c75-abc89f5fcec1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-fa6130-78b7c5ddfb-rt8hz"
Jul 29 00:05:06.142967 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:05:06.142941    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pbdwn\" (UniqueName: \"kubernetes.io/projected/38dcdca6-ce6a-449c-9c75-abc89f5fcec1-kube-api-access-pbdwn\") pod \"maas-api-e2e-shared-b-fa6130-78b7c5ddfb-rt8hz\" (UID: \"38dcdca6-ce6a-449c-9c75-abc89f5fcec1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-fa6130-78b7c5ddfb-rt8hz"
Jul 29 00:05:06.207557 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:05:06.207521    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-fa6130-78b7c5ddfb-rt8hz"
Jul 29 00:05:06.339700 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:05:06.339677    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-fa6130-78b7c5ddfb-rt8hz"]
Jul 29 00:05:06.341832 ip-10-0-131-112 kubenswrapper[2572]: W0729 00:05:06.341805    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38dcdca6_ce6a_449c_9c75_abc89f5fcec1.slice/crio-4b0bce7e7a2774ec6354985b8d7afe3a1b1b3c08e731ff6c8a052fc6e42d2479 WatchSource:0}: Error finding container 4b0bce7e7a2774ec6354985b8d7afe3a1b1b3c08e731ff6c8a052fc6e42d2479: Status 404 returned error can't find the container with id 4b0bce7e7a2774ec6354985b8d7afe3a1b1b3c08e731ff6c8a052fc6e42d2479
Jul 29 00:05:06.884090 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:05:06.884057    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-fa6130-78b7c5ddfb-rt8hz" event={"ID":"38dcdca6-ce6a-449c-9c75-abc89f5fcec1","Type":"ContainerStarted","Data":"4b0bce7e7a2774ec6354985b8d7afe3a1b1b3c08e731ff6c8a052fc6e42d2479"}
Jul 29 00:05:07.890260 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:05:07.890226    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-fa6130-78b7c5ddfb-rt8hz" event={"ID":"38dcdca6-ce6a-449c-9c75-abc89f5fcec1","Type":"ContainerStarted","Data":"a6a3fc420cbfdf285f4959f3710fecbed79fce8e62d5b04462b7d8ae87897834"}
Jul 29 00:05:07.890695 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:05:07.890367    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-fa6130-78b7c5ddfb-rt8hz"
Jul 29 00:05:07.907528 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:05:07.907479    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-fa6130-78b7c5ddfb-rt8hz" podStartSLOduration=2.327451785 podStartE2EDuration="2.90746522s" totalImagesPullingTime="580.013435ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 00:05:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:05:07.905933007 +0000 UTC m=+2584.598607520" watchObservedRunningTime="2026-07-29 00:05:07.90746522 +0000 UTC m=+2584.600139728"
Jul 29 00:05:13.900784 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:05:13.900748    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-fa6130-78b7c5ddfb-rt8hz"
Jul 29 00:05:20.876376 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:05:20.876318    2572 patch_prober.go:28] interesting pod/e2e-shared-b-fa6130-openshift-default-86f8b5f84-7nzs8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 29 00:05:20.876949 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:05:20.876392    2572 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-fa6130-openshift-default-86f8b5f84-7nzs8" podUID="cf43995d-f389-47b4-992f-4df567ed653c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 29 00:05:21.962578 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:05:21.962543    2572 generic.go:356] "Generic (PLEG): container finished" podID="cf43995d-f389-47b4-992f-4df567ed653c" containerID="d439127331bda97a8104e9851dfd95a758dc0e68af8128e2788f65c61e2c202f" exitCode=0
Jul 29 00:05:21.963071 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:05:21.962604    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-fa6130-openshift-default-86f8b5f84-7nzs8" event={"ID":"cf43995d-f389-47b4-992f-4df567ed653c","Type":"ContainerDied","Data":"d439127331bda97a8104e9851dfd95a758dc0e68af8128e2788f65c61e2c202f"}
Jul 29 00:05:21.963071 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:05:21.962980    2572 scope.go:117] "RemoveContainer" containerID="d439127331bda97a8104e9851dfd95a758dc0e68af8128e2788f65c61e2c202f"
Jul 29 00:05:21.963989 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:05:21.963960    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 29 00:05:21.964060 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:05:21.964047    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 29 00:05:21.964101 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:05:21.964080    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 29 00:05:22.968784 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:05:22.968748    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-fa6130-openshift-default-86f8b5f84-7nzs8" event={"ID":"cf43995d-f389-47b4-992f-4df567ed653c","Type":"ContainerStarted","Data":"ecdd532590da6af16ab9dc705fa50968cdae04b6dd3298a539fc09f3bb963433"}
Jul 29 00:05:23.142052 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:05:23.141269    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-fa6130-openshift-default-86f8b5f84-7nzs8"
Jul 29 00:05:23.142052 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:05:23.141730    2572 patch_prober.go:28] interesting pod/e2e-shared-b-fa6130-openshift-default-86f8b5f84-7nzs8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body=
Jul 29 00:05:23.142052 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:05:23.141778    2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-fa6130-openshift-default-86f8b5f84-7nzs8" podUID="cf43995d-f389-47b4-992f-4df567ed653c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused"
Jul 29 00:05:24.140668 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:05:24.140613    2572 patch_prober.go:28] interesting pod/e2e-shared-b-fa6130-openshift-default-86f8b5f84-7nzs8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body=
Jul 29 00:05:24.141280 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:05:24.141239    2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-fa6130-openshift-default-86f8b5f84-7nzs8" podUID="cf43995d-f389-47b4-992f-4df567ed653c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused"
Jul 29 00:05:25.141291 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:05:25.141252    2572 patch_prober.go:28] interesting pod/e2e-shared-b-fa6130-openshift-default-86f8b5f84-7nzs8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body=
Jul 29 00:05:25.141907 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:05:25.141870    2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-fa6130-openshift-default-86f8b5f84-7nzs8" podUID="cf43995d-f389-47b4-992f-4df567ed653c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused"
Jul 29 00:05:26.140838 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:05:26.140797    2572 patch_prober.go:28] interesting pod/e2e-shared-b-fa6130-openshift-default-86f8b5f84-7nzs8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body=
Jul 29 00:05:26.141119 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:05:26.141089    2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-fa6130-openshift-default-86f8b5f84-7nzs8" podUID="cf43995d-f389-47b4-992f-4df567ed653c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused"
Jul 29 00:05:27.143676 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:05:27.143645    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-fa6130-openshift-default-86f8b5f84-7nzs8"
Jul 29 00:05:27.144058 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:05:27.143905    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-fa6130-openshift-default-86f8b5f84-7nzs8"
Jul 29 00:05:27.144649 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:05:27.144613    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-fa6130-openshift-default-86f8b5f84-7nzs8"
Jul 29 00:15:12.240679 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:15:12.240641    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-a-f318ab-openshift-default-f84669684-pxthv"]
Jul 29 00:15:12.243846 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:15:12.243829    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-f318ab-openshift-default-f84669684-pxthv"
Jul 29 00:15:12.248393 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:15:12.248370    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-a-f318ab-openshift-default-dockercfg-xr7sd\""
Jul 29 00:15:12.256349 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:15:12.256323    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-f318ab-openshift-default-f84669684-pxthv"]
Jul 29 00:15:12.362770 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:15:12.362735    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2530b4c9-9988-48e5-89ed-f61207a086f2-workload-socket\") pod \"e2e-ipp-a-f318ab-openshift-default-f84669684-pxthv\" (UID: \"2530b4c9-9988-48e5-89ed-f61207a086f2\") " pod="openshift-ingress/e2e-ipp-a-f318ab-openshift-default-f84669684-pxthv"
Jul 29 00:15:12.362770 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:15:12.362773    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2530b4c9-9988-48e5-89ed-f61207a086f2-istio-token\") pod \"e2e-ipp-a-f318ab-openshift-default-f84669684-pxthv\" (UID: \"2530b4c9-9988-48e5-89ed-f61207a086f2\") " pod="openshift-ingress/e2e-ipp-a-f318ab-openshift-default-f84669684-pxthv"
Jul 29 00:15:12.363040 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:15:12.362791    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2530b4c9-9988-48e5-89ed-f61207a086f2-istio-envoy\") pod \"e2e-ipp-a-f318ab-openshift-default-f84669684-pxthv\" (UID: \"2530b4c9-9988-48e5-89ed-f61207a086f2\") " pod="openshift-ingress/e2e-ipp-a-f318ab-openshift-default-f84669684-pxthv"
Jul 29 00:15:12.363040 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:15:12.362856    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2530b4c9-9988-48e5-89ed-f61207a086f2-istio-data\") pod \"e2e-ipp-a-f318ab-openshift-default-f84669684-pxthv\" (UID: \"2530b4c9-9988-48e5-89ed-f61207a086f2\") " pod="openshift-ingress/e2e-ipp-a-f318ab-openshift-default-f84669684-pxthv"
Jul 29 00:15:12.363040 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:15:12.362935    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fcrrw\" (UniqueName: \"kubernetes.io/projected/2530b4c9-9988-48e5-89ed-f61207a086f2-kube-api-access-fcrrw\") pod \"e2e-ipp-a-f318ab-openshift-default-f84669684-pxthv\" (UID: \"2530b4c9-9988-48e5-89ed-f61207a086f2\") " pod="openshift-ingress/e2e-ipp-a-f318ab-openshift-default-f84669684-pxthv"
Jul 29 00:15:12.363040 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:15:12.363006    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2530b4c9-9988-48e5-89ed-f61207a086f2-credential-socket\") pod \"e2e-ipp-a-f318ab-openshift-default-f84669684-pxthv\" (UID: \"2530b4c9-9988-48e5-89ed-f61207a086f2\") " pod="openshift-ingress/e2e-ipp-a-f318ab-openshift-default-f84669684-pxthv"
Jul 29 00:15:12.363259 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:15:12.363073    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2530b4c9-9988-48e5-89ed-f61207a086f2-istio-podinfo\") pod \"e2e-ipp-a-f318ab-openshift-default-f84669684-pxthv\" (UID: \"2530b4c9-9988-48e5-89ed-f61207a086f2\") " pod="openshift-ingress/e2e-ipp-a-f318ab-openshift-default-f84669684-pxthv"
Jul 29 00:15:12.363259 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:15:12.363095    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2530b4c9-9988-48e5-89ed-f61207a086f2-workload-certs\") pod \"e2e-ipp-a-f318ab-openshift-default-f84669684-pxthv\" (UID: \"2530b4c9-9988-48e5-89ed-f61207a086f2\") " pod="openshift-ingress/e2e-ipp-a-f318ab-openshift-default-f84669684-pxthv"
Jul 29 00:15:12.363259 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:15:12.363115    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2530b4c9-9988-48e5-89ed-f61207a086f2-istiod-ca-cert\") pod \"e2e-ipp-a-f318ab-openshift-default-f84669684-pxthv\" (UID: \"2530b4c9-9988-48e5-89ed-f61207a086f2\") " pod="openshift-ingress/e2e-ipp-a-f318ab-openshift-default-f84669684-pxthv"
Jul 29 00:15:12.464117 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:15:12.464082    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2530b4c9-9988-48e5-89ed-f61207a086f2-credential-socket\") pod \"e2e-ipp-a-f318ab-openshift-default-f84669684-pxthv\" (UID: \"2530b4c9-9988-48e5-89ed-f61207a086f2\") " pod="openshift-ingress/e2e-ipp-a-f318ab-openshift-default-f84669684-pxthv"
Jul 29 00:15:12.464300 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:15:12.464126    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2530b4c9-9988-48e5-89ed-f61207a086f2-istio-podinfo\") pod \"e2e-ipp-a-f318ab-openshift-default-f84669684-pxthv\" (UID: \"2530b4c9-9988-48e5-89ed-f61207a086f2\") " pod="openshift-ingress/e2e-ipp-a-f318ab-openshift-default-f84669684-pxthv"
Jul 29 00:15:12.464300 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:15:12.464150    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2530b4c9-9988-48e5-89ed-f61207a086f2-workload-certs\") pod \"e2e-ipp-a-f318ab-openshift-default-f84669684-pxthv\" (UID: \"2530b4c9-9988-48e5-89ed-f61207a086f2\") " pod="openshift-ingress/e2e-ipp-a-f318ab-openshift-default-f84669684-pxthv"
Jul 29 00:15:12.464300 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:15:12.464175    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2530b4c9-9988-48e5-89ed-f61207a086f2-istiod-ca-cert\") pod \"e2e-ipp-a-f318ab-openshift-default-f84669684-pxthv\" (UID: \"2530b4c9-9988-48e5-89ed-f61207a086f2\") " pod="openshift-ingress/e2e-ipp-a-f318ab-openshift-default-f84669684-pxthv"
Jul 29 00:15:12.464442 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:15:12.464302    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2530b4c9-9988-48e5-89ed-f61207a086f2-workload-socket\") pod \"e2e-ipp-a-f318ab-openshift-default-f84669684-pxthv\" (UID: \"2530b4c9-9988-48e5-89ed-f61207a086f2\") " pod="openshift-ingress/e2e-ipp-a-f318ab-openshift-default-f84669684-pxthv"
Jul 29 00:15:12.464442 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:15:12.464352    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2530b4c9-9988-48e5-89ed-f61207a086f2-istio-token\") pod \"e2e-ipp-a-f318ab-openshift-default-f84669684-pxthv\" (UID: \"2530b4c9-9988-48e5-89ed-f61207a086f2\") " pod="openshift-ingress/e2e-ipp-a-f318ab-openshift-default-f84669684-pxthv"
Jul 29 00:15:12.464442 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:15:12.464384    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2530b4c9-9988-48e5-89ed-f61207a086f2-istio-envoy\") pod \"e2e-ipp-a-f318ab-openshift-default-f84669684-pxthv\" (UID: \"2530b4c9-9988-48e5-89ed-f61207a086f2\") " pod="openshift-ingress/e2e-ipp-a-f318ab-openshift-default-f84669684-pxthv"
Jul 29 00:15:12.464442 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:15:12.464405    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2530b4c9-9988-48e5-89ed-f61207a086f2-istio-data\") pod \"e2e-ipp-a-f318ab-openshift-default-f84669684-pxthv\" (UID: \"2530b4c9-9988-48e5-89ed-f61207a086f2\") " pod="openshift-ingress/e2e-ipp-a-f318ab-openshift-default-f84669684-pxthv"
Jul 29 00:15:12.464664 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:15:12.464553    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2530b4c9-9988-48e5-89ed-f61207a086f2-credential-socket\") pod \"e2e-ipp-a-f318ab-openshift-default-f84669684-pxthv\" (UID: \"2530b4c9-9988-48e5-89ed-f61207a086f2\") " pod="openshift-ingress/e2e-ipp-a-f318ab-openshift-default-f84669684-pxthv"
Jul 29 00:15:12.464664 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:15:12.464569    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2530b4c9-9988-48e5-89ed-f61207a086f2-workload-certs\") pod \"e2e-ipp-a-f318ab-openshift-default-f84669684-pxthv\" (UID: \"2530b4c9-9988-48e5-89ed-f61207a086f2\") " pod="openshift-ingress/e2e-ipp-a-f318ab-openshift-default-f84669684-pxthv"
Jul 29 00:15:12.464772 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:15:12.464662    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fcrrw\" (UniqueName: \"kubernetes.io/projected/2530b4c9-9988-48e5-89ed-f61207a086f2-kube-api-access-fcrrw\") pod \"e2e-ipp-a-f318ab-openshift-default-f84669684-pxthv\" (UID: \"2530b4c9-9988-48e5-89ed-f61207a086f2\") " pod="openshift-ingress/e2e-ipp-a-f318ab-openshift-default-f84669684-pxthv"
Jul 29 00:15:12.464772 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:15:12.464709    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2530b4c9-9988-48e5-89ed-f61207a086f2-workload-socket\") pod \"e2e-ipp-a-f318ab-openshift-default-f84669684-pxthv\" (UID: \"2530b4c9-9988-48e5-89ed-f61207a086f2\") " pod="openshift-ingress/e2e-ipp-a-f318ab-openshift-default-f84669684-pxthv"
Jul 29 00:15:12.464938 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:15:12.464917    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2530b4c9-9988-48e5-89ed-f61207a086f2-istio-data\") pod \"e2e-ipp-a-f318ab-openshift-default-f84669684-pxthv\" (UID: \"2530b4c9-9988-48e5-89ed-f61207a086f2\") " pod="openshift-ingress/e2e-ipp-a-f318ab-openshift-default-f84669684-pxthv"
Jul 29 00:15:12.465047 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:15:12.465026    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2530b4c9-9988-48e5-89ed-f61207a086f2-istiod-ca-cert\") pod \"e2e-ipp-a-f318ab-openshift-default-f84669684-pxthv\" (UID: \"2530b4c9-9988-48e5-89ed-f61207a086f2\") " pod="openshift-ingress/e2e-ipp-a-f318ab-openshift-default-f84669684-pxthv"
Jul 29 00:15:12.467096 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:15:12.467076    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2530b4c9-9988-48e5-89ed-f61207a086f2-istio-envoy\") pod \"e2e-ipp-a-f318ab-openshift-default-f84669684-pxthv\" (UID: \"2530b4c9-9988-48e5-89ed-f61207a086f2\") " pod="openshift-ingress/e2e-ipp-a-f318ab-openshift-default-f84669684-pxthv"
Jul 29 00:15:12.467324 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:15:12.467305    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2530b4c9-9988-48e5-89ed-f61207a086f2-istio-podinfo\") pod \"e2e-ipp-a-f318ab-openshift-default-f84669684-pxthv\" (UID: \"2530b4c9-9988-48e5-89ed-f61207a086f2\") " pod="openshift-ingress/e2e-ipp-a-f318ab-openshift-default-f84669684-pxthv"
Jul 29 00:15:12.477520 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:15:12.477497    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2530b4c9-9988-48e5-89ed-f61207a086f2-istio-token\") pod \"e2e-ipp-a-f318ab-openshift-default-f84669684-pxthv\" (UID: \"2530b4c9-9988-48e5-89ed-f61207a086f2\") " pod="openshift-ingress/e2e-ipp-a-f318ab-openshift-default-f84669684-pxthv"
Jul 29 00:15:12.479719 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:15:12.479696    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fcrrw\" (UniqueName: \"kubernetes.io/projected/2530b4c9-9988-48e5-89ed-f61207a086f2-kube-api-access-fcrrw\") pod \"e2e-ipp-a-f318ab-openshift-default-f84669684-pxthv\" (UID: \"2530b4c9-9988-48e5-89ed-f61207a086f2\") " pod="openshift-ingress/e2e-ipp-a-f318ab-openshift-default-f84669684-pxthv"
Jul 29 00:15:12.554099 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:15:12.553997    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-f318ab-openshift-default-f84669684-pxthv"
Jul 29 00:15:12.695808 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:15:12.695785    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-f318ab-openshift-default-f84669684-pxthv"]
Jul 29 00:15:12.697885 ip-10-0-131-112 kubenswrapper[2572]: W0729 00:15:12.697857    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2530b4c9_9988_48e5_89ed_f61207a086f2.slice/crio-25eddee4362c2bb124835ae0c5a0961129298aa63691ed35fa79e6cd5bd24766 WatchSource:0}: Error finding container 25eddee4362c2bb124835ae0c5a0961129298aa63691ed35fa79e6cd5bd24766: Status 404 returned error can't find the container with id 25eddee4362c2bb124835ae0c5a0961129298aa63691ed35fa79e6cd5bd24766
Jul 29 00:15:12.699911 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:15:12.699883    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 29 00:15:12.699993 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:15:12.699940    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 29 00:15:12.699993 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:15:12.699967    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 29 00:15:13.522820 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:15:13.522781    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-f318ab-openshift-default-f84669684-pxthv" event={"ID":"2530b4c9-9988-48e5-89ed-f61207a086f2","Type":"ContainerStarted","Data":"8b4f449ec422a059e31fed045b93154d9238d014d70180666eba14905531df55"}
Jul 29 00:15:13.523464 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:15:13.522835    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-f318ab-openshift-default-f84669684-pxthv" event={"ID":"2530b4c9-9988-48e5-89ed-f61207a086f2","Type":"ContainerStarted","Data":"25eddee4362c2bb124835ae0c5a0961129298aa63691ed35fa79e6cd5bd24766"}
Jul 29 00:15:13.551291 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:15:13.551237    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-a-f318ab-openshift-default-f84669684-pxthv" podStartSLOduration=1.551221613 podStartE2EDuration="1.551221613s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 00:15:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:15:13.549101566 +0000 UTC m=+3190.241776075" watchObservedRunningTime="2026-07-29 00:15:13.551221613 +0000 UTC m=+3190.243896121"
Jul 29 00:15:13.554382 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:15:13.554362    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-f318ab-openshift-default-f84669684-pxthv"
Jul 29 00:15:13.559444 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:15:13.559416    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-f318ab-openshift-default-f84669684-pxthv"
Jul 29 00:15:14.527603 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:15:14.527570    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-f318ab-openshift-default-f84669684-pxthv"
Jul 29 00:15:14.528702 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:15:14.528681    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-f318ab-openshift-default-f84669684-pxthv"
Jul 29 00:15:30.529407 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:15:30.529367    2572 patch_prober.go:28] interesting pod/e2e-ipp-a-f318ab-openshift-default-f84669684-pxthv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 29 00:15:30.529866 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:15:30.529443    2572 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-a-f318ab-openshift-default-f84669684-pxthv" podUID="2530b4c9-9988-48e5-89ed-f61207a086f2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 29 00:15:32.610474 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:15:32.610443    2572 generic.go:356] "Generic (PLEG): container finished" podID="2530b4c9-9988-48e5-89ed-f61207a086f2" containerID="8b4f449ec422a059e31fed045b93154d9238d014d70180666eba14905531df55" exitCode=0
Jul 29 00:15:32.610912 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:15:32.610516    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-f318ab-openshift-default-f84669684-pxthv" event={"ID":"2530b4c9-9988-48e5-89ed-f61207a086f2","Type":"ContainerDied","Data":"8b4f449ec422a059e31fed045b93154d9238d014d70180666eba14905531df55"}
Jul 29 00:15:32.610912 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:15:32.610890    2572 scope.go:117] "RemoveContainer" containerID="8b4f449ec422a059e31fed045b93154d9238d014d70180666eba14905531df55"
Jul 29 00:15:32.611953 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:15:32.611926    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 29 00:15:32.612008 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:15:32.611998    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 29 00:15:32.612044 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:15:32.612033    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 29 00:15:33.617252 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:15:33.617218    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-f318ab-openshift-default-f84669684-pxthv" event={"ID":"2530b4c9-9988-48e5-89ed-f61207a086f2","Type":"ContainerStarted","Data":"b133463e90c8c6e36e2ba7c714decbc97523af84f1915dca7d6dcb05cf6da44a"}
Jul 29 00:15:34.555172 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:15:34.555141    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-f318ab-openshift-default-f84669684-pxthv"
Jul 29 00:15:34.555364 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:15:34.555346    2572 patch_prober.go:28] interesting pod/e2e-ipp-a-f318ab-openshift-default-f84669684-pxthv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body=
Jul 29 00:15:34.555414 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:15:34.555388    2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-f318ab-openshift-default-f84669684-pxthv" podUID="2530b4c9-9988-48e5-89ed-f61207a086f2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused"
Jul 29 00:15:35.555337 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:15:35.555300    2572 patch_prober.go:28] interesting pod/e2e-ipp-a-f318ab-openshift-default-f84669684-pxthv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body=
Jul 29 00:15:35.555758 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:15:35.555360    2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-f318ab-openshift-default-f84669684-pxthv" podUID="2530b4c9-9988-48e5-89ed-f61207a086f2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused"
Jul 29 00:15:36.554369 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:15:36.554332    2572 patch_prober.go:28] interesting pod/e2e-ipp-a-f318ab-openshift-default-f84669684-pxthv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body=
Jul 29 00:15:36.554564 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:15:36.554518    2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-f318ab-openshift-default-f84669684-pxthv" podUID="2530b4c9-9988-48e5-89ed-f61207a086f2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused"
Jul 29 00:15:37.558212 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:15:37.558173    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-f318ab-openshift-default-f84669684-pxthv"
Jul 29 00:15:37.558702 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:15:37.558576    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-f318ab-openshift-default-f84669684-pxthv"
Jul 29 00:15:37.559198 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:15:37.559177    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-f318ab-openshift-default-f84669684-pxthv"
Jul 29 00:17:04.046665 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:17:04.046617    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-e26439-openshift-default-77854bc9c4-76fqp"]
Jul 29 00:17:04.049155 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:17:04.049128    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-e26439-openshift-default-77854bc9c4-76fqp"
Jul 29 00:17:04.051851 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:17:04.051825    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-cleanup-e26439-openshift-default-dockercfg-hg989\""
Jul 29 00:17:04.058046 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:17:04.058024    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/82b6488a-9627-4ac3-b8fb-8aa5d958fab0-istio-token\") pod \"e2e-ipp-cleanup-e26439-openshift-default-77854bc9c4-76fqp\" (UID: \"82b6488a-9627-4ac3-b8fb-8aa5d958fab0\") " pod="openshift-ingress/e2e-ipp-cleanup-e26439-openshift-default-77854bc9c4-76fqp"
Jul 29 00:17:04.058150 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:17:04.058081    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/82b6488a-9627-4ac3-b8fb-8aa5d958fab0-credential-socket\") pod \"e2e-ipp-cleanup-e26439-openshift-default-77854bc9c4-76fqp\" (UID: \"82b6488a-9627-4ac3-b8fb-8aa5d958fab0\") " pod="openshift-ingress/e2e-ipp-cleanup-e26439-openshift-default-77854bc9c4-76fqp"
Jul 29 00:17:04.058150 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:17:04.058107    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/82b6488a-9627-4ac3-b8fb-8aa5d958fab0-workload-certs\") pod \"e2e-ipp-cleanup-e26439-openshift-default-77854bc9c4-76fqp\" (UID: \"82b6488a-9627-4ac3-b8fb-8aa5d958fab0\") " pod="openshift-ingress/e2e-ipp-cleanup-e26439-openshift-default-77854bc9c4-76fqp"
Jul 29 00:17:04.058245 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:17:04.058153    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/82b6488a-9627-4ac3-b8fb-8aa5d958fab0-workload-socket\") pod \"e2e-ipp-cleanup-e26439-openshift-default-77854bc9c4-76fqp\" (UID: \"82b6488a-9627-4ac3-b8fb-8aa5d958fab0\") " pod="openshift-ingress/e2e-ipp-cleanup-e26439-openshift-default-77854bc9c4-76fqp"
Jul 29 00:17:04.058245 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:17:04.058224    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/82b6488a-9627-4ac3-b8fb-8aa5d958fab0-istio-data\") pod \"e2e-ipp-cleanup-e26439-openshift-default-77854bc9c4-76fqp\" (UID: \"82b6488a-9627-4ac3-b8fb-8aa5d958fab0\") " pod="openshift-ingress/e2e-ipp-cleanup-e26439-openshift-default-77854bc9c4-76fqp"
Jul 29 00:17:04.058329 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:17:04.058264    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/82b6488a-9627-4ac3-b8fb-8aa5d958fab0-istio-podinfo\") pod \"e2e-ipp-cleanup-e26439-openshift-default-77854bc9c4-76fqp\" (UID: \"82b6488a-9627-4ac3-b8fb-8aa5d958fab0\") " pod="openshift-ingress/e2e-ipp-cleanup-e26439-openshift-default-77854bc9c4-76fqp"
Jul 29 00:17:04.058376 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:17:04.058328    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/82b6488a-9627-4ac3-b8fb-8aa5d958fab0-istio-envoy\") pod \"e2e-ipp-cleanup-e26439-openshift-default-77854bc9c4-76fqp\" (UID: \"82b6488a-9627-4ac3-b8fb-8aa5d958fab0\") " pod="openshift-ingress/e2e-ipp-cleanup-e26439-openshift-default-77854bc9c4-76fqp"
Jul 29 00:17:04.058376 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:17:04.058355    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/82b6488a-9627-4ac3-b8fb-8aa5d958fab0-istiod-ca-cert\") pod \"e2e-ipp-cleanup-e26439-openshift-default-77854bc9c4-76fqp\" (UID: \"82b6488a-9627-4ac3-b8fb-8aa5d958fab0\") " pod="openshift-ingress/e2e-ipp-cleanup-e26439-openshift-default-77854bc9c4-76fqp"
Jul 29 00:17:04.058457 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:17:04.058390    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bkcpk\" (UniqueName: \"kubernetes.io/projected/82b6488a-9627-4ac3-b8fb-8aa5d958fab0-kube-api-access-bkcpk\") pod \"e2e-ipp-cleanup-e26439-openshift-default-77854bc9c4-76fqp\" (UID: \"82b6488a-9627-4ac3-b8fb-8aa5d958fab0\") " pod="openshift-ingress/e2e-ipp-cleanup-e26439-openshift-default-77854bc9c4-76fqp"
Jul 29 00:17:04.062127 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:17:04.062103    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-e26439-openshift-default-77854bc9c4-76fqp"]
Jul 29 00:17:04.159408 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:17:04.159365    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/82b6488a-9627-4ac3-b8fb-8aa5d958fab0-workload-socket\") pod \"e2e-ipp-cleanup-e26439-openshift-default-77854bc9c4-76fqp\" (UID: \"82b6488a-9627-4ac3-b8fb-8aa5d958fab0\") " pod="openshift-ingress/e2e-ipp-cleanup-e26439-openshift-default-77854bc9c4-76fqp"
Jul 29 00:17:04.160187 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:17:04.159415    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/82b6488a-9627-4ac3-b8fb-8aa5d958fab0-istio-data\") pod \"e2e-ipp-cleanup-e26439-openshift-default-77854bc9c4-76fqp\" (UID: \"82b6488a-9627-4ac3-b8fb-8aa5d958fab0\") " pod="openshift-ingress/e2e-ipp-cleanup-e26439-openshift-default-77854bc9c4-76fqp"
Jul 29 00:17:04.160187 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:17:04.159443    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/82b6488a-9627-4ac3-b8fb-8aa5d958fab0-istio-podinfo\") pod \"e2e-ipp-cleanup-e26439-openshift-default-77854bc9c4-76fqp\" (UID: \"82b6488a-9627-4ac3-b8fb-8aa5d958fab0\") " pod="openshift-ingress/e2e-ipp-cleanup-e26439-openshift-default-77854bc9c4-76fqp"
Jul 29 00:17:04.160187 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:17:04.159497    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/82b6488a-9627-4ac3-b8fb-8aa5d958fab0-istio-envoy\") pod \"e2e-ipp-cleanup-e26439-openshift-default-77854bc9c4-76fqp\" (UID: \"82b6488a-9627-4ac3-b8fb-8aa5d958fab0\") " pod="openshift-ingress/e2e-ipp-cleanup-e26439-openshift-default-77854bc9c4-76fqp"
Jul 29 00:17:04.160187 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:17:04.159521    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/82b6488a-9627-4ac3-b8fb-8aa5d958fab0-istiod-ca-cert\") pod \"e2e-ipp-cleanup-e26439-openshift-default-77854bc9c4-76fqp\" (UID: \"82b6488a-9627-4ac3-b8fb-8aa5d958fab0\") " pod="openshift-ingress/e2e-ipp-cleanup-e26439-openshift-default-77854bc9c4-76fqp"
Jul 29 00:17:04.160187 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:17:04.159560    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bkcpk\" (UniqueName: \"kubernetes.io/projected/82b6488a-9627-4ac3-b8fb-8aa5d958fab0-kube-api-access-bkcpk\") pod \"e2e-ipp-cleanup-e26439-openshift-default-77854bc9c4-76fqp\" (UID: \"82b6488a-9627-4ac3-b8fb-8aa5d958fab0\") " pod="openshift-ingress/e2e-ipp-cleanup-e26439-openshift-default-77854bc9c4-76fqp"
Jul 29 00:17:04.160187 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:17:04.159616    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/82b6488a-9627-4ac3-b8fb-8aa5d958fab0-istio-token\") pod \"e2e-ipp-cleanup-e26439-openshift-default-77854bc9c4-76fqp\" (UID: \"82b6488a-9627-4ac3-b8fb-8aa5d958fab0\") " pod="openshift-ingress/e2e-ipp-cleanup-e26439-openshift-default-77854bc9c4-76fqp"
Jul 29 00:17:04.160187 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:17:04.159698    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/82b6488a-9627-4ac3-b8fb-8aa5d958fab0-credential-socket\") pod \"e2e-ipp-cleanup-e26439-openshift-default-77854bc9c4-76fqp\" (UID: \"82b6488a-9627-4ac3-b8fb-8aa5d958fab0\") " pod="openshift-ingress/e2e-ipp-cleanup-e26439-openshift-default-77854bc9c4-76fqp"
Jul 29 00:17:04.160187 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:17:04.159723    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/82b6488a-9627-4ac3-b8fb-8aa5d958fab0-workload-certs\") pod \"e2e-ipp-cleanup-e26439-openshift-default-77854bc9c4-76fqp\" (UID: \"82b6488a-9627-4ac3-b8fb-8aa5d958fab0\") " pod="openshift-ingress/e2e-ipp-cleanup-e26439-openshift-default-77854bc9c4-76fqp"
Jul 29 00:17:04.160575 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:17:04.160330    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/82b6488a-9627-4ac3-b8fb-8aa5d958fab0-workload-socket\") pod \"e2e-ipp-cleanup-e26439-openshift-default-77854bc9c4-76fqp\" (UID: \"82b6488a-9627-4ac3-b8fb-8aa5d958fab0\") " pod="openshift-ingress/e2e-ipp-cleanup-e26439-openshift-default-77854bc9c4-76fqp"
Jul 29 00:17:04.160575 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:17:04.160401    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/82b6488a-9627-4ac3-b8fb-8aa5d958fab0-credential-socket\") pod \"e2e-ipp-cleanup-e26439-openshift-default-77854bc9c4-76fqp\" (UID: \"82b6488a-9627-4ac3-b8fb-8aa5d958fab0\") " pod="openshift-ingress/e2e-ipp-cleanup-e26439-openshift-default-77854bc9c4-76fqp"
Jul 29 00:17:04.160575 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:17:04.160556    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/82b6488a-9627-4ac3-b8fb-8aa5d958fab0-istio-data\") pod \"e2e-ipp-cleanup-e26439-openshift-default-77854bc9c4-76fqp\" (UID: \"82b6488a-9627-4ac3-b8fb-8aa5d958fab0\") " pod="openshift-ingress/e2e-ipp-cleanup-e26439-openshift-default-77854bc9c4-76fqp"
Jul 29 00:17:04.161107 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:17:04.161077    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/82b6488a-9627-4ac3-b8fb-8aa5d958fab0-istiod-ca-cert\") pod \"e2e-ipp-cleanup-e26439-openshift-default-77854bc9c4-76fqp\" (UID: \"82b6488a-9627-4ac3-b8fb-8aa5d958fab0\") " pod="openshift-ingress/e2e-ipp-cleanup-e26439-openshift-default-77854bc9c4-76fqp"
Jul 29 00:17:04.161217 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:17:04.161114    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/82b6488a-9627-4ac3-b8fb-8aa5d958fab0-workload-certs\") pod \"e2e-ipp-cleanup-e26439-openshift-default-77854bc9c4-76fqp\" (UID: \"82b6488a-9627-4ac3-b8fb-8aa5d958fab0\") " pod="openshift-ingress/e2e-ipp-cleanup-e26439-openshift-default-77854bc9c4-76fqp"
Jul 29 00:17:04.163254 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:17:04.163234    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/82b6488a-9627-4ac3-b8fb-8aa5d958fab0-istio-envoy\") pod \"e2e-ipp-cleanup-e26439-openshift-default-77854bc9c4-76fqp\" (UID: \"82b6488a-9627-4ac3-b8fb-8aa5d958fab0\") " pod="openshift-ingress/e2e-ipp-cleanup-e26439-openshift-default-77854bc9c4-76fqp"
Jul 29 00:17:04.169233 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:17:04.166163    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/82b6488a-9627-4ac3-b8fb-8aa5d958fab0-istio-podinfo\") pod \"e2e-ipp-cleanup-e26439-openshift-default-77854bc9c4-76fqp\" (UID: \"82b6488a-9627-4ac3-b8fb-8aa5d958fab0\") " pod="openshift-ingress/e2e-ipp-cleanup-e26439-openshift-default-77854bc9c4-76fqp"
Jul 29 00:17:04.173712 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:17:04.173683    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/82b6488a-9627-4ac3-b8fb-8aa5d958fab0-istio-token\") pod \"e2e-ipp-cleanup-e26439-openshift-default-77854bc9c4-76fqp\" (UID: \"82b6488a-9627-4ac3-b8fb-8aa5d958fab0\") " pod="openshift-ingress/e2e-ipp-cleanup-e26439-openshift-default-77854bc9c4-76fqp"
Jul 29 00:17:04.174013 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:17:04.173955    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bkcpk\" (UniqueName: \"kubernetes.io/projected/82b6488a-9627-4ac3-b8fb-8aa5d958fab0-kube-api-access-bkcpk\") pod \"e2e-ipp-cleanup-e26439-openshift-default-77854bc9c4-76fqp\" (UID: \"82b6488a-9627-4ac3-b8fb-8aa5d958fab0\") " pod="openshift-ingress/e2e-ipp-cleanup-e26439-openshift-default-77854bc9c4-76fqp"
Jul 29 00:17:04.360390 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:17:04.360284    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-e26439-openshift-default-77854bc9c4-76fqp"
Jul 29 00:17:04.498651 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:17:04.498595    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-e26439-openshift-default-77854bc9c4-76fqp"]
Jul 29 00:17:04.500498 ip-10-0-131-112 kubenswrapper[2572]: W0729 00:17:04.500472    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82b6488a_9627_4ac3_b8fb_8aa5d958fab0.slice/crio-2aa8fe1abe13cc0ccd1c6af5bddec1ae23118ffa4ab2aaf450d6fda104a7277c WatchSource:0}: Error finding container 2aa8fe1abe13cc0ccd1c6af5bddec1ae23118ffa4ab2aaf450d6fda104a7277c: Status 404 returned error can't find the container with id 2aa8fe1abe13cc0ccd1c6af5bddec1ae23118ffa4ab2aaf450d6fda104a7277c
Jul 29 00:17:04.502771 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:17:04.502742    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 29 00:17:04.502833 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:17:04.502807    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 29 00:17:04.502867 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:17:04.502837    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 29 00:17:05.030817 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:17:05.030774    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-e26439-openshift-default-77854bc9c4-76fqp" event={"ID":"82b6488a-9627-4ac3-b8fb-8aa5d958fab0","Type":"ContainerStarted","Data":"af5a241a7f0c1650fe1b8901dcc5d778e8cde48714b255606ebc36722a2a3a50"}
Jul 29 00:17:05.031010 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:17:05.030842    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-e26439-openshift-default-77854bc9c4-76fqp" event={"ID":"82b6488a-9627-4ac3-b8fb-8aa5d958fab0","Type":"ContainerStarted","Data":"2aa8fe1abe13cc0ccd1c6af5bddec1ae23118ffa4ab2aaf450d6fda104a7277c"}
Jul 29 00:17:05.049478 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:17:05.049433    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-cleanup-e26439-openshift-default-77854bc9c4-76fqp" podStartSLOduration=1.049418376 podStartE2EDuration="1.049418376s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 00:17:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:17:05.048098934 +0000 UTC m=+3301.740773448" watchObservedRunningTime="2026-07-29 00:17:05.049418376 +0000 UTC m=+3301.742092884"
Jul 29 00:17:05.361004 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:17:05.360908    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-e26439-openshift-default-77854bc9c4-76fqp"
Jul 29 00:17:05.365784 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:17:05.365760    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-e26439-openshift-default-77854bc9c4-76fqp"
Jul 29 00:17:06.034942 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:17:06.034907    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-e26439-openshift-default-77854bc9c4-76fqp"
Jul 29 00:17:06.035926 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:17:06.035904    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-e26439-openshift-default-77854bc9c4-76fqp"
Jul 29 00:17:07.293557 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:17:07.293525    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e26439-7c574cb686-w85n4"]
Jul 29 00:17:07.296178 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:17:07.296157    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e26439-7c574cb686-w85n4"
Jul 29 00:17:07.298680 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:17:07.298663    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-cleanup-e26439-serving-cert\""
Jul 29 00:17:07.307241 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:17:07.307213    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e26439-7c574cb686-w85n4"]
Jul 29 00:17:07.386541 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:17:07.386505    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kwfn6\" (UniqueName: \"kubernetes.io/projected/59064777-3621-46f7-9835-c4e1a604fbec-kube-api-access-kwfn6\") pod \"maas-api-e2e-ipp-cleanup-e26439-7c574cb686-w85n4\" (UID: \"59064777-3621-46f7-9835-c4e1a604fbec\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e26439-7c574cb686-w85n4"
Jul 29 00:17:07.386885 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:17:07.386856    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/59064777-3621-46f7-9835-c4e1a604fbec-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-e26439-7c574cb686-w85n4\" (UID: \"59064777-3621-46f7-9835-c4e1a604fbec\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e26439-7c574cb686-w85n4"
Jul 29 00:17:07.387022 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:17:07.386927    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/59064777-3621-46f7-9835-c4e1a604fbec-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-e26439-7c574cb686-w85n4\" (UID: \"59064777-3621-46f7-9835-c4e1a604fbec\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e26439-7c574cb686-w85n4"
Jul 29 00:17:07.487934 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:17:07.487898    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kwfn6\" (UniqueName: \"kubernetes.io/projected/59064777-3621-46f7-9835-c4e1a604fbec-kube-api-access-kwfn6\") pod \"maas-api-e2e-ipp-cleanup-e26439-7c574cb686-w85n4\" (UID: \"59064777-3621-46f7-9835-c4e1a604fbec\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e26439-7c574cb686-w85n4"
Jul 29 00:17:07.488109 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:17:07.487968    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/59064777-3621-46f7-9835-c4e1a604fbec-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-e26439-7c574cb686-w85n4\" (UID: \"59064777-3621-46f7-9835-c4e1a604fbec\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e26439-7c574cb686-w85n4"
Jul 29 00:17:07.488109 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:17:07.488006    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/59064777-3621-46f7-9835-c4e1a604fbec-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-e26439-7c574cb686-w85n4\" (UID: \"59064777-3621-46f7-9835-c4e1a604fbec\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e26439-7c574cb686-w85n4"
Jul 29 00:17:07.488760 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:17:07.488742    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/59064777-3621-46f7-9835-c4e1a604fbec-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-e26439-7c574cb686-w85n4\" (UID: \"59064777-3621-46f7-9835-c4e1a604fbec\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e26439-7c574cb686-w85n4"
Jul 29 00:17:07.490529 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:17:07.490503    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/59064777-3621-46f7-9835-c4e1a604fbec-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-e26439-7c574cb686-w85n4\" (UID: \"59064777-3621-46f7-9835-c4e1a604fbec\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e26439-7c574cb686-w85n4"
Jul 29 00:17:07.496272 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:17:07.496249    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kwfn6\" (UniqueName: \"kubernetes.io/projected/59064777-3621-46f7-9835-c4e1a604fbec-kube-api-access-kwfn6\") pod \"maas-api-e2e-ipp-cleanup-e26439-7c574cb686-w85n4\" (UID: \"59064777-3621-46f7-9835-c4e1a604fbec\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e26439-7c574cb686-w85n4"
Jul 29 00:17:07.610491 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:17:07.610408    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e26439-7c574cb686-w85n4"
Jul 29 00:17:07.744757 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:17:07.744733    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e26439-7c574cb686-w85n4"]
Jul 29 00:17:07.746341 ip-10-0-131-112 kubenswrapper[2572]: W0729 00:17:07.746310    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod59064777_3621_46f7_9835_c4e1a604fbec.slice/crio-fdace70c726795454f4edcfcc118cfff1fa54c70938bb9715ecfe7d41282fa4f WatchSource:0}: Error finding container fdace70c726795454f4edcfcc118cfff1fa54c70938bb9715ecfe7d41282fa4f: Status 404 returned error can't find the container with id fdace70c726795454f4edcfcc118cfff1fa54c70938bb9715ecfe7d41282fa4f
Jul 29 00:17:07.747846 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:17:07.747826    2572 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 29 00:17:08.051525 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:17:08.051490    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e26439-7c574cb686-w85n4" event={"ID":"59064777-3621-46f7-9835-c4e1a604fbec","Type":"ContainerStarted","Data":"fdace70c726795454f4edcfcc118cfff1fa54c70938bb9715ecfe7d41282fa4f"}
Jul 29 00:17:09.057124 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:17:09.057086    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e26439-7c574cb686-w85n4" event={"ID":"59064777-3621-46f7-9835-c4e1a604fbec","Type":"ContainerStarted","Data":"fb9a8508e08564c2631cb19e54a73bcc505e69231c9a28f586e44ef5db6e139e"}
Jul 29 00:17:09.057499 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:17:09.057180    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e26439-7c574cb686-w85n4"
Jul 29 00:17:09.074541 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:17:09.074489    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e26439-7c574cb686-w85n4" podStartSLOduration=1.095853048 podStartE2EDuration="2.074469318s" totalImagesPullingTime="978.61627ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 00:17:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:17:09.074119393 +0000 UTC m=+3305.766793904" watchObservedRunningTime="2026-07-29 00:17:09.074469318 +0000 UTC m=+3305.767143828"
Jul 29 00:17:15.067353 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:17:15.067320    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e26439-7c574cb686-w85n4"
Jul 29 00:17:20.111790 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:17:20.111752    2572 generic.go:356] "Generic (PLEG): container finished" podID="82b6488a-9627-4ac3-b8fb-8aa5d958fab0" containerID="af5a241a7f0c1650fe1b8901dcc5d778e8cde48714b255606ebc36722a2a3a50" exitCode=0
Jul 29 00:17:20.112225 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:17:20.111822    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-e26439-openshift-default-77854bc9c4-76fqp" event={"ID":"82b6488a-9627-4ac3-b8fb-8aa5d958fab0","Type":"ContainerDied","Data":"af5a241a7f0c1650fe1b8901dcc5d778e8cde48714b255606ebc36722a2a3a50"}
Jul 29 00:17:20.112225 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:17:20.112179    2572 scope.go:117] "RemoveContainer" containerID="af5a241a7f0c1650fe1b8901dcc5d778e8cde48714b255606ebc36722a2a3a50"
Jul 29 00:17:20.113264 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:17:20.113236    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 29 00:17:20.113328 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:17:20.113312    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 29 00:17:20.113369 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:17:20.113352    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 29 00:17:21.035795 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:17:21.035758    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-e26439-openshift-default-77854bc9c4-76fqp"
Jul 29 00:17:21.119124 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:17:21.119091    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-e26439-openshift-default-77854bc9c4-76fqp" event={"ID":"82b6488a-9627-4ac3-b8fb-8aa5d958fab0","Type":"ContainerStarted","Data":"f1cc10aab60dd75b1ad7d30f655e2fe4916f741339e51e5ceab26cddae08dbb9"}
Jul 29 00:17:21.361278 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:17:21.361177    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-e26439-openshift-default-77854bc9c4-76fqp"
Jul 29 00:17:21.362360 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:17:21.362331    2572 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-e26439-openshift-default-77854bc9c4-76fqp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body=
Jul 29 00:17:21.362486 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:17:21.362385    2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-e26439-openshift-default-77854bc9c4-76fqp" podUID="82b6488a-9627-4ac3-b8fb-8aa5d958fab0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused"
Jul 29 00:17:22.361338 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:17:22.361292    2572 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-e26439-openshift-default-77854bc9c4-76fqp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body=
Jul 29 00:17:22.361771 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:17:22.361356    2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-e26439-openshift-default-77854bc9c4-76fqp" podUID="82b6488a-9627-4ac3-b8fb-8aa5d958fab0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused"
Jul 29 00:17:23.361044 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:17:23.361007    2572 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-e26439-openshift-default-77854bc9c4-76fqp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body=
Jul 29 00:17:23.361232 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:17:23.361074    2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-e26439-openshift-default-77854bc9c4-76fqp" podUID="82b6488a-9627-4ac3-b8fb-8aa5d958fab0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused"
Jul 29 00:17:24.361549 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:17:24.361509    2572 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-e26439-openshift-default-77854bc9c4-76fqp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body=
Jul 29 00:17:24.362003 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:17:24.361583    2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-e26439-openshift-default-77854bc9c4-76fqp" podUID="82b6488a-9627-4ac3-b8fb-8aa5d958fab0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused"
Jul 29 00:17:25.361070 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:17:25.360985    2572 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-e26439-openshift-default-77854bc9c4-76fqp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body=
Jul 29 00:17:25.361070 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:17:25.361040    2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-e26439-openshift-default-77854bc9c4-76fqp" podUID="82b6488a-9627-4ac3-b8fb-8aa5d958fab0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused"
Jul 29 00:17:26.364953 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:17:26.364922    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-e26439-openshift-default-77854bc9c4-76fqp"
Jul 29 00:17:26.365413 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:17:26.365312    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-e26439-openshift-default-77854bc9c4-76fqp"
Jul 29 00:17:26.366063 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:17:26.366040    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-e26439-openshift-default-77854bc9c4-76fqp"
Jul 29 00:17:37.773192 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:17:37.773144    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e26439-7c574cb686-w85n4"]
Jul 29 00:17:37.773753 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:17:37.773374    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e26439-7c574cb686-w85n4" podUID="59064777-3621-46f7-9835-c4e1a604fbec" containerName="maas-api" containerID="cri-o://fb9a8508e08564c2631cb19e54a73bcc505e69231c9a28f586e44ef5db6e139e" gracePeriod=30
Jul 29 00:17:38.037971 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:17:38.037893    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e26439-7c574cb686-w85n4"
Jul 29 00:17:38.197446 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:17:38.197406    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/59064777-3621-46f7-9835-c4e1a604fbec-openshift-service-ca\") pod \"59064777-3621-46f7-9835-c4e1a604fbec\" (UID: \"59064777-3621-46f7-9835-c4e1a604fbec\") "
Jul 29 00:17:38.197663 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:17:38.197463    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kwfn6\" (UniqueName: \"kubernetes.io/projected/59064777-3621-46f7-9835-c4e1a604fbec-kube-api-access-kwfn6\") pod \"59064777-3621-46f7-9835-c4e1a604fbec\" (UID: \"59064777-3621-46f7-9835-c4e1a604fbec\") "
Jul 29 00:17:38.197663 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:17:38.197524    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/59064777-3621-46f7-9835-c4e1a604fbec-maas-api-tls\") pod \"59064777-3621-46f7-9835-c4e1a604fbec\" (UID: \"59064777-3621-46f7-9835-c4e1a604fbec\") "
Jul 29 00:17:38.197846 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:17:38.197822    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/59064777-3621-46f7-9835-c4e1a604fbec-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "59064777-3621-46f7-9835-c4e1a604fbec" (UID: "59064777-3621-46f7-9835-c4e1a604fbec"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 00:17:38.199654 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:17:38.199595    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/59064777-3621-46f7-9835-c4e1a604fbec-kube-api-access-kwfn6" (OuterVolumeSpecName: "kube-api-access-kwfn6") pod "59064777-3621-46f7-9835-c4e1a604fbec" (UID: "59064777-3621-46f7-9835-c4e1a604fbec"). InnerVolumeSpecName "kube-api-access-kwfn6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 00:17:38.199654 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:17:38.199609    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/59064777-3621-46f7-9835-c4e1a604fbec-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "59064777-3621-46f7-9835-c4e1a604fbec" (UID: "59064777-3621-46f7-9835-c4e1a604fbec"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 00:17:38.208328 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:17:38.208298    2572 generic.go:356] "Generic (PLEG): container finished" podID="59064777-3621-46f7-9835-c4e1a604fbec" containerID="fb9a8508e08564c2631cb19e54a73bcc505e69231c9a28f586e44ef5db6e139e" exitCode=0
Jul 29 00:17:38.208443 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:17:38.208354    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e26439-7c574cb686-w85n4"
Jul 29 00:17:38.208443 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:17:38.208377    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e26439-7c574cb686-w85n4" event={"ID":"59064777-3621-46f7-9835-c4e1a604fbec","Type":"ContainerDied","Data":"fb9a8508e08564c2631cb19e54a73bcc505e69231c9a28f586e44ef5db6e139e"}
Jul 29 00:17:38.208443 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:17:38.208439    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e26439-7c574cb686-w85n4" event={"ID":"59064777-3621-46f7-9835-c4e1a604fbec","Type":"ContainerDied","Data":"fdace70c726795454f4edcfcc118cfff1fa54c70938bb9715ecfe7d41282fa4f"}
Jul 29 00:17:38.208553 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:17:38.208468    2572 scope.go:117] "RemoveContainer" containerID="fb9a8508e08564c2631cb19e54a73bcc505e69231c9a28f586e44ef5db6e139e"
Jul 29 00:17:38.221657 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:17:38.221636    2572 scope.go:117] "RemoveContainer" containerID="fb9a8508e08564c2631cb19e54a73bcc505e69231c9a28f586e44ef5db6e139e"
Jul 29 00:17:38.221978 ip-10-0-131-112 kubenswrapper[2572]: E0729 00:17:38.221957    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fb9a8508e08564c2631cb19e54a73bcc505e69231c9a28f586e44ef5db6e139e\": container with ID starting with fb9a8508e08564c2631cb19e54a73bcc505e69231c9a28f586e44ef5db6e139e not found: ID does not exist" containerID="fb9a8508e08564c2631cb19e54a73bcc505e69231c9a28f586e44ef5db6e139e"
Jul 29 00:17:38.222072 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:17:38.221987    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fb9a8508e08564c2631cb19e54a73bcc505e69231c9a28f586e44ef5db6e139e"} err="failed to get container status \"fb9a8508e08564c2631cb19e54a73bcc505e69231c9a28f586e44ef5db6e139e\": rpc error: code = NotFound desc = could not find container \"fb9a8508e08564c2631cb19e54a73bcc505e69231c9a28f586e44ef5db6e139e\": container with ID starting with fb9a8508e08564c2631cb19e54a73bcc505e69231c9a28f586e44ef5db6e139e not found: ID does not exist"
Jul 29 00:17:38.232381 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:17:38.232355    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e26439-7c574cb686-w85n4"]
Jul 29 00:17:38.236767 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:17:38.236744    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e26439-7c574cb686-w85n4"]
Jul 29 00:17:38.299032 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:17:38.298941    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/59064777-3621-46f7-9835-c4e1a604fbec-maas-api-tls\") on node \"ip-10-0-131-112.ec2.internal\" DevicePath \"\""
Jul 29 00:17:38.299032 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:17:38.298971    2572 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/59064777-3621-46f7-9835-c4e1a604fbec-openshift-service-ca\") on node \"ip-10-0-131-112.ec2.internal\" DevicePath \"\""
Jul 29 00:17:38.299032 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:17:38.298990    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kwfn6\" (UniqueName: \"kubernetes.io/projected/59064777-3621-46f7-9835-c4e1a604fbec-kube-api-access-kwfn6\") on node \"ip-10-0-131-112.ec2.internal\" DevicePath \"\""
Jul 29 00:17:39.865638 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:17:39.865593    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="59064777-3621-46f7-9835-c4e1a604fbec" path="/var/lib/kubelet/pods/59064777-3621-46f7-9835-c4e1a604fbec/volumes"
Jul 29 00:17:44.319376 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:17:44.319340    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-e26439-openshift-default-77854bc9c4-76fqp"]
Jul 29 00:17:44.319779 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:17:44.319564    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-cleanup-e26439-openshift-default-77854bc9c4-76fqp" podUID="82b6488a-9627-4ac3-b8fb-8aa5d958fab0" containerName="istio-proxy" containerID="cri-o://f1cc10aab60dd75b1ad7d30f655e2fe4916f741339e51e5ceab26cddae08dbb9" gracePeriod=30
Jul 29 00:17:49.576666 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:17:49.576641    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-e26439-openshift-default-77854bc9c4-76fqp"
Jul 29 00:17:49.592113 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:17:49.592060    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/82b6488a-9627-4ac3-b8fb-8aa5d958fab0-istio-data\") pod \"82b6488a-9627-4ac3-b8fb-8aa5d958fab0\" (UID: \"82b6488a-9627-4ac3-b8fb-8aa5d958fab0\") "
Jul 29 00:17:49.592113 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:17:49.592103    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/82b6488a-9627-4ac3-b8fb-8aa5d958fab0-istio-envoy\") pod \"82b6488a-9627-4ac3-b8fb-8aa5d958fab0\" (UID: \"82b6488a-9627-4ac3-b8fb-8aa5d958fab0\") "
Jul 29 00:17:49.592244 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:17:49.592149    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/82b6488a-9627-4ac3-b8fb-8aa5d958fab0-workload-socket\") pod \"82b6488a-9627-4ac3-b8fb-8aa5d958fab0\" (UID: \"82b6488a-9627-4ac3-b8fb-8aa5d958fab0\") "
Jul 29 00:17:49.592244 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:17:49.592167    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/82b6488a-9627-4ac3-b8fb-8aa5d958fab0-istiod-ca-cert\") pod \"82b6488a-9627-4ac3-b8fb-8aa5d958fab0\" (UID: \"82b6488a-9627-4ac3-b8fb-8aa5d958fab0\") "
Jul 29 00:17:49.592244 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:17:49.592185    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/82b6488a-9627-4ac3-b8fb-8aa5d958fab0-credential-socket\") pod \"82b6488a-9627-4ac3-b8fb-8aa5d958fab0\" (UID: \"82b6488a-9627-4ac3-b8fb-8aa5d958fab0\") "
Jul 29 00:17:49.592244 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:17:49.592241    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bkcpk\" (UniqueName: \"kubernetes.io/projected/82b6488a-9627-4ac3-b8fb-8aa5d958fab0-kube-api-access-bkcpk\") pod \"82b6488a-9627-4ac3-b8fb-8aa5d958fab0\" (UID: \"82b6488a-9627-4ac3-b8fb-8aa5d958fab0\") "
Jul 29 00:17:49.592440 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:17:49.592280    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/82b6488a-9627-4ac3-b8fb-8aa5d958fab0-istio-token\") pod \"82b6488a-9627-4ac3-b8fb-8aa5d958fab0\" (UID: \"82b6488a-9627-4ac3-b8fb-8aa5d958fab0\") "
Jul 29 00:17:49.592440 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:17:49.592328    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/82b6488a-9627-4ac3-b8fb-8aa5d958fab0-workload-certs\") pod \"82b6488a-9627-4ac3-b8fb-8aa5d958fab0\" (UID: \"82b6488a-9627-4ac3-b8fb-8aa5d958fab0\") "
Jul 29 00:17:49.592440 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:17:49.592355    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/82b6488a-9627-4ac3-b8fb-8aa5d958fab0-istio-podinfo\") pod \"82b6488a-9627-4ac3-b8fb-8aa5d958fab0\" (UID: \"82b6488a-9627-4ac3-b8fb-8aa5d958fab0\") "
Jul 29 00:17:49.592601 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:17:49.592535    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/82b6488a-9627-4ac3-b8fb-8aa5d958fab0-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "82b6488a-9627-4ac3-b8fb-8aa5d958fab0" (UID: "82b6488a-9627-4ac3-b8fb-8aa5d958fab0"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 00:17:49.592682 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:17:49.592654    2572 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/82b6488a-9627-4ac3-b8fb-8aa5d958fab0-workload-socket\") on node \"ip-10-0-131-112.ec2.internal\" DevicePath \"\""
Jul 29 00:17:49.592742 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:17:49.592689    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/82b6488a-9627-4ac3-b8fb-8aa5d958fab0-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "82b6488a-9627-4ac3-b8fb-8aa5d958fab0" (UID: "82b6488a-9627-4ac3-b8fb-8aa5d958fab0"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 00:17:49.592869 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:17:49.592844    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/82b6488a-9627-4ac3-b8fb-8aa5d958fab0-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "82b6488a-9627-4ac3-b8fb-8aa5d958fab0" (UID: "82b6488a-9627-4ac3-b8fb-8aa5d958fab0"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 00:17:49.592971 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:17:49.592878    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/82b6488a-9627-4ac3-b8fb-8aa5d958fab0-istio-data" (OuterVolumeSpecName: "istio-data") pod "82b6488a-9627-4ac3-b8fb-8aa5d958fab0" (UID: "82b6488a-9627-4ac3-b8fb-8aa5d958fab0"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 00:17:49.593030 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:17:49.592974    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/82b6488a-9627-4ac3-b8fb-8aa5d958fab0-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "82b6488a-9627-4ac3-b8fb-8aa5d958fab0" (UID: "82b6488a-9627-4ac3-b8fb-8aa5d958fab0"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 00:17:49.594548 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:17:49.594524    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/82b6488a-9627-4ac3-b8fb-8aa5d958fab0-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "82b6488a-9627-4ac3-b8fb-8aa5d958fab0" (UID: "82b6488a-9627-4ac3-b8fb-8aa5d958fab0"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 00:17:49.594665 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:17:49.594526    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/82b6488a-9627-4ac3-b8fb-8aa5d958fab0-kube-api-access-bkcpk" (OuterVolumeSpecName: "kube-api-access-bkcpk") pod "82b6488a-9627-4ac3-b8fb-8aa5d958fab0" (UID: "82b6488a-9627-4ac3-b8fb-8aa5d958fab0"). InnerVolumeSpecName "kube-api-access-bkcpk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 00:17:49.594665 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:17:49.594656    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/82b6488a-9627-4ac3-b8fb-8aa5d958fab0-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "82b6488a-9627-4ac3-b8fb-8aa5d958fab0" (UID: "82b6488a-9627-4ac3-b8fb-8aa5d958fab0"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 29 00:17:49.594786 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:17:49.594740    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/82b6488a-9627-4ac3-b8fb-8aa5d958fab0-istio-token" (OuterVolumeSpecName: "istio-token") pod "82b6488a-9627-4ac3-b8fb-8aa5d958fab0" (UID: "82b6488a-9627-4ac3-b8fb-8aa5d958fab0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 00:17:49.693873 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:17:49.693841    2572 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/82b6488a-9627-4ac3-b8fb-8aa5d958fab0-istio-token\") on node \"ip-10-0-131-112.ec2.internal\" DevicePath \"\""
Jul 29 00:17:49.693873 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:17:49.693870    2572 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/82b6488a-9627-4ac3-b8fb-8aa5d958fab0-workload-certs\") on node \"ip-10-0-131-112.ec2.internal\" DevicePath \"\""
Jul 29 00:17:49.693873 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:17:49.693879    2572 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/82b6488a-9627-4ac3-b8fb-8aa5d958fab0-istio-podinfo\") on node \"ip-10-0-131-112.ec2.internal\" DevicePath \"\""
Jul 29 00:17:49.694106 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:17:49.693887    2572 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/82b6488a-9627-4ac3-b8fb-8aa5d958fab0-istio-data\") on node \"ip-10-0-131-112.ec2.internal\" DevicePath \"\""
Jul 29 00:17:49.694106 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:17:49.693896    2572 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/82b6488a-9627-4ac3-b8fb-8aa5d958fab0-istio-envoy\") on node \"ip-10-0-131-112.ec2.internal\" DevicePath \"\""
Jul 29 00:17:49.694106 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:17:49.693902    2572 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/82b6488a-9627-4ac3-b8fb-8aa5d958fab0-istiod-ca-cert\") on node \"ip-10-0-131-112.ec2.internal\" DevicePath \"\""
Jul 29 00:17:49.694106 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:17:49.693910    2572 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/82b6488a-9627-4ac3-b8fb-8aa5d958fab0-credential-socket\") on node \"ip-10-0-131-112.ec2.internal\" DevicePath \"\""
Jul 29 00:17:49.694106 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:17:49.693931    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bkcpk\" (UniqueName: \"kubernetes.io/projected/82b6488a-9627-4ac3-b8fb-8aa5d958fab0-kube-api-access-bkcpk\") on node \"ip-10-0-131-112.ec2.internal\" DevicePath \"\""
Jul 29 00:17:50.264995 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:17:50.264964    2572 generic.go:356] "Generic (PLEG): container finished" podID="82b6488a-9627-4ac3-b8fb-8aa5d958fab0" containerID="f1cc10aab60dd75b1ad7d30f655e2fe4916f741339e51e5ceab26cddae08dbb9" exitCode=0
Jul 29 00:17:50.265174 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:17:50.265010    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-e26439-openshift-default-77854bc9c4-76fqp"
Jul 29 00:17:50.265174 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:17:50.265052    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-e26439-openshift-default-77854bc9c4-76fqp" event={"ID":"82b6488a-9627-4ac3-b8fb-8aa5d958fab0","Type":"ContainerDied","Data":"f1cc10aab60dd75b1ad7d30f655e2fe4916f741339e51e5ceab26cddae08dbb9"}
Jul 29 00:17:50.265174 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:17:50.265106    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-e26439-openshift-default-77854bc9c4-76fqp" event={"ID":"82b6488a-9627-4ac3-b8fb-8aa5d958fab0","Type":"ContainerDied","Data":"2aa8fe1abe13cc0ccd1c6af5bddec1ae23118ffa4ab2aaf450d6fda104a7277c"}
Jul 29 00:17:50.265174 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:17:50.265128    2572 scope.go:117] "RemoveContainer" containerID="f1cc10aab60dd75b1ad7d30f655e2fe4916f741339e51e5ceab26cddae08dbb9"
Jul 29 00:17:50.273857 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:17:50.273840    2572 scope.go:117] "RemoveContainer" containerID="af5a241a7f0c1650fe1b8901dcc5d778e8cde48714b255606ebc36722a2a3a50"
Jul 29 00:17:50.281555 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:17:50.281535    2572 scope.go:117] "RemoveContainer" containerID="f1cc10aab60dd75b1ad7d30f655e2fe4916f741339e51e5ceab26cddae08dbb9"
Jul 29 00:17:50.281834 ip-10-0-131-112 kubenswrapper[2572]: E0729 00:17:50.281816    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f1cc10aab60dd75b1ad7d30f655e2fe4916f741339e51e5ceab26cddae08dbb9\": container with ID starting with f1cc10aab60dd75b1ad7d30f655e2fe4916f741339e51e5ceab26cddae08dbb9 not found: ID does not exist" containerID="f1cc10aab60dd75b1ad7d30f655e2fe4916f741339e51e5ceab26cddae08dbb9"
Jul 29 00:17:50.281880 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:17:50.281842    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f1cc10aab60dd75b1ad7d30f655e2fe4916f741339e51e5ceab26cddae08dbb9"} err="failed to get container status \"f1cc10aab60dd75b1ad7d30f655e2fe4916f741339e51e5ceab26cddae08dbb9\": rpc error: code = NotFound desc = could not find container \"f1cc10aab60dd75b1ad7d30f655e2fe4916f741339e51e5ceab26cddae08dbb9\": container with ID starting with f1cc10aab60dd75b1ad7d30f655e2fe4916f741339e51e5ceab26cddae08dbb9 not found: ID does not exist"
Jul 29 00:17:50.281880 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:17:50.281858    2572 scope.go:117] "RemoveContainer" containerID="af5a241a7f0c1650fe1b8901dcc5d778e8cde48714b255606ebc36722a2a3a50"
Jul 29 00:17:50.282054 ip-10-0-131-112 kubenswrapper[2572]: E0729 00:17:50.282037    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"af5a241a7f0c1650fe1b8901dcc5d778e8cde48714b255606ebc36722a2a3a50\": container with ID starting with af5a241a7f0c1650fe1b8901dcc5d778e8cde48714b255606ebc36722a2a3a50 not found: ID does not exist" containerID="af5a241a7f0c1650fe1b8901dcc5d778e8cde48714b255606ebc36722a2a3a50"
Jul 29 00:17:50.282091 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:17:50.282060    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"af5a241a7f0c1650fe1b8901dcc5d778e8cde48714b255606ebc36722a2a3a50"} err="failed to get container status \"af5a241a7f0c1650fe1b8901dcc5d778e8cde48714b255606ebc36722a2a3a50\": rpc error: code = NotFound desc = could not find container \"af5a241a7f0c1650fe1b8901dcc5d778e8cde48714b255606ebc36722a2a3a50\": container with ID starting with af5a241a7f0c1650fe1b8901dcc5d778e8cde48714b255606ebc36722a2a3a50 not found: ID does not exist"
Jul 29 00:17:50.284871 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:17:50.284850    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-e26439-openshift-default-77854bc9c4-76fqp"]
Jul 29 00:17:50.288665 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:17:50.288645    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-e26439-openshift-default-77854bc9c4-76fqp"]
Jul 29 00:17:51.865138 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:17:51.865104    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="82b6488a-9627-4ac3-b8fb-8aa5d958fab0" path="/var/lib/kubelet/pods/82b6488a-9627-4ac3-b8fb-8aa5d958fab0/volumes"
Jul 29 00:18:52.407806 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:18:52.407768    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-f318ab-openshift-default-f84669684-pxthv"]
Jul 29 00:18:52.408363 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:18:52.408013    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-a-f318ab-openshift-default-f84669684-pxthv" podUID="2530b4c9-9988-48e5-89ed-f61207a086f2" containerName="istio-proxy" containerID="cri-o://b133463e90c8c6e36e2ba7c714decbc97523af84f1915dca7d6dcb05cf6da44a" gracePeriod=30
Jul 29 00:18:52.560419 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:18:52.560376    2572 patch_prober.go:28] interesting pod/e2e-ipp-a-f318ab-openshift-default-f84669684-pxthv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 29 00:18:52.560583 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:18:52.560449    2572 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-a-f318ab-openshift-default-f84669684-pxthv" podUID="2530b4c9-9988-48e5-89ed-f61207a086f2" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 29 00:18:57.567856 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:18:57.567823    2572 generic.go:356] "Generic (PLEG): container finished" podID="2530b4c9-9988-48e5-89ed-f61207a086f2" containerID="b133463e90c8c6e36e2ba7c714decbc97523af84f1915dca7d6dcb05cf6da44a" exitCode=0
Jul 29 00:18:57.568239 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:18:57.567867    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-f318ab-openshift-default-f84669684-pxthv" event={"ID":"2530b4c9-9988-48e5-89ed-f61207a086f2","Type":"ContainerDied","Data":"b133463e90c8c6e36e2ba7c714decbc97523af84f1915dca7d6dcb05cf6da44a"}
Jul 29 00:18:57.568239 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:18:57.567940    2572 scope.go:117] "RemoveContainer" containerID="8b4f449ec422a059e31fed045b93154d9238d014d70180666eba14905531df55"
Jul 29 00:18:57.654780 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:18:57.654752    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-f318ab-openshift-default-f84669684-pxthv"
Jul 29 00:18:57.800011 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:18:57.799975    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2530b4c9-9988-48e5-89ed-f61207a086f2-workload-certs\") pod \"2530b4c9-9988-48e5-89ed-f61207a086f2\" (UID: \"2530b4c9-9988-48e5-89ed-f61207a086f2\") "
Jul 29 00:18:57.800206 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:18:57.800023    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2530b4c9-9988-48e5-89ed-f61207a086f2-istiod-ca-cert\") pod \"2530b4c9-9988-48e5-89ed-f61207a086f2\" (UID: \"2530b4c9-9988-48e5-89ed-f61207a086f2\") "
Jul 29 00:18:57.800206 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:18:57.800046    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2530b4c9-9988-48e5-89ed-f61207a086f2-workload-socket\") pod \"2530b4c9-9988-48e5-89ed-f61207a086f2\" (UID: \"2530b4c9-9988-48e5-89ed-f61207a086f2\") "
Jul 29 00:18:57.800326 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:18:57.800250    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2530b4c9-9988-48e5-89ed-f61207a086f2-istio-podinfo\") pod \"2530b4c9-9988-48e5-89ed-f61207a086f2\" (UID: \"2530b4c9-9988-48e5-89ed-f61207a086f2\") "
Jul 29 00:18:57.800326 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:18:57.800297    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2530b4c9-9988-48e5-89ed-f61207a086f2-istio-token\") pod \"2530b4c9-9988-48e5-89ed-f61207a086f2\" (UID: \"2530b4c9-9988-48e5-89ed-f61207a086f2\") "
Jul 29 00:18:57.800326 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:18:57.800272    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2530b4c9-9988-48e5-89ed-f61207a086f2-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "2530b4c9-9988-48e5-89ed-f61207a086f2" (UID: "2530b4c9-9988-48e5-89ed-f61207a086f2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 00:18:57.800489 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:18:57.800336    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fcrrw\" (UniqueName: \"kubernetes.io/projected/2530b4c9-9988-48e5-89ed-f61207a086f2-kube-api-access-fcrrw\") pod \"2530b4c9-9988-48e5-89ed-f61207a086f2\" (UID: \"2530b4c9-9988-48e5-89ed-f61207a086f2\") "
Jul 29 00:18:57.800489 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:18:57.800336    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2530b4c9-9988-48e5-89ed-f61207a086f2-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "2530b4c9-9988-48e5-89ed-f61207a086f2" (UID: "2530b4c9-9988-48e5-89ed-f61207a086f2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 00:18:57.800489 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:18:57.800388    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2530b4c9-9988-48e5-89ed-f61207a086f2-istio-data\") pod \"2530b4c9-9988-48e5-89ed-f61207a086f2\" (UID: \"2530b4c9-9988-48e5-89ed-f61207a086f2\") "
Jul 29 00:18:57.800489 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:18:57.800440    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2530b4c9-9988-48e5-89ed-f61207a086f2-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "2530b4c9-9988-48e5-89ed-f61207a086f2" (UID: "2530b4c9-9988-48e5-89ed-f61207a086f2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 00:18:57.800489 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:18:57.800456    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2530b4c9-9988-48e5-89ed-f61207a086f2-istio-envoy\") pod \"2530b4c9-9988-48e5-89ed-f61207a086f2\" (UID: \"2530b4c9-9988-48e5-89ed-f61207a086f2\") "
Jul 29 00:18:57.800852 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:18:57.800491    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2530b4c9-9988-48e5-89ed-f61207a086f2-credential-socket\") pod \"2530b4c9-9988-48e5-89ed-f61207a086f2\" (UID: \"2530b4c9-9988-48e5-89ed-f61207a086f2\") "
Jul 29 00:18:57.800852 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:18:57.800800    2572 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2530b4c9-9988-48e5-89ed-f61207a086f2-workload-certs\") on node \"ip-10-0-131-112.ec2.internal\" DevicePath \"\""
Jul 29 00:18:57.800852 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:18:57.800807    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2530b4c9-9988-48e5-89ed-f61207a086f2-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "2530b4c9-9988-48e5-89ed-f61207a086f2" (UID: "2530b4c9-9988-48e5-89ed-f61207a086f2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 00:18:57.800852 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:18:57.800820    2572 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2530b4c9-9988-48e5-89ed-f61207a086f2-istiod-ca-cert\") on node \"ip-10-0-131-112.ec2.internal\" DevicePath \"\""
Jul 29 00:18:57.801060 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:18:57.800860    2572 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2530b4c9-9988-48e5-89ed-f61207a086f2-workload-socket\") on node \"ip-10-0-131-112.ec2.internal\" DevicePath \"\""
Jul 29 00:18:57.801290 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:18:57.801264    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2530b4c9-9988-48e5-89ed-f61207a086f2-istio-data" (OuterVolumeSpecName: "istio-data") pod "2530b4c9-9988-48e5-89ed-f61207a086f2" (UID: "2530b4c9-9988-48e5-89ed-f61207a086f2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 00:18:57.802612 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:18:57.802583    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2530b4c9-9988-48e5-89ed-f61207a086f2-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "2530b4c9-9988-48e5-89ed-f61207a086f2" (UID: "2530b4c9-9988-48e5-89ed-f61207a086f2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 29 00:18:57.802738 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:18:57.802605    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2530b4c9-9988-48e5-89ed-f61207a086f2-istio-token" (OuterVolumeSpecName: "istio-token") pod "2530b4c9-9988-48e5-89ed-f61207a086f2" (UID: "2530b4c9-9988-48e5-89ed-f61207a086f2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 00:18:57.802738 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:18:57.802589    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2530b4c9-9988-48e5-89ed-f61207a086f2-kube-api-access-fcrrw" (OuterVolumeSpecName: "kube-api-access-fcrrw") pod "2530b4c9-9988-48e5-89ed-f61207a086f2" (UID: "2530b4c9-9988-48e5-89ed-f61207a086f2"). InnerVolumeSpecName "kube-api-access-fcrrw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 00:18:57.802738 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:18:57.802669    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2530b4c9-9988-48e5-89ed-f61207a086f2-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "2530b4c9-9988-48e5-89ed-f61207a086f2" (UID: "2530b4c9-9988-48e5-89ed-f61207a086f2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 00:18:57.902069 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:18:57.902038    2572 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2530b4c9-9988-48e5-89ed-f61207a086f2-istio-token\") on node \"ip-10-0-131-112.ec2.internal\" DevicePath \"\""
Jul 29 00:18:57.902069 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:18:57.902066    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fcrrw\" (UniqueName: \"kubernetes.io/projected/2530b4c9-9988-48e5-89ed-f61207a086f2-kube-api-access-fcrrw\") on node \"ip-10-0-131-112.ec2.internal\" DevicePath \"\""
Jul 29 00:18:57.902229 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:18:57.902077    2572 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2530b4c9-9988-48e5-89ed-f61207a086f2-istio-data\") on node \"ip-10-0-131-112.ec2.internal\" DevicePath \"\""
Jul 29 00:18:57.902229 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:18:57.902085    2572 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2530b4c9-9988-48e5-89ed-f61207a086f2-istio-envoy\") on node \"ip-10-0-131-112.ec2.internal\" DevicePath \"\""
Jul 29 00:18:57.902229 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:18:57.902093    2572 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2530b4c9-9988-48e5-89ed-f61207a086f2-credential-socket\") on node \"ip-10-0-131-112.ec2.internal\" DevicePath \"\""
Jul 29 00:18:57.902229 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:18:57.902101    2572 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2530b4c9-9988-48e5-89ed-f61207a086f2-istio-podinfo\") on node \"ip-10-0-131-112.ec2.internal\" DevicePath \"\""
Jul 29 00:18:58.574787 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:18:58.574757    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-f318ab-openshift-default-f84669684-pxthv"
Jul 29 00:18:58.575229 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:18:58.574796    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-f318ab-openshift-default-f84669684-pxthv" event={"ID":"2530b4c9-9988-48e5-89ed-f61207a086f2","Type":"ContainerDied","Data":"25eddee4362c2bb124835ae0c5a0961129298aa63691ed35fa79e6cd5bd24766"}
Jul 29 00:18:58.575229 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:18:58.574853    2572 scope.go:117] "RemoveContainer" containerID="b133463e90c8c6e36e2ba7c714decbc97523af84f1915dca7d6dcb05cf6da44a"
Jul 29 00:18:58.591799 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:18:58.591774    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-f318ab-openshift-default-f84669684-pxthv"]
Jul 29 00:18:58.595564 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:18:58.595542    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-a-f318ab-openshift-default-f84669684-pxthv"]
Jul 29 00:18:59.865226 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:18:59.865189    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2530b4c9-9988-48e5-89ed-f61207a086f2" path="/var/lib/kubelet/pods/2530b4c9-9988-48e5-89ed-f61207a086f2/volumes"
Jul 29 00:19:10.315236 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:19:10.315200    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-daedc5-mf9lb"]
Jul 29 00:19:10.315597 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:19:10.315503    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="82b6488a-9627-4ac3-b8fb-8aa5d958fab0" containerName="istio-proxy"
Jul 29 00:19:10.315597 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:19:10.315514    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="82b6488a-9627-4ac3-b8fb-8aa5d958fab0" containerName="istio-proxy"
Jul 29 00:19:10.315597 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:19:10.315528    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2530b4c9-9988-48e5-89ed-f61207a086f2" containerName="istio-proxy"
Jul 29 00:19:10.315597 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:19:10.315534    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="2530b4c9-9988-48e5-89ed-f61207a086f2" containerName="istio-proxy"
Jul 29 00:19:10.315597 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:19:10.315541    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="59064777-3621-46f7-9835-c4e1a604fbec" containerName="maas-api"
Jul 29 00:19:10.315597 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:19:10.315547    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="59064777-3621-46f7-9835-c4e1a604fbec" containerName="maas-api"
Jul 29 00:19:10.315597 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:19:10.315555    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2530b4c9-9988-48e5-89ed-f61207a086f2" containerName="istio-proxy"
Jul 29 00:19:10.315597 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:19:10.315560    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="2530b4c9-9988-48e5-89ed-f61207a086f2" containerName="istio-proxy"
Jul 29 00:19:10.315597 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:19:10.315566    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="82b6488a-9627-4ac3-b8fb-8aa5d958fab0" containerName="istio-proxy"
Jul 29 00:19:10.315597 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:19:10.315571    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="82b6488a-9627-4ac3-b8fb-8aa5d958fab0" containerName="istio-proxy"
Jul 29 00:19:10.315961 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:19:10.315642    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="2530b4c9-9988-48e5-89ed-f61207a086f2" containerName="istio-proxy"
Jul 29 00:19:10.315961 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:19:10.315651    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="82b6488a-9627-4ac3-b8fb-8aa5d958fab0" containerName="istio-proxy"
Jul 29 00:19:10.315961 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:19:10.315658    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="82b6488a-9627-4ac3-b8fb-8aa5d958fab0" containerName="istio-proxy"
Jul 29 00:19:10.315961 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:19:10.315665    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="59064777-3621-46f7-9835-c4e1a604fbec" containerName="maas-api"
Jul 29 00:19:10.319985 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:19:10.319966    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-daedc5-mf9lb"
Jul 29 00:19:10.322829 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:19:10.322797    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-rszgs\""
Jul 29 00:19:10.325492 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:19:10.325463    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-daedc5-mf9lb"]
Jul 29 00:19:10.512289 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:19:10.512251    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/40943638-4a91-4afd-b810-eca062eb7d05-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-daedc5-mf9lb\" (UID: \"40943638-4a91-4afd-b810-eca062eb7d05\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-daedc5-mf9lb"
Jul 29 00:19:10.613491 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:19:10.613407    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/40943638-4a91-4afd-b810-eca062eb7d05-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-daedc5-mf9lb\" (UID: \"40943638-4a91-4afd-b810-eca062eb7d05\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-daedc5-mf9lb"
Jul 29 00:19:10.614096 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:19:10.614077    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/40943638-4a91-4afd-b810-eca062eb7d05-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-daedc5-mf9lb\" (UID: \"40943638-4a91-4afd-b810-eca062eb7d05\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-daedc5-mf9lb"
Jul 29 00:19:10.632320 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:19:10.632299    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-daedc5-mf9lb"
Jul 29 00:19:10.765164 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:19:10.765138    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-daedc5-mf9lb"]
Jul 29 00:19:10.767774 ip-10-0-131-112 kubenswrapper[2572]: W0729 00:19:10.767744    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40943638_4a91_4afd_b810_eca062eb7d05.slice/crio-90bfe7fd88faae3475ae890b58c3a83ea14bc71b65d46c13a6fc405df0e5523d WatchSource:0}: Error finding container 90bfe7fd88faae3475ae890b58c3a83ea14bc71b65d46c13a6fc405df0e5523d: Status 404 returned error can't find the container with id 90bfe7fd88faae3475ae890b58c3a83ea14bc71b65d46c13a6fc405df0e5523d
Jul 29 00:19:11.634305 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:19:11.634265    2572 generic.go:356] "Generic (PLEG): container finished" podID="40943638-4a91-4afd-b810-eca062eb7d05" containerID="f04b8e501cbdccc3e1188e50ea564247c83b41d25cfa2569b2529de63d326ff8" exitCode=0
Jul 29 00:19:11.634702 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:19:11.634351    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-daedc5-mf9lb" event={"ID":"40943638-4a91-4afd-b810-eca062eb7d05","Type":"ContainerDied","Data":"f04b8e501cbdccc3e1188e50ea564247c83b41d25cfa2569b2529de63d326ff8"}
Jul 29 00:19:11.634702 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:19:11.634406    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-daedc5-mf9lb" event={"ID":"40943638-4a91-4afd-b810-eca062eb7d05","Type":"ContainerStarted","Data":"90bfe7fd88faae3475ae890b58c3a83ea14bc71b65d46c13a6fc405df0e5523d"}
Jul 29 00:19:12.763792 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:19:12.763771    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-daedc5-mf9lb"
Jul 29 00:19:12.835661 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:19:12.835619    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/40943638-4a91-4afd-b810-eca062eb7d05-maas-api-service-ca\") pod \"40943638-4a91-4afd-b810-eca062eb7d05\" (UID: \"40943638-4a91-4afd-b810-eca062eb7d05\") "
Jul 29 00:19:12.835946 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:19:12.835926    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/40943638-4a91-4afd-b810-eca062eb7d05-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "40943638-4a91-4afd-b810-eca062eb7d05" (UID: "40943638-4a91-4afd-b810-eca062eb7d05"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 00:19:12.936258 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:19:12.936170    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/40943638-4a91-4afd-b810-eca062eb7d05-maas-api-service-ca\") on node \"ip-10-0-131-112.ec2.internal\" DevicePath \"\""
Jul 29 00:19:13.643519 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:19:13.643490    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-daedc5-mf9lb"
Jul 29 00:19:13.643741 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:19:13.643486    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-daedc5-mf9lb" event={"ID":"40943638-4a91-4afd-b810-eca062eb7d05","Type":"ContainerDied","Data":"90bfe7fd88faae3475ae890b58c3a83ea14bc71b65d46c13a6fc405df0e5523d"}
Jul 29 00:19:13.643741 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:19:13.643619    2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="90bfe7fd88faae3475ae890b58c3a83ea14bc71b65d46c13a6fc405df0e5523d"
Jul 29 00:19:20.824189 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:19:20.824151    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-daedc5-649c5c6ff8-vvd4h"]
Jul 29 00:19:20.824686 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:19:20.824431    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-daedc5-649c5c6ff8-vvd4h" podUID="26c0d3a1-95f7-45eb-bc66-b81b60b4260d" containerName="maas-api" containerID="cri-o://bc9a3d0f81f6a06264f17e1f316e6b5acb16f8ba2263aeafa3a7996016f72cb6" gracePeriod=30
Jul 29 00:19:21.044727 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:19:21.044691    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-daedc5-mf9lb"]
Jul 29 00:19:21.049257 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:19:21.049230    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-daedc5-mf9lb"]
Jul 29 00:19:21.079926 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:19:21.079867    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-daedc5-649c5c6ff8-vvd4h"
Jul 29 00:19:21.094730 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:19:21.094708    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/26c0d3a1-95f7-45eb-bc66-b81b60b4260d-maas-api-tls\") pod \"26c0d3a1-95f7-45eb-bc66-b81b60b4260d\" (UID: \"26c0d3a1-95f7-45eb-bc66-b81b60b4260d\") "
Jul 29 00:19:21.094851 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:19:21.094753    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j5m8w\" (UniqueName: \"kubernetes.io/projected/26c0d3a1-95f7-45eb-bc66-b81b60b4260d-kube-api-access-j5m8w\") pod \"26c0d3a1-95f7-45eb-bc66-b81b60b4260d\" (UID: \"26c0d3a1-95f7-45eb-bc66-b81b60b4260d\") "
Jul 29 00:19:21.096589 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:19:21.096559    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/26c0d3a1-95f7-45eb-bc66-b81b60b4260d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "26c0d3a1-95f7-45eb-bc66-b81b60b4260d" (UID: "26c0d3a1-95f7-45eb-bc66-b81b60b4260d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 00:19:21.096789 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:19:21.096770    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/26c0d3a1-95f7-45eb-bc66-b81b60b4260d-kube-api-access-j5m8w" (OuterVolumeSpecName: "kube-api-access-j5m8w") pod "26c0d3a1-95f7-45eb-bc66-b81b60b4260d" (UID: "26c0d3a1-95f7-45eb-bc66-b81b60b4260d"). InnerVolumeSpecName "kube-api-access-j5m8w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 00:19:21.195178 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:19:21.195148    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/26c0d3a1-95f7-45eb-bc66-b81b60b4260d-openshift-service-ca\") pod \"26c0d3a1-95f7-45eb-bc66-b81b60b4260d\" (UID: \"26c0d3a1-95f7-45eb-bc66-b81b60b4260d\") "
Jul 29 00:19:21.195335 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:19:21.195324    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/26c0d3a1-95f7-45eb-bc66-b81b60b4260d-maas-api-tls\") on node \"ip-10-0-131-112.ec2.internal\" DevicePath \"\""
Jul 29 00:19:21.195391 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:19:21.195339    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j5m8w\" (UniqueName: \"kubernetes.io/projected/26c0d3a1-95f7-45eb-bc66-b81b60b4260d-kube-api-access-j5m8w\") on node \"ip-10-0-131-112.ec2.internal\" DevicePath \"\""
Jul 29 00:19:21.195502 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:19:21.195479    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/26c0d3a1-95f7-45eb-bc66-b81b60b4260d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "26c0d3a1-95f7-45eb-bc66-b81b60b4260d" (UID: "26c0d3a1-95f7-45eb-bc66-b81b60b4260d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 00:19:21.296546 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:19:21.296513    2572 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/26c0d3a1-95f7-45eb-bc66-b81b60b4260d-openshift-service-ca\") on node \"ip-10-0-131-112.ec2.internal\" DevicePath \"\""
Jul 29 00:19:21.679207 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:19:21.679170    2572 generic.go:356] "Generic (PLEG): container finished" podID="26c0d3a1-95f7-45eb-bc66-b81b60b4260d" containerID="bc9a3d0f81f6a06264f17e1f316e6b5acb16f8ba2263aeafa3a7996016f72cb6" exitCode=0
Jul 29 00:19:21.679406 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:19:21.679219    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-daedc5-649c5c6ff8-vvd4h"
Jul 29 00:19:21.679406 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:19:21.679259    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-daedc5-649c5c6ff8-vvd4h" event={"ID":"26c0d3a1-95f7-45eb-bc66-b81b60b4260d","Type":"ContainerDied","Data":"bc9a3d0f81f6a06264f17e1f316e6b5acb16f8ba2263aeafa3a7996016f72cb6"}
Jul 29 00:19:21.679406 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:19:21.679315    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-daedc5-649c5c6ff8-vvd4h" event={"ID":"26c0d3a1-95f7-45eb-bc66-b81b60b4260d","Type":"ContainerDied","Data":"bc52a909a5755cfc5c09e36eac89b9dd372ddc44666147e02be4bffc341ad073"}
Jul 29 00:19:21.679406 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:19:21.679337    2572 scope.go:117] "RemoveContainer" containerID="bc9a3d0f81f6a06264f17e1f316e6b5acb16f8ba2263aeafa3a7996016f72cb6"
Jul 29 00:19:21.688548 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:19:21.688534    2572 scope.go:117] "RemoveContainer" containerID="bc9a3d0f81f6a06264f17e1f316e6b5acb16f8ba2263aeafa3a7996016f72cb6"
Jul 29 00:19:21.688826 ip-10-0-131-112 kubenswrapper[2572]: E0729 00:19:21.688807    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bc9a3d0f81f6a06264f17e1f316e6b5acb16f8ba2263aeafa3a7996016f72cb6\": container with ID starting with bc9a3d0f81f6a06264f17e1f316e6b5acb16f8ba2263aeafa3a7996016f72cb6 not found: ID does not exist" containerID="bc9a3d0f81f6a06264f17e1f316e6b5acb16f8ba2263aeafa3a7996016f72cb6"
Jul 29 00:19:21.688890 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:19:21.688833    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bc9a3d0f81f6a06264f17e1f316e6b5acb16f8ba2263aeafa3a7996016f72cb6"} err="failed to get container status \"bc9a3d0f81f6a06264f17e1f316e6b5acb16f8ba2263aeafa3a7996016f72cb6\": rpc error: code = NotFound desc = could not find container \"bc9a3d0f81f6a06264f17e1f316e6b5acb16f8ba2263aeafa3a7996016f72cb6\": container with ID starting with bc9a3d0f81f6a06264f17e1f316e6b5acb16f8ba2263aeafa3a7996016f72cb6 not found: ID does not exist"
Jul 29 00:19:21.703493 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:19:21.703466    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-daedc5-649c5c6ff8-vvd4h"]
Jul 29 00:19:21.710998 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:19:21.710977    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-daedc5-649c5c6ff8-vvd4h"]
Jul 29 00:19:21.865951 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:19:21.865916    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="26c0d3a1-95f7-45eb-bc66-b81b60b4260d" path="/var/lib/kubelet/pods/26c0d3a1-95f7-45eb-bc66-b81b60b4260d/volumes"
Jul 29 00:19:21.866304 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:19:21.866230    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="40943638-4a91-4afd-b810-eca062eb7d05" path="/var/lib/kubelet/pods/40943638-4a91-4afd-b810-eca062eb7d05/volumes"
Jul 29 00:19:38.549075 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:19:38.549041    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-fa6130-78b7c5ddfb-rt8hz"]
Jul 29 00:19:38.549568 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:19:38.549319    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-fa6130-78b7c5ddfb-rt8hz" podUID="38dcdca6-ce6a-449c-9c75-abc89f5fcec1" containerName="maas-api" containerID="cri-o://a6a3fc420cbfdf285f4959f3710fecbed79fce8e62d5b04462b7d8ae87897834" gracePeriod=30
Jul 29 00:19:38.759297 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:19:38.759262    2572 generic.go:356] "Generic (PLEG): container finished" podID="38dcdca6-ce6a-449c-9c75-abc89f5fcec1" containerID="a6a3fc420cbfdf285f4959f3710fecbed79fce8e62d5b04462b7d8ae87897834" exitCode=0
Jul 29 00:19:38.759479 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:19:38.759320    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-fa6130-78b7c5ddfb-rt8hz" event={"ID":"38dcdca6-ce6a-449c-9c75-abc89f5fcec1","Type":"ContainerDied","Data":"a6a3fc420cbfdf285f4959f3710fecbed79fce8e62d5b04462b7d8ae87897834"}
Jul 29 00:19:38.800712 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:19:38.800646    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-fa6130-78b7c5ddfb-rt8hz"
Jul 29 00:19:38.845258 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:19:38.845226    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/38dcdca6-ce6a-449c-9c75-abc89f5fcec1-maas-api-tls\") pod \"38dcdca6-ce6a-449c-9c75-abc89f5fcec1\" (UID: \"38dcdca6-ce6a-449c-9c75-abc89f5fcec1\") "
Jul 29 00:19:38.845412 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:19:38.845277    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/38dcdca6-ce6a-449c-9c75-abc89f5fcec1-openshift-service-ca\") pod \"38dcdca6-ce6a-449c-9c75-abc89f5fcec1\" (UID: \"38dcdca6-ce6a-449c-9c75-abc89f5fcec1\") "
Jul 29 00:19:38.845412 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:19:38.845307    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pbdwn\" (UniqueName: \"kubernetes.io/projected/38dcdca6-ce6a-449c-9c75-abc89f5fcec1-kube-api-access-pbdwn\") pod \"38dcdca6-ce6a-449c-9c75-abc89f5fcec1\" (UID: \"38dcdca6-ce6a-449c-9c75-abc89f5fcec1\") "
Jul 29 00:19:38.845694 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:19:38.845663    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/38dcdca6-ce6a-449c-9c75-abc89f5fcec1-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "38dcdca6-ce6a-449c-9c75-abc89f5fcec1" (UID: "38dcdca6-ce6a-449c-9c75-abc89f5fcec1"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 00:19:38.847442 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:19:38.847419    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/38dcdca6-ce6a-449c-9c75-abc89f5fcec1-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "38dcdca6-ce6a-449c-9c75-abc89f5fcec1" (UID: "38dcdca6-ce6a-449c-9c75-abc89f5fcec1"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 00:19:38.847520 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:19:38.847499    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/38dcdca6-ce6a-449c-9c75-abc89f5fcec1-kube-api-access-pbdwn" (OuterVolumeSpecName: "kube-api-access-pbdwn") pod "38dcdca6-ce6a-449c-9c75-abc89f5fcec1" (UID: "38dcdca6-ce6a-449c-9c75-abc89f5fcec1"). InnerVolumeSpecName "kube-api-access-pbdwn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 00:19:38.946887 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:19:38.946849    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/38dcdca6-ce6a-449c-9c75-abc89f5fcec1-maas-api-tls\") on node \"ip-10-0-131-112.ec2.internal\" DevicePath \"\""
Jul 29 00:19:38.946887 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:19:38.946887    2572 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/38dcdca6-ce6a-449c-9c75-abc89f5fcec1-openshift-service-ca\") on node \"ip-10-0-131-112.ec2.internal\" DevicePath \"\""
Jul 29 00:19:38.947138 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:19:38.946903    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pbdwn\" (UniqueName: \"kubernetes.io/projected/38dcdca6-ce6a-449c-9c75-abc89f5fcec1-kube-api-access-pbdwn\") on node \"ip-10-0-131-112.ec2.internal\" DevicePath \"\""
Jul 29 00:19:39.764738 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:19:39.764707    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-fa6130-78b7c5ddfb-rt8hz"
Jul 29 00:19:39.764738 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:19:39.764720    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-fa6130-78b7c5ddfb-rt8hz" event={"ID":"38dcdca6-ce6a-449c-9c75-abc89f5fcec1","Type":"ContainerDied","Data":"4b0bce7e7a2774ec6354985b8d7afe3a1b1b3c08e731ff6c8a052fc6e42d2479"}
Jul 29 00:19:39.765223 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:19:39.764779    2572 scope.go:117] "RemoveContainer" containerID="a6a3fc420cbfdf285f4959f3710fecbed79fce8e62d5b04462b7d8ae87897834"
Jul 29 00:19:39.787326 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:19:39.787291    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-fa6130-78b7c5ddfb-rt8hz"]
Jul 29 00:19:39.788875 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:19:39.788854    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-fa6130-78b7c5ddfb-rt8hz"]
Jul 29 00:19:39.865615 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:19:39.865584    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="38dcdca6-ce6a-449c-9c75-abc89f5fcec1" path="/var/lib/kubelet/pods/38dcdca6-ce6a-449c-9c75-abc89f5fcec1/volumes"
Jul 29 00:19:45.280204 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:19:45.280161    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-fa6130-openshift-default-86f8b5f84-7nzs8"]
Jul 29 00:19:45.280800 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:19:45.280411    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-fa6130-openshift-default-86f8b5f84-7nzs8" podUID="cf43995d-f389-47b4-992f-4df567ed653c" containerName="istio-proxy" containerID="cri-o://ecdd532590da6af16ab9dc705fa50968cdae04b6dd3298a539fc09f3bb963433" gracePeriod=30
Jul 29 00:19:50.534853 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:19:50.534827    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-fa6130-openshift-default-86f8b5f84-7nzs8"
Jul 29 00:19:50.650146 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:19:50.650062    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf43995d-f389-47b4-992f-4df567ed653c-workload-socket\") pod \"cf43995d-f389-47b4-992f-4df567ed653c\" (UID: \"cf43995d-f389-47b4-992f-4df567ed653c\") "
Jul 29 00:19:50.650146 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:19:50.650100    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cf43995d-f389-47b4-992f-4df567ed653c-workload-certs\") pod \"cf43995d-f389-47b4-992f-4df567ed653c\" (UID: \"cf43995d-f389-47b4-992f-4df567ed653c\") "
Jul 29 00:19:50.650146 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:19:50.650133    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cf43995d-f389-47b4-992f-4df567ed653c-istio-envoy\") pod \"cf43995d-f389-47b4-992f-4df567ed653c\" (UID: \"cf43995d-f389-47b4-992f-4df567ed653c\") "
Jul 29 00:19:50.650447 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:19:50.650155    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cf43995d-f389-47b4-992f-4df567ed653c-istio-data\") pod \"cf43995d-f389-47b4-992f-4df567ed653c\" (UID: \"cf43995d-f389-47b4-992f-4df567ed653c\") "
Jul 29 00:19:50.650447 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:19:50.650182    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cf43995d-f389-47b4-992f-4df567ed653c-istio-token\") pod \"cf43995d-f389-47b4-992f-4df567ed653c\" (UID: \"cf43995d-f389-47b4-992f-4df567ed653c\") "
Jul 29 00:19:50.650447 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:19:50.650205    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dvjsd\" (UniqueName: \"kubernetes.io/projected/cf43995d-f389-47b4-992f-4df567ed653c-kube-api-access-dvjsd\") pod \"cf43995d-f389-47b4-992f-4df567ed653c\" (UID: \"cf43995d-f389-47b4-992f-4df567ed653c\") "
Jul 29 00:19:50.650447 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:19:50.650279    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cf43995d-f389-47b4-992f-4df567ed653c-istio-podinfo\") pod \"cf43995d-f389-47b4-992f-4df567ed653c\" (UID: \"cf43995d-f389-47b4-992f-4df567ed653c\") "
Jul 29 00:19:50.650447 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:19:50.650307    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cf43995d-f389-47b4-992f-4df567ed653c-istiod-ca-cert\") pod \"cf43995d-f389-47b4-992f-4df567ed653c\" (UID: \"cf43995d-f389-47b4-992f-4df567ed653c\") "
Jul 29 00:19:50.650447 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:19:50.650334    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf43995d-f389-47b4-992f-4df567ed653c-credential-socket\") pod \"cf43995d-f389-47b4-992f-4df567ed653c\" (UID: \"cf43995d-f389-47b4-992f-4df567ed653c\") "
Jul 29 00:19:50.650447 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:19:50.650350    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf43995d-f389-47b4-992f-4df567ed653c-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "cf43995d-f389-47b4-992f-4df567ed653c" (UID: "cf43995d-f389-47b4-992f-4df567ed653c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 00:19:50.650447 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:19:50.650395    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf43995d-f389-47b4-992f-4df567ed653c-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "cf43995d-f389-47b4-992f-4df567ed653c" (UID: "cf43995d-f389-47b4-992f-4df567ed653c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 00:19:50.650897 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:19:50.650590    2572 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf43995d-f389-47b4-992f-4df567ed653c-workload-socket\") on node \"ip-10-0-131-112.ec2.internal\" DevicePath \"\""
Jul 29 00:19:50.650897 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:19:50.650609    2572 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cf43995d-f389-47b4-992f-4df567ed653c-workload-certs\") on node \"ip-10-0-131-112.ec2.internal\" DevicePath \"\""
Jul 29 00:19:50.650897 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:19:50.650648    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf43995d-f389-47b4-992f-4df567ed653c-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "cf43995d-f389-47b4-992f-4df567ed653c" (UID: "cf43995d-f389-47b4-992f-4df567ed653c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 00:19:50.650897 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:19:50.650730    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cf43995d-f389-47b4-992f-4df567ed653c-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "cf43995d-f389-47b4-992f-4df567ed653c" (UID: "cf43995d-f389-47b4-992f-4df567ed653c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 00:19:50.651211 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:19:50.651190    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf43995d-f389-47b4-992f-4df567ed653c-istio-data" (OuterVolumeSpecName: "istio-data") pod "cf43995d-f389-47b4-992f-4df567ed653c" (UID: "cf43995d-f389-47b4-992f-4df567ed653c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 00:19:50.652656 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:19:50.652603    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/cf43995d-f389-47b4-992f-4df567ed653c-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "cf43995d-f389-47b4-992f-4df567ed653c" (UID: "cf43995d-f389-47b4-992f-4df567ed653c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 29 00:19:50.652798 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:19:50.652775    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cf43995d-f389-47b4-992f-4df567ed653c-istio-token" (OuterVolumeSpecName: "istio-token") pod "cf43995d-f389-47b4-992f-4df567ed653c" (UID: "cf43995d-f389-47b4-992f-4df567ed653c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 00:19:50.652985 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:19:50.652948    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cf43995d-f389-47b4-992f-4df567ed653c-kube-api-access-dvjsd" (OuterVolumeSpecName: "kube-api-access-dvjsd") pod "cf43995d-f389-47b4-992f-4df567ed653c" (UID: "cf43995d-f389-47b4-992f-4df567ed653c"). InnerVolumeSpecName "kube-api-access-dvjsd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 00:19:50.653077 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:19:50.653005    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf43995d-f389-47b4-992f-4df567ed653c-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "cf43995d-f389-47b4-992f-4df567ed653c" (UID: "cf43995d-f389-47b4-992f-4df567ed653c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 00:19:50.751819 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:19:50.751780    2572 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cf43995d-f389-47b4-992f-4df567ed653c-istio-envoy\") on node \"ip-10-0-131-112.ec2.internal\" DevicePath \"\""
Jul 29 00:19:50.751819 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:19:50.751811    2572 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cf43995d-f389-47b4-992f-4df567ed653c-istio-data\") on node \"ip-10-0-131-112.ec2.internal\" DevicePath \"\""
Jul 29 00:19:50.751819 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:19:50.751820    2572 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cf43995d-f389-47b4-992f-4df567ed653c-istio-token\") on node \"ip-10-0-131-112.ec2.internal\" DevicePath \"\""
Jul 29 00:19:50.752052 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:19:50.751830    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dvjsd\" (UniqueName: \"kubernetes.io/projected/cf43995d-f389-47b4-992f-4df567ed653c-kube-api-access-dvjsd\") on node \"ip-10-0-131-112.ec2.internal\" DevicePath \"\""
Jul 29 00:19:50.752052 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:19:50.751841    2572 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cf43995d-f389-47b4-992f-4df567ed653c-istio-podinfo\") on node \"ip-10-0-131-112.ec2.internal\" DevicePath \"\""
Jul 29 00:19:50.752052 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:19:50.751849    2572 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cf43995d-f389-47b4-992f-4df567ed653c-istiod-ca-cert\") on node \"ip-10-0-131-112.ec2.internal\" DevicePath \"\""
Jul 29 00:19:50.752052 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:19:50.751858    2572 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf43995d-f389-47b4-992f-4df567ed653c-credential-socket\") on node \"ip-10-0-131-112.ec2.internal\" DevicePath \"\""
Jul 29 00:19:50.816553 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:19:50.816518    2572 generic.go:356] "Generic (PLEG): container finished" podID="cf43995d-f389-47b4-992f-4df567ed653c" containerID="ecdd532590da6af16ab9dc705fa50968cdae04b6dd3298a539fc09f3bb963433" exitCode=0
Jul 29 00:19:50.816752 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:19:50.816566    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-fa6130-openshift-default-86f8b5f84-7nzs8"
Jul 29 00:19:50.816752 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:19:50.816602    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-fa6130-openshift-default-86f8b5f84-7nzs8" event={"ID":"cf43995d-f389-47b4-992f-4df567ed653c","Type":"ContainerDied","Data":"ecdd532590da6af16ab9dc705fa50968cdae04b6dd3298a539fc09f3bb963433"}
Jul 29 00:19:50.816752 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:19:50.816672    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-fa6130-openshift-default-86f8b5f84-7nzs8" event={"ID":"cf43995d-f389-47b4-992f-4df567ed653c","Type":"ContainerDied","Data":"2d6d3bfe9a75400903c32b09560a61e469b147a42fc9b123ab67af4ce39cc070"}
Jul 29 00:19:50.816752 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:19:50.816697    2572 scope.go:117] "RemoveContainer" containerID="ecdd532590da6af16ab9dc705fa50968cdae04b6dd3298a539fc09f3bb963433"
Jul 29 00:19:50.825553 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:19:50.825538    2572 scope.go:117] "RemoveContainer" containerID="d439127331bda97a8104e9851dfd95a758dc0e68af8128e2788f65c61e2c202f"
Jul 29 00:19:50.833255 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:19:50.833238    2572 scope.go:117] "RemoveContainer" containerID="ecdd532590da6af16ab9dc705fa50968cdae04b6dd3298a539fc09f3bb963433"
Jul 29 00:19:50.833531 ip-10-0-131-112 kubenswrapper[2572]: E0729 00:19:50.833513    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ecdd532590da6af16ab9dc705fa50968cdae04b6dd3298a539fc09f3bb963433\": container with ID starting with ecdd532590da6af16ab9dc705fa50968cdae04b6dd3298a539fc09f3bb963433 not found: ID does not exist" containerID="ecdd532590da6af16ab9dc705fa50968cdae04b6dd3298a539fc09f3bb963433"
Jul 29 00:19:50.833589 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:19:50.833540    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ecdd532590da6af16ab9dc705fa50968cdae04b6dd3298a539fc09f3bb963433"} err="failed to get container status \"ecdd532590da6af16ab9dc705fa50968cdae04b6dd3298a539fc09f3bb963433\": rpc error: code = NotFound desc = could not find container \"ecdd532590da6af16ab9dc705fa50968cdae04b6dd3298a539fc09f3bb963433\": container with ID starting with ecdd532590da6af16ab9dc705fa50968cdae04b6dd3298a539fc09f3bb963433 not found: ID does not exist"
Jul 29 00:19:50.833589 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:19:50.833557    2572 scope.go:117] "RemoveContainer" containerID="d439127331bda97a8104e9851dfd95a758dc0e68af8128e2788f65c61e2c202f"
Jul 29 00:19:50.833817 ip-10-0-131-112 kubenswrapper[2572]: E0729 00:19:50.833802    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d439127331bda97a8104e9851dfd95a758dc0e68af8128e2788f65c61e2c202f\": container with ID starting with d439127331bda97a8104e9851dfd95a758dc0e68af8128e2788f65c61e2c202f not found: ID does not exist" containerID="d439127331bda97a8104e9851dfd95a758dc0e68af8128e2788f65c61e2c202f"
Jul 29 00:19:50.833870 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:19:50.833820    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d439127331bda97a8104e9851dfd95a758dc0e68af8128e2788f65c61e2c202f"} err="failed to get container status \"d439127331bda97a8104e9851dfd95a758dc0e68af8128e2788f65c61e2c202f\": rpc error: code = NotFound desc = could not find container \"d439127331bda97a8104e9851dfd95a758dc0e68af8128e2788f65c61e2c202f\": container with ID starting with d439127331bda97a8104e9851dfd95a758dc0e68af8128e2788f65c61e2c202f not found: ID does not exist"
Jul 29 00:19:50.837601 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:19:50.837575    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-fa6130-openshift-default-86f8b5f84-7nzs8"]
Jul 29 00:19:50.841358 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:19:50.841340    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-fa6130-openshift-default-86f8b5f84-7nzs8"]
Jul 29 00:19:51.534268 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:19:51.534240    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-99946b6f7-x5hp9_e1c66ad8-11e8-4447-b28f-decc108cfaf5/manager/0.log"
Jul 29 00:19:51.864764 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:19:51.864680    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cf43995d-f389-47b4-992f-4df567ed653c" path="/var/lib/kubelet/pods/cf43995d-f389-47b4-992f-4df567ed653c/volumes"
Jul 29 00:19:52.109737 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:19:52.109710    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-dc9c5fb48-rldzn_5957eb3d-3eb0-4d40-86d9-92c0e31b9021/copy-manifests/0.log"
Jul 29 00:19:52.121436 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:19:52.121374    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-dc9c5fb48-rldzn_5957eb3d-3eb0-4d40-86d9-92c0e31b9021/manager/0.log"
Jul 29 00:19:52.466206 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:19:52.466131    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-d69cbc5b-6r7tb_233d13f2-5403-433a-9c4a-09fb722e747d/maas-api/0.log"
Jul 29 00:19:55.414897 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:19:55.414863    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-sf2rk_2cbc6725-3392-4245-b66f-cefbadc7b5b4/discovery/0.log"
Jul 29 00:19:55.746064 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:19:55.746032    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-wvxnh_f59a2eeb-6c07-48ed-88e5-35f2bd35db3c/istio-proxy/1.log"
Jul 29 00:19:56.448005 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:19:56.447975    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-6zd86_36a2cdec-3fba-4930-9fb7-afc7ceec1e0f/main/0.log"
Jul 29 00:19:56.782842 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:19:56.782811    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbjw4x_706f10a5-2ca0-4135-8f41-9ec3e86f05df/main/0.log"
Jul 29 00:19:56.885699 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:19:56.885668    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-k268w_ebfea51e-ea81-4923-afe3-986b98a60be4/main/0.log"
Jul 29 00:20:05.274470 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:20:05.274434    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-8pczd_f0137726-3966-44f4-bcc7-c69cc72143be/global-pull-secret-syncer/0.log"
Jul 29 00:20:05.438436 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:20:05.438396    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-6dwz5_b923a656-b1ea-41e7-8c65-440a24abcee6/konnectivity-agent/0.log"
Jul 29 00:20:05.489142 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:20:05.489112    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-131-112.ec2.internal_48ad21d512355ab82436faafc0eae6b0/haproxy/0.log"
Jul 29 00:20:13.358432 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:20:13.358395    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-ft86p_40858a66-ae35-4c67-850b-2ee7c1d4ace3/node-exporter/0.log"
Jul 29 00:20:13.380691 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:20:13.380618    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-ft86p_40858a66-ae35-4c67-850b-2ee7c1d4ace3/kube-rbac-proxy/0.log"
Jul 29 00:20:13.402659 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:20:13.402612    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-ft86p_40858a66-ae35-4c67-850b-2ee7c1d4ace3/init-textfile/0.log"
Jul 29 00:20:15.706681 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:20:15.706637    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-bwncz/perf-node-gather-daemonset-vqx5v"]
Jul 29 00:20:15.707165 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:20:15.707109    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="40943638-4a91-4afd-b810-eca062eb7d05" containerName="revoke-keys"
Jul 29 00:20:15.707165 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:20:15.707130    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="40943638-4a91-4afd-b810-eca062eb7d05" containerName="revoke-keys"
Jul 29 00:20:15.707165 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:20:15.707146    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="26c0d3a1-95f7-45eb-bc66-b81b60b4260d" containerName="maas-api"
Jul 29 00:20:15.707165 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:20:15.707153    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="26c0d3a1-95f7-45eb-bc66-b81b60b4260d" containerName="maas-api"
Jul 29 00:20:15.707379 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:20:15.707169    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="38dcdca6-ce6a-449c-9c75-abc89f5fcec1" containerName="maas-api"
Jul 29 00:20:15.707379 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:20:15.707178    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="38dcdca6-ce6a-449c-9c75-abc89f5fcec1" containerName="maas-api"
Jul 29 00:20:15.707379 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:20:15.707190    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cf43995d-f389-47b4-992f-4df567ed653c" containerName="istio-proxy"
Jul 29 00:20:15.707379 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:20:15.707199    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="cf43995d-f389-47b4-992f-4df567ed653c" containerName="istio-proxy"
Jul 29 00:20:15.707379 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:20:15.707223    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cf43995d-f389-47b4-992f-4df567ed653c" containerName="istio-proxy"
Jul 29 00:20:15.707379 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:20:15.707232    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="cf43995d-f389-47b4-992f-4df567ed653c" containerName="istio-proxy"
Jul 29 00:20:15.707379 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:20:15.707305    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="cf43995d-f389-47b4-992f-4df567ed653c" containerName="istio-proxy"
Jul 29 00:20:15.707379 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:20:15.707317    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="40943638-4a91-4afd-b810-eca062eb7d05" containerName="revoke-keys"
Jul 29 00:20:15.707379 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:20:15.707327    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="2530b4c9-9988-48e5-89ed-f61207a086f2" containerName="istio-proxy"
Jul 29 00:20:15.707379 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:20:15.707337    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="cf43995d-f389-47b4-992f-4df567ed653c" containerName="istio-proxy"
Jul 29 00:20:15.707379 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:20:15.707347    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="38dcdca6-ce6a-449c-9c75-abc89f5fcec1" containerName="maas-api"
Jul 29 00:20:15.707379 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:20:15.707357    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="26c0d3a1-95f7-45eb-bc66-b81b60b4260d" containerName="maas-api"
Jul 29 00:20:15.711860 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:20:15.711836    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-bwncz/perf-node-gather-daemonset-vqx5v"
Jul 29 00:20:15.714208 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:20:15.714190    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-bwncz\"/\"kube-root-ca.crt\""
Jul 29 00:20:15.715249 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:20:15.715229    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-bwncz\"/\"default-dockercfg-42lcq\""
Jul 29 00:20:15.715342 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:20:15.715238    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-bwncz\"/\"openshift-service-ca.crt\""
Jul 29 00:20:15.721919 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:20:15.721893    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-bwncz/perf-node-gather-daemonset-vqx5v"]
Jul 29 00:20:15.764057 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:20:15.764014    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/5786d5c4-24fe-464f-bfe6-e247c3dad7c9-proc\") pod \"perf-node-gather-daemonset-vqx5v\" (UID: \"5786d5c4-24fe-464f-bfe6-e247c3dad7c9\") " pod="openshift-must-gather-bwncz/perf-node-gather-daemonset-vqx5v"
Jul 29 00:20:15.764218 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:20:15.764090    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5786d5c4-24fe-464f-bfe6-e247c3dad7c9-sys\") pod \"perf-node-gather-daemonset-vqx5v\" (UID: \"5786d5c4-24fe-464f-bfe6-e247c3dad7c9\") " pod="openshift-must-gather-bwncz/perf-node-gather-daemonset-vqx5v"
Jul 29 00:20:15.764218 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:20:15.764117    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/5786d5c4-24fe-464f-bfe6-e247c3dad7c9-podres\") pod \"perf-node-gather-daemonset-vqx5v\" (UID: \"5786d5c4-24fe-464f-bfe6-e247c3dad7c9\") " pod="openshift-must-gather-bwncz/perf-node-gather-daemonset-vqx5v"
Jul 29 00:20:15.764218 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:20:15.764137    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dwnsw\" (UniqueName: \"kubernetes.io/projected/5786d5c4-24fe-464f-bfe6-e247c3dad7c9-kube-api-access-dwnsw\") pod \"perf-node-gather-daemonset-vqx5v\" (UID: \"5786d5c4-24fe-464f-bfe6-e247c3dad7c9\") " pod="openshift-must-gather-bwncz/perf-node-gather-daemonset-vqx5v"
Jul 29 00:20:15.764328 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:20:15.764245    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5786d5c4-24fe-464f-bfe6-e247c3dad7c9-lib-modules\") pod \"perf-node-gather-daemonset-vqx5v\" (UID: \"5786d5c4-24fe-464f-bfe6-e247c3dad7c9\") " pod="openshift-must-gather-bwncz/perf-node-gather-daemonset-vqx5v"
Jul 29 00:20:15.864607 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:20:15.864574    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5786d5c4-24fe-464f-bfe6-e247c3dad7c9-lib-modules\") pod \"perf-node-gather-daemonset-vqx5v\" (UID: \"5786d5c4-24fe-464f-bfe6-e247c3dad7c9\") " pod="openshift-must-gather-bwncz/perf-node-gather-daemonset-vqx5v"
Jul 29 00:20:15.864767 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:20:15.864618    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/5786d5c4-24fe-464f-bfe6-e247c3dad7c9-proc\") pod \"perf-node-gather-daemonset-vqx5v\" (UID: \"5786d5c4-24fe-464f-bfe6-e247c3dad7c9\") " pod="openshift-must-gather-bwncz/perf-node-gather-daemonset-vqx5v"
Jul 29 00:20:15.864767 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:20:15.864676    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5786d5c4-24fe-464f-bfe6-e247c3dad7c9-sys\") pod \"perf-node-gather-daemonset-vqx5v\" (UID: \"5786d5c4-24fe-464f-bfe6-e247c3dad7c9\") " pod="openshift-must-gather-bwncz/perf-node-gather-daemonset-vqx5v"
Jul 29 00:20:15.864767 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:20:15.864701    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/5786d5c4-24fe-464f-bfe6-e247c3dad7c9-podres\") pod \"perf-node-gather-daemonset-vqx5v\" (UID: \"5786d5c4-24fe-464f-bfe6-e247c3dad7c9\") " pod="openshift-must-gather-bwncz/perf-node-gather-daemonset-vqx5v"
Jul 29 00:20:15.864767 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:20:15.864727    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dwnsw\" (UniqueName: \"kubernetes.io/projected/5786d5c4-24fe-464f-bfe6-e247c3dad7c9-kube-api-access-dwnsw\") pod \"perf-node-gather-daemonset-vqx5v\" (UID: \"5786d5c4-24fe-464f-bfe6-e247c3dad7c9\") " pod="openshift-must-gather-bwncz/perf-node-gather-daemonset-vqx5v"
Jul 29 00:20:15.864945 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:20:15.864781    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/5786d5c4-24fe-464f-bfe6-e247c3dad7c9-proc\") pod \"perf-node-gather-daemonset-vqx5v\" (UID: \"5786d5c4-24fe-464f-bfe6-e247c3dad7c9\") " pod="openshift-must-gather-bwncz/perf-node-gather-daemonset-vqx5v"
Jul 29 00:20:15.864945 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:20:15.864800    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5786d5c4-24fe-464f-bfe6-e247c3dad7c9-sys\") pod \"perf-node-gather-daemonset-vqx5v\" (UID: \"5786d5c4-24fe-464f-bfe6-e247c3dad7c9\") " pod="openshift-must-gather-bwncz/perf-node-gather-daemonset-vqx5v"
Jul 29 00:20:15.864945 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:20:15.864781    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5786d5c4-24fe-464f-bfe6-e247c3dad7c9-lib-modules\") pod \"perf-node-gather-daemonset-vqx5v\" (UID: \"5786d5c4-24fe-464f-bfe6-e247c3dad7c9\") " pod="openshift-must-gather-bwncz/perf-node-gather-daemonset-vqx5v"
Jul 29 00:20:15.864945 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:20:15.864859    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/5786d5c4-24fe-464f-bfe6-e247c3dad7c9-podres\") pod \"perf-node-gather-daemonset-vqx5v\" (UID: \"5786d5c4-24fe-464f-bfe6-e247c3dad7c9\") " pod="openshift-must-gather-bwncz/perf-node-gather-daemonset-vqx5v"
Jul 29 00:20:15.872402 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:20:15.872380    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dwnsw\" (UniqueName: \"kubernetes.io/projected/5786d5c4-24fe-464f-bfe6-e247c3dad7c9-kube-api-access-dwnsw\") pod \"perf-node-gather-daemonset-vqx5v\" (UID: \"5786d5c4-24fe-464f-bfe6-e247c3dad7c9\") " pod="openshift-must-gather-bwncz/perf-node-gather-daemonset-vqx5v"
Jul 29 00:20:16.023135 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:20:16.023095    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-bwncz/perf-node-gather-daemonset-vqx5v"
Jul 29 00:20:16.148776 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:20:16.148748    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-bwncz/perf-node-gather-daemonset-vqx5v"]
Jul 29 00:20:16.150267 ip-10-0-131-112 kubenswrapper[2572]: W0729 00:20:16.150239    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod5786d5c4_24fe_464f_bfe6_e247c3dad7c9.slice/crio-7d540b7296e7c69d5b796f22e20f9aab9931fc5517463191785c5abb5e448eb5 WatchSource:0}: Error finding container 7d540b7296e7c69d5b796f22e20f9aab9931fc5517463191785c5abb5e448eb5: Status 404 returned error can't find the container with id 7d540b7296e7c69d5b796f22e20f9aab9931fc5517463191785c5abb5e448eb5
Jul 29 00:20:16.929167 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:20:16.929133    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-bwncz/perf-node-gather-daemonset-vqx5v" event={"ID":"5786d5c4-24fe-464f-bfe6-e247c3dad7c9","Type":"ContainerStarted","Data":"692411918bb50defad6196fca92c77ca196c8f571d45985724a94531fd2e6fe2"}
Jul 29 00:20:16.929752 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:20:16.929184    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-bwncz/perf-node-gather-daemonset-vqx5v" event={"ID":"5786d5c4-24fe-464f-bfe6-e247c3dad7c9","Type":"ContainerStarted","Data":"7d540b7296e7c69d5b796f22e20f9aab9931fc5517463191785c5abb5e448eb5"}
Jul 29 00:20:16.929752 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:20:16.929268    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-bwncz/perf-node-gather-daemonset-vqx5v"
Jul 29 00:20:16.945229 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:20:16.945182    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-bwncz/perf-node-gather-daemonset-vqx5v" podStartSLOduration=1.9451687629999999 podStartE2EDuration="1.945168763s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 00:20:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:20:16.943029562 +0000 UTC m=+3493.635704073" watchObservedRunningTime="2026-07-29 00:20:16.945168763 +0000 UTC m=+3493.637843272"
Jul 29 00:20:18.271513 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:20:18.271484    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-8k4xj_3d670d49-7bd6-47f9-befe-d76cce20834a/dns/0.log"
Jul 29 00:20:18.292374 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:20:18.292341    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-8k4xj_3d670d49-7bd6-47f9-befe-d76cce20834a/kube-rbac-proxy/0.log"
Jul 29 00:20:18.341433 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:20:18.341402    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-xqqk5_82ce0587-81a8-4b5e-b841-14fed1b2364c/dns-node-resolver/0.log"
Jul 29 00:20:18.922888 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:20:18.922862    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-5ndfr_5c9cc37c-daaf-4fa1-b71e-94bcc6732adb/node-ca/0.log"
Jul 29 00:20:20.086085 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:20:20.086049    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-sf2rk_2cbc6725-3392-4245-b66f-cefbadc7b5b4/discovery/0.log"
Jul 29 00:20:20.156767 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:20:20.156730    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-wvxnh_f59a2eeb-6c07-48ed-88e5-35f2bd35db3c/istio-proxy/0.log"
Jul 29 00:20:20.184167 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:20:20.184131    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-wvxnh_f59a2eeb-6c07-48ed-88e5-35f2bd35db3c/istio-proxy/1.log"
Jul 29 00:20:21.086743 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:20:21.086713    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-gwrtv_ad652290-064c-4b42-8419-bdd779497c86/serve-healthcheck-canary/0.log"
Jul 29 00:20:21.628066 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:20:21.628031    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-4p55m_2e08e8ce-573f-4068-8c3b-96315b3d60ce/kube-rbac-proxy/0.log"
Jul 29 00:20:21.649871 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:20:21.649845    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-4p55m_2e08e8ce-573f-4068-8c3b-96315b3d60ce/exporter/0.log"
Jul 29 00:20:21.670131 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:20:21.670107    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-4p55m_2e08e8ce-573f-4068-8c3b-96315b3d60ce/extractor/0.log"
Jul 29 00:20:22.942956 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:20:22.942923    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-bwncz/perf-node-gather-daemonset-vqx5v"
Jul 29 00:20:24.338997 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:20:24.338958    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-99946b6f7-x5hp9_e1c66ad8-11e8-4447-b28f-decc108cfaf5/manager/0.log"
Jul 29 00:20:24.627575 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:20:24.627490    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-dc9c5fb48-rldzn_5957eb3d-3eb0-4d40-86d9-92c0e31b9021/manager/0.log"
Jul 29 00:20:24.643979 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:20:24.643953    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-dc9c5fb48-rldzn_5957eb3d-3eb0-4d40-86d9-92c0e31b9021/copy-manifests/0.log"
Jul 29 00:20:26.060478 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:20:26.060452    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-7858699d7d-p25qn_bb9153ff-89de-4865-8ebe-0c91921dda7c/manager/0.log"
Jul 29 00:20:26.517039 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:20:26.516994    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-sf2rk_2cbc6725-3392-4245-b66f-cefbadc7b5b4/discovery/0.log"
Jul 29 00:20:26.565453 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:20:26.565420    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-wvxnh_f59a2eeb-6c07-48ed-88e5-35f2bd35db3c/istio-proxy/0.log"
Jul 29 00:20:26.592471 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:20:26.592441    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-wvxnh_f59a2eeb-6c07-48ed-88e5-35f2bd35db3c/istio-proxy/1.log"
Jul 29 00:20:28.202706 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:20:28.202672    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-sf2rk_2cbc6725-3392-4245-b66f-cefbadc7b5b4/discovery/0.log"
Jul 29 00:20:28.251111 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:20:28.251061    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-wvxnh_f59a2eeb-6c07-48ed-88e5-35f2bd35db3c/istio-proxy/0.log"
Jul 29 00:20:28.278525 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:20:28.278488    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-wvxnh_f59a2eeb-6c07-48ed-88e5-35f2bd35db3c/istio-proxy/1.log"
Jul 29 00:20:31.132896 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:20:31.132870    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-6865c9967-gvg5f_de30b99d-4a8d-4307-8e38-ec8ca095c1e0/migrator/0.log"
Jul 29 00:20:31.155386 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:20:31.155356    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-6865c9967-gvg5f_de30b99d-4a8d-4307-8e38-ec8ca095c1e0/graceful-termination/0.log"
Jul 29 00:20:32.817697 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:20:32.817657    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-v6nmr_f220827f-670e-407f-b5c9-664800ba743d/kube-multus-additional-cni-plugins/0.log"
Jul 29 00:20:32.840343 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:20:32.840315    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-v6nmr_f220827f-670e-407f-b5c9-664800ba743d/egress-router-binary-copy/0.log"
Jul 29 00:20:32.864088 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:20:32.864064    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-v6nmr_f220827f-670e-407f-b5c9-664800ba743d/cni-plugins/0.log"
Jul 29 00:20:32.888977 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:20:32.888951    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-v6nmr_f220827f-670e-407f-b5c9-664800ba743d/bond-cni-plugin/0.log"
Jul 29 00:20:32.915275 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:20:32.915245    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-v6nmr_f220827f-670e-407f-b5c9-664800ba743d/routeoverride-cni/0.log"
Jul 29 00:20:32.942697 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:20:32.942674    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-v6nmr_f220827f-670e-407f-b5c9-664800ba743d/whereabouts-cni-bincopy/0.log"
Jul 29 00:20:32.965253 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:20:32.965228    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-v6nmr_f220827f-670e-407f-b5c9-664800ba743d/whereabouts-cni/0.log"
Jul 29 00:20:33.077051 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:20:33.076973    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-l85q7_dc10426a-a32b-4910-a2c8-21345f7b691b/kube-multus/0.log"
Jul 29 00:20:33.159986 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:20:33.159955    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-lpj5x_df9e50e4-7650-40bb-83eb-becf4ba8c051/network-metrics-daemon/0.log"
Jul 29 00:20:33.187194 ip-10-0-131-112 kubenswrapper[2572]: I0729 00:20:33.187168    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-lpj5x_df9e50e4-7650-40bb-83eb-becf4ba8c051/kube-rbac-proxy/0.log"