Jul 06 14:17:47.588857 ip-10-0-133-112 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 06 14:17:47.588870 ip-10-0-133-112 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 06 14:17:47.588880 ip-10-0-133-112 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 06 14:17:47.589198 ip-10-0-133-112 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 06 14:17:57.593817 ip-10-0-133-112 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 06 14:17:57.593833 ip-10-0-133-112 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 57b7aeecc624443d851cceed3228af6e --
Jul 06 14:19:37.572920 ip-10-0-133-112 systemd[1]: Starting Kubernetes Kubelet...
Jul 06 14:19:38.180557 ip-10-0-133-112 kubenswrapper[2559]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 06 14:19:38.180557 ip-10-0-133-112 kubenswrapper[2559]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 06 14:19:38.180557 ip-10-0-133-112 kubenswrapper[2559]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 06 14:19:38.180557 ip-10-0-133-112 kubenswrapper[2559]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 06 14:19:38.180557 ip-10-0-133-112 kubenswrapper[2559]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 06 14:19:38.180557 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.005152    2559 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 06 14:19:38.180557 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.010832    2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 06 14:19:38.180557 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.010846    2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 06 14:19:38.180557 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.010850    2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 06 14:19:38.180557 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.010853    2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 06 14:19:38.180557 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.010856    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 06 14:19:38.180557 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.010859    2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 06 14:19:38.180557 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.010862    2559 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 06 14:19:38.180557 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.010864    2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 06 14:19:38.081044 ip-10-0-133-112 systemd[1]: Started Kubernetes Kubelet.
Jul 06 14:19:38.198523 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.010867    2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 06 14:19:38.198523 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.010869    2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 06 14:19:38.198523 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.010874    2559 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 06 14:19:38.198523 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.010877    2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 06 14:19:38.198523 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.010879    2559 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 06 14:19:38.198523 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.010882    2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 06 14:19:38.198523 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.010884    2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 06 14:19:38.198523 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.010887    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 06 14:19:38.198523 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.010890    2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 06 14:19:38.198523 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.010893    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 06 14:19:38.198523 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.010895    2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 06 14:19:38.198523 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.010898    2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 06 14:19:38.198523 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.010907    2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 06 14:19:38.198523 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.010910    2559 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 06 14:19:38.198523 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.010914    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 06 14:19:38.198523 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.010916    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 06 14:19:38.198523 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.010919    2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 06 14:19:38.198523 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.010921    2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 06 14:19:38.198523 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.010924    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 06 14:19:38.198523 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.010927    2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 06 14:19:38.216835 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.010930    2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 06 14:19:38.216835 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.010933    2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 06 14:19:38.216835 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.010936    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 06 14:19:38.216835 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.010938    2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 06 14:19:38.216835 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.010941    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 06 14:19:38.216835 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.010943    2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 06 14:19:38.216835 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.010946    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 06 14:19:38.216835 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.010949    2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 06 14:19:38.216835 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.010959    2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 06 14:19:38.216835 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.010962    2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 06 14:19:38.216835 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.010966    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 06 14:19:38.216835 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.010968    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 06 14:19:38.216835 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.010971    2559 feature_gate.go:328] unrecognized feature gate: Example
Jul 06 14:19:38.216835 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.010974    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 06 14:19:38.216835 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.010977    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 06 14:19:38.216835 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.010979    2559 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 06 14:19:38.216835 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.010982    2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 06 14:19:38.216835 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.010984    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 06 14:19:38.216835 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.010987    2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 06 14:19:38.218591 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.010989    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 06 14:19:38.218591 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.010992    2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 06 14:19:38.218591 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.010995    2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 06 14:19:38.218591 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.010997    2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 06 14:19:38.218591 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.010999    2559 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 06 14:19:38.218591 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.011002    2559 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 06 14:19:38.218591 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.011005    2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 06 14:19:38.218591 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.011007    2559 feature_gate.go:328] unrecognized feature gate: Example2
Jul 06 14:19:38.218591 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.011009    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 06 14:19:38.218591 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.011012    2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 06 14:19:38.218591 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.011015    2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 06 14:19:38.218591 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.011018    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 06 14:19:38.218591 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.011020    2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 06 14:19:38.218591 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.011024    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 06 14:19:38.218591 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.011027    2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 06 14:19:38.218591 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.011030    2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 06 14:19:38.218591 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.011035    2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 06 14:19:38.218591 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.011040    2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 06 14:19:38.218591 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.011043    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 06 14:19:38.220459 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.011046    2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 06 14:19:38.220459 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.011049    2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 06 14:19:38.220459 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.011052    2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 06 14:19:38.220459 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.011055    2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 06 14:19:38.220459 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.011058    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 06 14:19:38.220459 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.011061    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 06 14:19:38.220459 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.011065    2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 06 14:19:38.220459 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.011069    2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 06 14:19:38.220459 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.011071    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 06 14:19:38.220459 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.011074    2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 06 14:19:38.220459 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.011076    2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 06 14:19:38.220459 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.011079    2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 06 14:19:38.220459 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.011083    2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 06 14:19:38.220459 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.011086    2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 06 14:19:38.220459 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.011089    2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 06 14:19:38.220459 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.011091    2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 06 14:19:38.220459 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.011094    2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 06 14:19:38.220459 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.011098    2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 06 14:19:38.220459 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.011101    2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 06 14:19:38.221329 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.011104    2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 06 14:19:38.221329 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.011107    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 06 14:19:38.221329 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.011109    2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 06 14:19:38.221329 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.011114    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 06 14:19:38.221329 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.011117    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 06 14:19:38.221329 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.011120    2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 06 14:19:38.221329 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.011122    2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 06 14:19:38.221329 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.011125    2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 06 14:19:38.221329 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.011128    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 06 14:19:38.221329 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.011130    2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 06 14:19:38.221329 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.011133    2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 06 14:19:38.221329 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.011135    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 06 14:19:38.221329 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.011138    2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 06 14:19:38.221329 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.011141    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 06 14:19:38.221329 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.011923    2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 06 14:19:38.221329 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.011930    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 06 14:19:38.221329 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.011934    2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 06 14:19:38.221329 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.011936    2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 06 14:19:38.221329 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.011939    2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 06 14:19:38.221329 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.011943    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 06 14:19:38.223400 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.011946    2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 06 14:19:38.223400 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.011948    2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 06 14:19:38.223400 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.011951    2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 06 14:19:38.223400 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.011954    2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 06 14:19:38.223400 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.011957    2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 06 14:19:38.223400 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.011960    2559 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 06 14:19:38.223400 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.011962    2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 06 14:19:38.223400 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.011965    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 06 14:19:38.223400 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.011967    2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 06 14:19:38.223400 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.011970    2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 06 14:19:38.223400 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.011973    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 06 14:19:38.223400 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.011975    2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 06 14:19:38.223400 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.011978    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 06 14:19:38.223400 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.011980    2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 06 14:19:38.223400 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.011983    2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 06 14:19:38.223400 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.011985    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 06 14:19:38.223400 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.011988    2559 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 06 14:19:38.223400 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.011990    2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 06 14:19:38.223400 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.011993    2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 06 14:19:38.223400 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.011996    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 06 14:19:38.224227 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.011998    2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 06 14:19:38.224227 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012001    2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 06 14:19:38.224227 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012003    2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 06 14:19:38.224227 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012007    2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 06 14:19:38.224227 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012010    2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 06 14:19:38.224227 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012013    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 06 14:19:38.224227 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012016    2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 06 14:19:38.224227 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012019    2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 06 14:19:38.224227 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012021    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 06 14:19:38.224227 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012024    2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 06 14:19:38.224227 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012026    2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 06 14:19:38.224227 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012029    2559 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 06 14:19:38.224227 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012033    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 06 14:19:38.224227 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012035    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 06 14:19:38.224227 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012038    2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 06 14:19:38.224227 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012041    2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 06 14:19:38.224227 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012044    2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 06 14:19:38.224227 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012046    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 06 14:19:38.224227 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012049    2559 feature_gate.go:328] unrecognized feature gate: Example
Jul 06 14:19:38.224227 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012051    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 06 14:19:38.226508 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012054    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 06 14:19:38.226508 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012056    2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 06 14:19:38.226508 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012059    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 06 14:19:38.226508 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012061    2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 06 14:19:38.226508 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012064    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 06 14:19:38.226508 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012066    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 06 14:19:38.226508 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012069    2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 06 14:19:38.226508 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012071    2559 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 06 14:19:38.226508 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012074    2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 06 14:19:38.226508 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012076    2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 06 14:19:38.226508 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012079    2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 06 14:19:38.226508 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012082    2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 06 14:19:38.226508 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012084    2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 06 14:19:38.226508 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012087    2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 06 14:19:38.226508 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012090    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 06 14:19:38.226508 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012093    2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 06 14:19:38.226508 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012095    2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 06 14:19:38.226508 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012098    2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 06 14:19:38.226508 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012100    2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 06 14:19:38.227157 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012103    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 06 14:19:38.227157 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012107    2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 06 14:19:38.227157 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012111    2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 06 14:19:38.227157 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012115    2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 06 14:19:38.227157 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012119    2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 06 14:19:38.227157 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012121    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 06 14:19:38.227157 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012130    2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 06 14:19:38.227157 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012134    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 06 14:19:38.227157 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012139    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 06 14:19:38.227157 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012142    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 06 14:19:38.227157 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012144    2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 06 14:19:38.227157 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012147    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 06 14:19:38.227157 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012150    2559 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 06 14:19:38.227157 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012154    2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 06 14:19:38.227157 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012157    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 06 14:19:38.227157 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012159    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 06 14:19:38.227157 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012162    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 06 14:19:38.227157 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012165    2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 06 14:19:38.228849 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012167    2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 06 14:19:38.228849 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012171    2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 06 14:19:38.228849 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012173    2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 06 14:19:38.228849 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012176    2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 06 14:19:38.228849 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012178    2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 06 14:19:38.228849 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012181    2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 06 14:19:38.228849 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012183    2559 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 06 14:19:38.228849 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012186    2559 feature_gate.go:328] unrecognized feature gate: Example2
Jul 06 14:19:38.228849 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012188    2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 06 14:19:38.228849 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012191    2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 06 14:19:38.228849 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012193    2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 06 14:19:38.228849 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012196    2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 06 14:19:38.228849 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012198    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 06 14:19:38.228849 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012201    2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 06 14:19:38.228849 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012203    2559 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 06 14:19:38.228849 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012206    2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 06 14:19:38.228849 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012678    2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 06 14:19:38.228849 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012683    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 06 14:19:38.228849 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012686    2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 06 14:19:38.228849 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012689    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 06 14:19:38.229668 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012697    2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 06 14:19:38.229668 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012700    2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 06 14:19:38.229668 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012705    2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 06 14:19:38.229668 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012707    2559 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 06 14:19:38.229668 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012710    2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 06 14:19:38.229668 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012712    2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 06 14:19:38.229668 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012715    2559 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 06 14:19:38.229668 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012717    2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 06 14:19:38.229668 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012720    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 06 14:19:38.229668 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012723    2559 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 06 14:19:38.229668 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012726    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 06 14:19:38.229668 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012728    2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 06 14:19:38.229668 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012731    2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 06 14:19:38.229668 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012734    2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 06 14:19:38.229668 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012737    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 06 14:19:38.229668 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012739    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 06 14:19:38.229668 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012742    2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 06 14:19:38.229668 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012744    2559 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 06 14:19:38.229668 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012748    2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 06 14:19:38.229668 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012750    2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 06 14:19:38.230538 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012753    2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 06 14:19:38.230538 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012755    2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 06 14:19:38.230538 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012758    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 06 14:19:38.230538 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012760    2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 06 14:19:38.230538 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012763    2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 06 14:19:38.230538 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012766    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 06 14:19:38.230538 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012768    2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 06 14:19:38.230538 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012771    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 06 14:19:38.230538 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012773    2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 06 14:19:38.230538 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012776    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 06 14:19:38.230538 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012778    2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 06 14:19:38.230538 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012781    2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 06 14:19:38.230538 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012783    2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 06 14:19:38.230538 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012792    2559 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 06 14:19:38.230538 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012796    2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 06 14:19:38.230538 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012799    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 06 14:19:38.230538 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012801    2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 06 14:19:38.230538 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012804    2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 06 14:19:38.230538 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012806    2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 06 14:19:38.230538 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012809    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 06 14:19:38.231299 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012812    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 06 14:19:38.231299 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012814    2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 06 14:19:38.231299 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012817    2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 06 14:19:38.231299 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012820    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 06 14:19:38.231299 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012824    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 06 14:19:38.231299 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012827    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 06 14:19:38.231299 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012831    2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 06 14:19:38.231299 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012835    2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 06 14:19:38.231299 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012838    2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 06 14:19:38.231299 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012840    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 06 14:19:38.231299 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012843    2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 06 14:19:38.231299 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012845    2559 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 06 14:19:38.231299 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012848    2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 06 14:19:38.231299 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012850    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 06 14:19:38.231299 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012852    2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 06 14:19:38.231299 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012855    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 06 14:19:38.231299 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012857    2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 06 14:19:38.231299 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012860    2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 06 14:19:38.231299 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012862    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 06 14:19:38.233917 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012865    2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 06 14:19:38.233917 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012867    2559 feature_gate.go:328] unrecognized feature gate: Example
Jul 06 14:19:38.233917 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012870    2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 06 14:19:38.233917 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012872    2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 06 14:19:38.233917 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012875    2559 feature_gate.go:328] unrecognized feature gate: Example2
Jul 06 14:19:38.233917 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012877    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 06 14:19:38.233917 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012880    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 06 14:19:38.233917 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012889    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 06 14:19:38.233917 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012892    2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 06 14:19:38.233917 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012894    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 06 14:19:38.233917 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012897    2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 06 14:19:38.233917 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012899    2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 06 14:19:38.233917 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012902    2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 06 14:19:38.233917 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012904    2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 06 14:19:38.233917 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012907    2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 06 14:19:38.233917 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012909    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 06 14:19:38.233917 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012912    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 06 14:19:38.233917 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012917    2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 06 14:19:38.233917 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012919    2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 06 14:19:38.233917 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012922    2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 06 14:19:38.234942 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012925    2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 06 14:19:38.234942 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012928    2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 06 14:19:38.234942 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012930    2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 06 14:19:38.234942 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012933    2559 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 06 14:19:38.234942 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012935    2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 06 14:19:38.234942 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012943    2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 06 14:19:38.234942 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012946    2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 06 14:19:38.234942 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012948    2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 06 14:19:38.234942 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012951    2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 06 14:19:38.234942 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012955    2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 06 14:19:38.234942 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012959    2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 06 14:19:38.234942 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012962    2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 06 14:19:38.234942 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012965    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 06 14:19:38.234942 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012968    2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 06 14:19:38.234942 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012971    2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 06 14:19:38.234942 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.012974    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 06 14:19:38.234942 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.013053    2559 flags.go:64] FLAG: --address="0.0.0.0"
Jul 06 14:19:38.234942 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.013060    2559 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 06 14:19:38.234942 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.013074    2559 flags.go:64] FLAG: --anonymous-auth="true"
Jul 06 14:19:38.234942 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.013081    2559 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 06 14:19:38.237254 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.013095    2559 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 06 14:19:38.237254 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.013101    2559 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 06 14:19:38.237254 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.013109    2559 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 06 14:19:38.237254 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.013117    2559 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 06 14:19:38.237254 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.013122    2559 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 06 14:19:38.237254 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.013127    2559 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 06 14:19:38.237254 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.013130    2559 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 06 14:19:38.237254 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.013133    2559 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 06 14:19:38.237254 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.013137    2559 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 06 14:19:38.237254 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.013141    2559 flags.go:64] FLAG: --cgroup-root=""
Jul 06 14:19:38.237254 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.013144    2559 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 06 14:19:38.237254 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.013147    2559 flags.go:64] FLAG: --client-ca-file=""
Jul 06 14:19:38.237254 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.013150    2559 flags.go:64] FLAG: --cloud-provider="external"
Jul 06 14:19:38.237254 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.013153    2559 flags.go:64] FLAG: --cluster-dns="[]"
Jul 06 14:19:38.237254 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.013161    2559 flags.go:64] FLAG: --cluster-domain=""
Jul 06 14:19:38.237254 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.013164    2559 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 06 14:19:38.237254 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.013167    2559 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 06 14:19:38.237254 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.013170    2559 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 06 14:19:38.237254 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.013173    2559 flags.go:64] FLAG: --container-log-max-files="5"
Jul 06 14:19:38.237254 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.013177    2559 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 06 14:19:38.237254 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.013180    2559 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 06 14:19:38.237254 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.013183    2559 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 06 14:19:38.237254 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.013187    2559 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 06 14:19:38.238090 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.013190    2559 flags.go:64] FLAG: --contention-profiling="false"
Jul 06 14:19:38.238090 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.013193    2559 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 06 14:19:38.238090 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.013195    2559 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 06 14:19:38.238090 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.013199    2559 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 06 14:19:38.238090 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.013202    2559 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 06 14:19:38.238090 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.013206    2559 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 06 14:19:38.238090 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.013209    2559 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 06 14:19:38.238090 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.013212    2559 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 06 14:19:38.238090 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.013216    2559 flags.go:64] FLAG: --enable-load-reader="false"
Jul 06 14:19:38.238090 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.013219    2559 flags.go:64] FLAG: --enable-server="true"
Jul 06 14:19:38.238090 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.013228    2559 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 06 14:19:38.238090 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.013235    2559 flags.go:64] FLAG: --event-burst="100"
Jul 06 14:19:38.238090 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.013238    2559 flags.go:64] FLAG: --event-qps="50"
Jul 06 14:19:38.238090 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.013241    2559 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 06 14:19:38.238090 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.013244    2559 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 06 14:19:38.238090 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.013247    2559 flags.go:64] FLAG: --eviction-hard=""
Jul 06 14:19:38.238090 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.013251    2559 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 06 14:19:38.238090 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.013254    2559 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 06 14:19:38.238090 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.013258    2559 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 06 14:19:38.238090 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.013261    2559 flags.go:64] FLAG: --eviction-soft=""
Jul 06 14:19:38.238090 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.013264    2559 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 06 14:19:38.238090 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.013267    2559 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 06 14:19:38.238090 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.013270    2559 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 06 14:19:38.238090 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.013273    2559 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 06 14:19:38.238090 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.013276    2559 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 06 14:19:38.240238 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.013278    2559 flags.go:64] FLAG: --fail-swap-on="true"
Jul 06 14:19:38.240238 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.013281    2559 flags.go:64] FLAG: --feature-gates=""
Jul 06 14:19:38.240238 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.013285    2559 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 06 14:19:38.240238 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.013288    2559 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 06 14:19:38.240238 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.013291    2559 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 06 14:19:38.240238 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.013294    2559 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 06 14:19:38.240238 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.013297    2559 flags.go:64] FLAG: --healthz-port="10248"
Jul 06 14:19:38.240238 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.013300    2559 flags.go:64] FLAG: --help="false"
Jul 06 14:19:38.240238 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.013303    2559 flags.go:64] FLAG: --hostname-override="ip-10-0-133-112.ec2.internal"
Jul 06 14:19:38.240238 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.013306    2559 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 06 14:19:38.240238 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.013309    2559 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 06 14:19:38.240238 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.013312    2559 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 06 14:19:38.240238 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.013315    2559 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 06 14:19:38.240238 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.013319    2559 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 06 14:19:38.240238 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.013321    2559 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 06 14:19:38.240238 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.013326    2559 flags.go:64] FLAG: --image-service-endpoint=""
Jul 06 14:19:38.240238 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.013329    2559 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 06 14:19:38.240238 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.013332    2559 flags.go:64] FLAG: --kube-api-burst="100"
Jul 06 14:19:38.240238 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.013340    2559 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 06 14:19:38.240238 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.013344    2559 flags.go:64] FLAG: --kube-api-qps="50"
Jul 06 14:19:38.240238 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.013347    2559 flags.go:64] FLAG: --kube-reserved=""
Jul 06 14:19:38.240238 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.013349    2559 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 06 14:19:38.240238 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.013352    2559 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 06 14:19:38.240238 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.013355    2559 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 06 14:19:38.240936 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.013358    2559 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 06 14:19:38.240936 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.013362    2559 flags.go:64] FLAG: --lock-file=""
Jul 06 14:19:38.240936 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.013365    2559 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 06 14:19:38.240936 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.013368    2559 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 06 14:19:38.240936 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.013371    2559 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 06 14:19:38.240936 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.013376    2559 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 06 14:19:38.240936 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.013379    2559 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 06 14:19:38.240936 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.013381    2559 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 06 14:19:38.240936 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.013384    2559 flags.go:64] FLAG: --logging-format="text"
Jul 06 14:19:38.240936 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.013387    2559 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 06 14:19:38.240936 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.013390    2559 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 06 14:19:38.240936 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.013393    2559 flags.go:64] FLAG: --manifest-url=""
Jul 06 14:19:38.240936 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.013396    2559 flags.go:64] FLAG: --manifest-url-header=""
Jul 06 14:19:38.240936 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.013400    2559 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 06 14:19:38.240936 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.013403    2559 flags.go:64] FLAG: --max-open-files="1000000"
Jul 06 14:19:38.240936 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.013408    2559 flags.go:64] FLAG: --max-pods="110"
Jul 06 14:19:38.240936 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.013410    2559 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 06 14:19:38.240936 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.013413    2559 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 06 14:19:38.240936 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.013416    2559 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 06 14:19:38.240936 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.013419    2559 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 06 14:19:38.240936 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.013422    2559 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 06 14:19:38.240936 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.013425    2559 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 06 14:19:38.240936 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.013428    2559 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 06 14:19:38.240936 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.013437    2559 flags.go:64] FLAG: --node-status-max-images="50"
Jul 06 14:19:38.240936 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.013440    2559 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 06 14:19:38.241893 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.013443    2559 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 06 14:19:38.241893 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.013446    2559 flags.go:64] FLAG: --pod-cidr=""
Jul 06 14:19:38.241893 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.013455    2559 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5"
Jul 06 14:19:38.241893 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.013461    2559 flags.go:64] FLAG: --pod-manifest-path=""
Jul 06 14:19:38.241893 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.013464    2559 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 06 14:19:38.241893 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.013467    2559 flags.go:64] FLAG: --pods-per-core="0"
Jul 06 14:19:38.241893 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.013470    2559 flags.go:64] FLAG: --port="10250"
Jul 06 14:19:38.241893 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.013472    2559 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 06 14:19:38.241893 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.013477    2559 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0ccbd7f630aa804bf"
Jul 06 14:19:38.241893 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.013480    2559 flags.go:64] FLAG: --qos-reserved=""
Jul 06 14:19:38.241893 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.013483    2559 flags.go:64] FLAG: --read-only-port="10255"
Jul 06 14:19:38.241893 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.013486    2559 flags.go:64] FLAG: --register-node="true"
Jul 06 14:19:38.241893 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.013489    2559 flags.go:64] FLAG: --register-with-taints=""
Jul 06 14:19:38.241893 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.013492    2559 flags.go:64] FLAG: --registry-burst="10"
Jul 06 14:19:38.241893 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.013495    2559 flags.go:64] FLAG: --registry-qps="5"
Jul 06 14:19:38.241893 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.013498    2559 flags.go:64] FLAG: --reserved-cpus=""
Jul 06 14:19:38.241893 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.013501    2559 flags.go:64] FLAG: --reserved-memory=""
Jul 06 14:19:38.241893 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.013505    2559 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 06 14:19:38.241893 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.013507    2559 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 06 14:19:38.241893 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.013510    2559 flags.go:64] FLAG: --rotate-certificates="false"
Jul 06 14:19:38.241893 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.013513    2559 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 06 14:19:38.241893 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.013516    2559 flags.go:64] FLAG: --runonce="false"
Jul 06 14:19:38.241893 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.013519    2559 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 06 14:19:38.241893 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.013522    2559 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 06 14:19:38.241893 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.013525    2559 flags.go:64] FLAG: --seccomp-default="false"
Jul 06 14:19:38.242613 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.013528    2559 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 06 14:19:38.242613 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.013531    2559 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 06 14:19:38.242613 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.013534    2559 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 06 14:19:38.242613 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.013537    2559 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 06 14:19:38.242613 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.013540    2559 flags.go:64] FLAG: --storage-driver-password="root"
Jul 06 14:19:38.242613 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.013542    2559 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 06 14:19:38.242613 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.013548    2559 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 06 14:19:38.242613 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.013551    2559 flags.go:64] FLAG: --storage-driver-user="root"
Jul 06 14:19:38.242613 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.013554    2559 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 06 14:19:38.242613 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.013558    2559 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 06 14:19:38.242613 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.013562    2559 flags.go:64] FLAG: --system-cgroups=""
Jul 06 14:19:38.242613 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.013566    2559 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 06 14:19:38.242613 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.013591    2559 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 06 14:19:38.242613 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.013594    2559 flags.go:64] FLAG: --tls-cert-file=""
Jul 06 14:19:38.242613 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.013597    2559 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 06 14:19:38.242613 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.013605    2559 flags.go:64] FLAG: --tls-min-version=""
Jul 06 14:19:38.242613 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.013608    2559 flags.go:64] FLAG: --tls-private-key-file=""
Jul 06 14:19:38.242613 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.013610    2559 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 06 14:19:38.242613 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.013613    2559 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 06 14:19:38.242613 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.013616    2559 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 06 14:19:38.242613 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.013619    2559 flags.go:64] FLAG: --v="2"
Jul 06 14:19:38.242613 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.013623    2559 flags.go:64] FLAG: --version="false"
Jul 06 14:19:38.242613 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.013627    2559 flags.go:64] FLAG: --vmodule=""
Jul 06 14:19:38.242613 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.013631    2559 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 06 14:19:38.242613 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.013634    2559 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 06 14:19:38.354973 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.013771    2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 06 14:19:38.354973 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.013775    2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 06 14:19:38.354973 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.013778    2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 06 14:19:38.354973 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.013781    2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 06 14:19:38.354973 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.013785    2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 06 14:19:38.354973 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.013787    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 06 14:19:38.354973 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.013790    2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 06 14:19:38.354973 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.013793    2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 06 14:19:38.354973 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.013795    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 06 14:19:38.354973 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.013798    2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 06 14:19:38.354973 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.013800    2559 feature_gate.go:328] unrecognized feature gate: Example
Jul 06 14:19:38.354973 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.013803    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 06 14:19:38.354973 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.013807    2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 06 14:19:38.354973 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.013812    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 06 14:19:38.354973 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.013815    2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 06 14:19:38.354973 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.013818    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 06 14:19:38.354973 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.013820    2559 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 06 14:19:38.354973 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.013823    2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 06 14:19:38.354973 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.013829    2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 06 14:19:38.355810 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.013832    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 06 14:19:38.355810 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.013835    2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 06 14:19:38.355810 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.013837    2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 06 14:19:38.355810 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.013841    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 06 14:19:38.355810 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.013844    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 06 14:19:38.355810 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.013847    2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 06 14:19:38.355810 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.013849    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 06 14:19:38.355810 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.013852    2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 06 14:19:38.355810 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.013855    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 06 14:19:38.355810 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.013857    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 06 14:19:38.355810 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.013860    2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 06 14:19:38.355810 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.013863    2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 06 14:19:38.355810 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.013865    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 06 14:19:38.355810 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.013868    2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 06 14:19:38.355810 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.013870    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 06 14:19:38.355810 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.013873    2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 06 14:19:38.355810 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.013877    2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 06 14:19:38.355810 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.013880    2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 06 14:19:38.355810 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.013883    2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 06 14:19:38.358155 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.013886    2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 06 14:19:38.358155 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.013889    2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 06 14:19:38.358155 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.013892    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 06 14:19:38.358155 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.013894    2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 06 14:19:38.358155 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.013897    2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 06 14:19:38.358155 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.013900    2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 06 14:19:38.358155 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.013902    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 06 14:19:38.358155 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.013906    2559 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 06 14:19:38.358155 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.013908    2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 06 14:19:38.358155 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.013911    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 06 14:19:38.358155 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.013914    2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 06 14:19:38.358155 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.013916    2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 06 14:19:38.358155 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.013919    2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 06 14:19:38.358155 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.013922    2559 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 06 14:19:38.358155 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.013925    2559 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 06 14:19:38.358155 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.013927    2559 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 06 14:19:38.358155 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.013931    2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 06 14:19:38.358155 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.013934    2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 06 14:19:38.358155 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.013936    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 06 14:19:38.358155 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.013939    2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 06 14:19:38.358792 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.013941    2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 06 14:19:38.358792 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.013944    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 06 14:19:38.358792 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.013946    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 06 14:19:38.358792 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.013949    2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 06 14:19:38.358792 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.013951    2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 06 14:19:38.358792 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.013954    2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 06 14:19:38.358792 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.013956    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 06 14:19:38.358792 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.013959    2559 feature_gate.go:328] unrecognized feature gate: Example2
Jul 06 14:19:38.358792 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.013961    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 06 14:19:38.358792 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.013964    2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 06 14:19:38.358792 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.013966    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 06 14:19:38.358792 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.013969    2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 06 14:19:38.358792 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.013972    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 06 14:19:38.358792 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.013974    2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 06 14:19:38.358792 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.013977    2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 06 14:19:38.358792 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.013980    2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 06 14:19:38.358792 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.013982    2559 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 06 14:19:38.358792 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.013985    2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 06 14:19:38.358792 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.013987    2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 06 14:19:38.360497 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.013991    2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 06 14:19:38.360497 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.014434    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 06 14:19:38.360497 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.014439    2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 06 14:19:38.360497 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.014442    2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 06 14:19:38.360497 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.014445    2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 06 14:19:38.360497 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.014448    2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 06 14:19:38.360497 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.014450    2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 06 14:19:38.360497 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.014454    2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 06 14:19:38.360497 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.014457    2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 06 14:19:38.360497 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.014461    2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 06 14:19:38.360497 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.014465    2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 06 14:19:38.360497 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.014469    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 06 14:19:38.360497 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.014472    2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 06 14:19:38.360497 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.014475    2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 06 14:19:38.360497 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.014478    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 06 14:19:38.360497 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.014481    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 06 14:19:38.360497 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.014483    2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 06 14:19:38.360497 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.014486    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 06 14:19:38.360497 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.014488    2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 06 14:19:38.361133 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.014491    2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 06 14:19:38.361133 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.014494    2559 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 06 14:19:38.361133 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.014496    2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 06 14:19:38.361133 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.015016    2559 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 06 14:19:38.361133 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.022855    2559 server.go:529] "Kubelet version" kubeletVersion="v1.34.8"
Jul 06 14:19:38.361133 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.022872    2559 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 06 14:19:38.361133 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.022890    2559 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 06 14:19:38.361133 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.022897    2559 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 06 14:19:38.361133 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.022962    2559 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 06 14:19:38.361133 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.022969    2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 06 14:19:38.361133 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.022972    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 06 14:19:38.361133 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.022976    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 06 14:19:38.361133 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.022979    2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 06 14:19:38.361133 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.022982    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 06 14:19:38.361133 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.022985    2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 06 14:19:38.363678 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.022988    2559 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 06 14:19:38.363678 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.022990    2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 06 14:19:38.363678 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.022993    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 06 14:19:38.363678 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.022995    2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 06 14:19:38.363678 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023000    2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 06 14:19:38.363678 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023003    2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 06 14:19:38.363678 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023007    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 06 14:19:38.363678 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023010    2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 06 14:19:38.363678 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023013    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 06 14:19:38.363678 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023016    2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 06 14:19:38.363678 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023018    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 06 14:19:38.363678 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023021    2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 06 14:19:38.363678 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023025    2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 06 14:19:38.363678 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023028    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 06 14:19:38.363678 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023031    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 06 14:19:38.363678 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023034    2559 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 06 14:19:38.363678 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023037    2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 06 14:19:38.363678 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023043    2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 06 14:19:38.363678 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023046    2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 06 14:19:38.363678 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023048    2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 06 14:19:38.364287 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023051    2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 06 14:19:38.364287 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023053    2559 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 06 14:19:38.364287 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023059    2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 06 14:19:38.364287 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023062    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 06 14:19:38.364287 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023065    2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 06 14:19:38.364287 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023068    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 06 14:19:38.364287 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023071    2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 06 14:19:38.364287 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023073    2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 06 14:19:38.364287 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023076    2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 06 14:19:38.364287 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023079    2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 06 14:19:38.364287 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023081    2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 06 14:19:38.364287 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023084    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 06 14:19:38.364287 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023087    2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 06 14:19:38.364287 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023089    2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 06 14:19:38.364287 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023092    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 06 14:19:38.364287 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023095    2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 06 14:19:38.364287 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023097    2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 06 14:19:38.364287 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023101    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 06 14:19:38.364287 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023103    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 06 14:19:38.364287 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023106    2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 06 14:19:38.365148 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023109    2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 06 14:19:38.365148 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023111    2559 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 06 14:19:38.365148 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023114    2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 06 14:19:38.365148 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023116    2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 06 14:19:38.365148 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023120    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 06 14:19:38.365148 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023123    2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 06 14:19:38.365148 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023126    2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 06 14:19:38.365148 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023128    2559 feature_gate.go:328] unrecognized feature gate: Example2
Jul 06 14:19:38.365148 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023131    2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 06 14:19:38.365148 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023135    2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 06 14:19:38.365148 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023138    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 06 14:19:38.365148 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023141    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 06 14:19:38.365148 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023143    2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 06 14:19:38.365148 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023147    2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 06 14:19:38.365148 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023150    2559 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 06 14:19:38.365148 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023154    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 06 14:19:38.365148 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023157    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 06 14:19:38.365148 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023160    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 06 14:19:38.365148 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023162    2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 06 14:19:38.365148 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023165    2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 06 14:19:38.365728 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023168    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 06 14:19:38.365728 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023171    2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 06 14:19:38.365728 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023174    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 06 14:19:38.365728 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023177    2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 06 14:19:38.365728 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023180    2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 06 14:19:38.365728 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023182    2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 06 14:19:38.365728 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023185    2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 06 14:19:38.365728 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023188    2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 06 14:19:38.365728 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023190    2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 06 14:19:38.365728 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023193    2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 06 14:19:38.365728 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023195    2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 06 14:19:38.365728 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023198    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 06 14:19:38.365728 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023201    2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 06 14:19:38.365728 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023203    2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 06 14:19:38.365728 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023206    2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 06 14:19:38.365728 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023208    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 06 14:19:38.365728 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023214    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 06 14:19:38.365728 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023217    2559 feature_gate.go:328] unrecognized feature gate: Example
Jul 06 14:19:38.365728 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023220    2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 06 14:19:38.366270 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023222    2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 06 14:19:38.366270 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023225    2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 06 14:19:38.366270 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023229    2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 06 14:19:38.366270 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023232    2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 06 14:19:38.366270 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023235    2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 06 14:19:38.366270 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023237    2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 06 14:19:38.366270 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023240    2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 06 14:19:38.366270 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023243    2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 06 14:19:38.366270 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023245    2559 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 06 14:19:38.366270 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023249    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 06 14:19:38.366270 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023251    2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 06 14:19:38.366270 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023254    2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 06 14:19:38.366270 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023257    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 06 14:19:38.366270 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.023262    2559 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 06 14:19:38.366972 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023379    2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 06 14:19:38.366972 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023384    2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 06 14:19:38.366972 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023388    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 06 14:19:38.366972 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023391    2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 06 14:19:38.366972 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023394    2559 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 06 14:19:38.366972 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023396    2559 feature_gate.go:328] unrecognized feature gate: Example2
Jul 06 14:19:38.366972 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023399    2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 06 14:19:38.366972 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023402    2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 06 14:19:38.366972 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023404    2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 06 14:19:38.366972 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023407    2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 06 14:19:38.366972 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023410    2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 06 14:19:38.366972 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023412    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 06 14:19:38.366972 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023415    2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 06 14:19:38.366972 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023417    2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 06 14:19:38.366972 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023420    2559 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 06 14:19:38.366972 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023424    2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 06 14:19:38.366972 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023427    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 06 14:19:38.366972 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023429    2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 06 14:19:38.366972 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023432    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 06 14:19:38.366972 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023434    2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 06 14:19:38.367892 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023437    2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 06 14:19:38.367892 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023443    2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 06 14:19:38.367892 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023446    2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 06 14:19:38.367892 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023448    2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 06 14:19:38.367892 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023451    2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 06 14:19:38.367892 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023454    2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 06 14:19:38.367892 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023456    2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 06 14:19:38.367892 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023459    2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 06 14:19:38.367892 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023462    2559 feature_gate.go:328] unrecognized feature gate: Example
Jul 06 14:19:38.367892 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023465    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 06 14:19:38.367892 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023467    2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 06 14:19:38.367892 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023470    2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 06 14:19:38.367892 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023473    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 06 14:19:38.367892 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023475    2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 06 14:19:38.367892 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023478    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 06 14:19:38.367892 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023480    2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 06 14:19:38.367892 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023483    2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 06 14:19:38.367892 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023485    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 06 14:19:38.367892 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023488    2559 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 06 14:19:38.367892 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023490    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 06 14:19:38.368537 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023493    2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 06 14:19:38.368537 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023495    2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 06 14:19:38.368537 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023498    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 06 14:19:38.368537 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023502    2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 06 14:19:38.368537 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023505    2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 06 14:19:38.368537 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023508    2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 06 14:19:38.368537 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023511    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 06 14:19:38.368537 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023515    2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 06 14:19:38.368537 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023518    2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 06 14:19:38.368537 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023521    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 06 14:19:38.368537 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023524    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 06 14:19:38.368537 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023526    2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 06 14:19:38.368537 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023529    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 06 14:19:38.368537 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023533    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 06 14:19:38.368537 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023535    2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 06 14:19:38.368537 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023538    2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 06 14:19:38.368537 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023540    2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 06 14:19:38.368537 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023543    2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 06 14:19:38.368537 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023546    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 06 14:19:38.469143 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023549    2559 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 06 14:19:38.469143 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023551    2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 06 14:19:38.469143 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023555    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 06 14:19:38.469143 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023558    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 06 14:19:38.469143 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023560    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 06 14:19:38.469143 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023563    2559 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 06 14:19:38.469143 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023565    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 06 14:19:38.469143 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023583    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 06 14:19:38.469143 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023588    2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 06 14:19:38.469143 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023591    2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 06 14:19:38.469143 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023593    2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 06 14:19:38.469143 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023596    2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 06 14:19:38.469143 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023598    2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 06 14:19:38.469143 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023601    2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 06 14:19:38.469143 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023604    2559 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 06 14:19:38.469143 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023607    2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 06 14:19:38.469143 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023609    2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 06 14:19:38.469143 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023612    2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 06 14:19:38.469143 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023614    2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 06 14:19:38.469143 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023617    2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 06 14:19:38.470027 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023621    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 06 14:19:38.470027 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023624    2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 06 14:19:38.470027 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023627    2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 06 14:19:38.470027 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023629    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 06 14:19:38.470027 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023632    2559 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 06 14:19:38.470027 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023635    2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 06 14:19:38.470027 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023639    2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 06 14:19:38.470027 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023642    2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 06 14:19:38.470027 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023645    2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 06 14:19:38.470027 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023647    2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 06 14:19:38.470027 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023650    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 06 14:19:38.470027 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023652    2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 06 14:19:38.470027 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023655    2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 06 14:19:38.470027 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023658    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 06 14:19:38.470027 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023661    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 06 14:19:38.470027 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023664    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 06 14:19:38.470027 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023667    2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 06 14:19:38.470027 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023669    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 06 14:19:38.470027 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023672    2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 06 14:19:38.472361 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:38.023675    2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 06 14:19:38.472361 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.023681    2559 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 06 14:19:38.472361 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.023824    2559 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 06 14:19:38.472361 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.025881    2559 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 06 14:19:38.472361 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.027230    2559 server.go:1019] "Starting client certificate rotation"
Jul 06 14:19:38.472361 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.027331    2559 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 06 14:19:38.472361 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.027370    2559 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 06 14:19:38.472361 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.032752    2559 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 06 14:19:38.472361 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.036049    2559 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 06 14:19:38.472361 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.046638    2559 log.go:25] "Validated CRI v1 runtime API"
Jul 06 14:19:38.472361 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.048711    2559 log.go:25] "Validated CRI v1 image API"
Jul 06 14:19:38.472361 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.050306    2559 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 06 14:19:38.472753 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.052790    2559 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 aaf43666-5fb9-4d1b-971f-81ddb43723d5:/dev/nvme0n1p3 ea409685-b088-4235-803b-4cdec6ca137d:/dev/nvme0n1p4]
Jul 06 14:19:38.472753 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.052807    2559 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 06 14:19:38.472753 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.056753    2559 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 06 14:19:38.473153 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.057069    2559 manager.go:216] Machine: {Timestamp:2026-07-06 14:19:38.056550596 +0000 UTC m=+0.363842671 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099778 MemoryCapacity:33164492800 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec211823af71afed344b1c458b225755 SystemUUID:ec211823-af71-afed-344b-1c458b225755 BootID:57b7aeec-c624-443d-851c-ceed3228af6e Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:34:e5:ce:c6:fb Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:34:e5:ce:c6:fb Speed:0 Mtu:9001} {Name:ovs-system MacAddress:3e:5a:48:4f:d9:0b Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164492800 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 06 14:19:38.473153 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.057177    2559 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 06 14:19:38.473153 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.057260    2559 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 06 14:19:38.473153 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.061709    2559 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 06 14:19:38.473153 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.061739    2559 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-133-112.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 06 14:19:38.473153 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.062445    2559 topology_manager.go:138] "Creating topology manager with none policy"
Jul 06 14:19:38.473153 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.062456    2559 container_manager_linux.go:309] "Creating device plugin manager"
Jul 06 14:19:38.473153 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.062464    2559 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 06 14:19:38.473153 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.062477    2559 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 06 14:19:38.473153 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.064147    2559 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 06 14:19:38.473153 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.068267    2559 state_mem.go:36] "Initialized new in-memory state store"
Jul 06 14:19:38.473153 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.068402    2559 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 06 14:19:38.473153 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.068433    2559 kubelet.go:486] "Attempting to sync node with API server"
Jul 06 14:19:38.473153 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.068443    2559 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 06 14:19:38.473153 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.068456    2559 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 06 14:19:38.473153 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.068469    2559 kubelet.go:398] "Adding apiserver pod source"
Jul 06 14:19:38.473153 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.068477    2559 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 06 14:19:38.473153 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.068891    2559 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 06 14:19:38.567303 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.068906    2559 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 06 14:19:38.567303 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.072843    2559 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1"
Jul 06 14:19:38.567303 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.075165    2559 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 06 14:19:38.567303 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.075197    2559 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 06 14:19:38.567303 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.076181    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 06 14:19:38.567303 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.076193    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 06 14:19:38.567303 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.076199    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 06 14:19:38.567303 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.076205    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 06 14:19:38.567303 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.076214    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 06 14:19:38.567303 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.076219    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 06 14:19:38.567303 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.076229    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 06 14:19:38.567303 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.076234    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 06 14:19:38.567303 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.076241    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 06 14:19:38.567303 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.076247    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 06 14:19:38.567303 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.076255    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 06 14:19:38.567303 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.076261    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 06 14:19:38.567303 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.076935    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 06 14:19:38.567303 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.076943    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 06 14:19:38.567303 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.080106    2559 server.go:1268] "Started kubelet"
Jul 06 14:19:38.567303 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.080194    2559 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 06 14:19:38.567303 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.080219    2559 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 06 14:19:38.567303 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.080307    2559 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 06 14:19:38.567303 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.081525    2559 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 06 14:19:38.567303 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.081591    2559 server.go:310] "Adding debug handlers to kubelet server"
Jul 06 14:19:38.567303 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.082106    2559 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-133-112.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 06 14:19:38.567303 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:19:38.082106    2559 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-133-112.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 06 14:19:38.567303 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:19:38.082218    2559 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 06 14:19:38.567303 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:19:38.085497    2559 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 06 14:19:38.567303 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:19:38.087047    2559 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-112.ec2.internal.18bfb92c76842e55  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-112.ec2.internal,UID:ip-10-0-133-112.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-133-112.ec2.internal,},FirstTimestamp:2026-07-06 14:19:38.080079445 +0000 UTC m=+0.387371518,LastTimestamp:2026-07-06 14:19:38.080079445 +0000 UTC m=+0.387371518,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-112.ec2.internal,}"
Jul 06 14:19:38.567303 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.087942    2559 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 06 14:19:38.567303 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.087946    2559 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 06 14:19:38.567303 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:19:38.087993    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-112.ec2.internal\" not found"
Jul 06 14:19:38.567303 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.088462    2559 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 06 14:19:38.567303 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.088465    2559 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 06 14:19:38.567303 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.088487    2559 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 06 14:19:38.567303 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.088636    2559 reconstruct.go:97] "Volume reconstruction finished"
Jul 06 14:19:38.567303 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.088645    2559 reconciler.go:29] "Reconciler: start to sync state"
Jul 06 14:19:38.567303 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.088846    2559 factory.go:55] Registering systemd factory
Jul 06 14:19:38.567303 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.088863    2559 factory.go:223] Registration of the systemd container factory successfully
Jul 06 14:19:38.567303 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.089094    2559 factory.go:153] Registering CRI-O factory
Jul 06 14:19:38.567303 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.089108    2559 factory.go:223] Registration of the crio container factory successfully
Jul 06 14:19:38.568490 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.089159    2559 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 06 14:19:38.568490 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.089185    2559 factory.go:103] Registering Raw factory
Jul 06 14:19:38.568490 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.089200    2559 manager.go:1196] Started watching for new ooms in manager
Jul 06 14:19:38.568490 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.089720    2559 manager.go:349] Starting recovery of all containers
Jul 06 14:19:38.568490 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:19:38.095352    2559 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-133-112.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 06 14:19:38.568490 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:19:38.095487    2559 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 06 14:19:38.568490 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.100908    2559 manager.go:354] Recovery completed
Jul 06 14:19:38.568490 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.107052    2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 06 14:19:38.568490 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.109775    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-112.ec2.internal" event="NodeHasSufficientMemory"
Jul 06 14:19:38.568490 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.109804    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-112.ec2.internal" event="NodeHasNoDiskPressure"
Jul 06 14:19:38.568490 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.109814    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-112.ec2.internal" event="NodeHasSufficientPID"
Jul 06 14:19:38.568490 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.110319    2559 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 06 14:19:38.568490 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.110330    2559 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 06 14:19:38.568490 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.110346    2559 state_mem.go:36] "Initialized new in-memory state store"
Jul 06 14:19:38.568490 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.110389    2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 06 14:19:38.568490 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.111122    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-112.ec2.internal" event="NodeHasSufficientMemory"
Jul 06 14:19:38.568490 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.111147    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-112.ec2.internal" event="NodeHasNoDiskPressure"
Jul 06 14:19:38.568490 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.111160    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-112.ec2.internal" event="NodeHasSufficientPID"
Jul 06 14:19:38.568490 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:19:38.111898    2559 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-112.ec2.internal.18bfb92c7849862a  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-112.ec2.internal,UID:ip-10-0-133-112.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-133-112.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-133-112.ec2.internal,},FirstTimestamp:2026-07-06 14:19:38.109789738 +0000 UTC m=+0.417081814,LastTimestamp:2026-07-06 14:19:38.109789738 +0000 UTC m=+0.417081814,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-112.ec2.internal,}"
Jul 06 14:19:38.568490 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.112522    2559 policy_none.go:49] "None policy: Start"
Jul 06 14:19:38.568490 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.112539    2559 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 06 14:19:38.568490 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.112549    2559 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 06 14:19:38.568490 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.114508    2559 policy_none.go:47] "Start"
Jul 06 14:19:38.568490 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.119485    2559 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-frwb8"
Jul 06 14:19:38.568490 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:19:38.121030    2559 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-112.ec2.internal.18bfb92c7849d221  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-112.ec2.internal,UID:ip-10-0-133-112.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-133-112.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-133-112.ec2.internal,},FirstTimestamp:2026-07-06 14:19:38.109809185 +0000 UTC m=+0.417101257,LastTimestamp:2026-07-06 14:19:38.109809185 +0000 UTC m=+0.417101257,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-112.ec2.internal,}"
Jul 06 14:19:38.568490 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.129121    2559 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-frwb8"
Jul 06 14:19:38.569212 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:19:38.130816    2559 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-112.ec2.internal.18bfb92c7849f54d  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-112.ec2.internal,UID:ip-10-0-133-112.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-133-112.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-133-112.ec2.internal,},FirstTimestamp:2026-07-06 14:19:38.109818189 +0000 UTC m=+0.417110263,LastTimestamp:2026-07-06 14:19:38.109818189 +0000 UTC m=+0.417110263,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-112.ec2.internal,}"
Jul 06 14:19:38.569212 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:19:38.189091    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-112.ec2.internal\" not found"
Jul 06 14:19:38.569212 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.218887    2559 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 06 14:19:38.569212 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.220384    2559 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 06 14:19:38.569212 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.220426    2559 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 06 14:19:38.569212 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.220456    2559 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 06 14:19:38.569212 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:19:38.220495    2559 kubelet.go:2472] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]"
Jul 06 14:19:38.569212 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.230240    2559 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 06 14:19:38.569212 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.231982    2559 manager.go:337] "Starting Device Plugin manager"
Jul 06 14:19:38.569212 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:19:38.232017    2559 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 06 14:19:38.569212 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.232029    2559 server.go:87] "Starting device plugin registration server"
Jul 06 14:19:38.569212 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.232313    2559 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 06 14:19:38.569212 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.232324    2559 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 06 14:19:38.569212 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.232677    2559 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 06 14:19:38.569212 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.232745    2559 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 06 14:19:38.569212 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.232753    2559 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 06 14:19:38.569212 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:19:38.233242    2559 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 06 14:19:38.569212 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:19:38.233285    2559 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-133-112.ec2.internal\" not found"
Jul 06 14:19:38.569212 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:19:38.309686    2559 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-133-112.ec2.internal\" not found" node="ip-10-0-133-112.ec2.internal"
Jul 06 14:19:38.569212 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.325772    2559 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-112.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-133-112.ec2.internal"]
Jul 06 14:19:38.569212 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.325847    2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 06 14:19:38.569212 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.326808    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-112.ec2.internal" event="NodeHasSufficientMemory"
Jul 06 14:19:38.569212 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.326836    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-112.ec2.internal" event="NodeHasNoDiskPressure"
Jul 06 14:19:38.569212 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.326848    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-112.ec2.internal" event="NodeHasSufficientPID"
Jul 06 14:19:38.569212 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.329187    2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 06 14:19:38.569212 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.329305    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-112.ec2.internal"
Jul 06 14:19:38.569212 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.329329    2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 06 14:19:38.569212 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.330933    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-112.ec2.internal" event="NodeHasSufficientMemory"
Jul 06 14:19:38.569212 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.330968    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-112.ec2.internal" event="NodeHasNoDiskPressure"
Jul 06 14:19:38.569212 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.330983    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-112.ec2.internal" event="NodeHasSufficientPID"
Jul 06 14:19:38.569212 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.330985    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-112.ec2.internal" event="NodeHasSufficientMemory"
Jul 06 14:19:38.569212 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.331110    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-112.ec2.internal" event="NodeHasNoDiskPressure"
Jul 06 14:19:38.569212 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.331119    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-112.ec2.internal" event="NodeHasSufficientPID"
Jul 06 14:19:38.569212 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.332863    2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 06 14:19:38.569212 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.333149    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-112.ec2.internal"
Jul 06 14:19:38.569212 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.333183    2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 06 14:19:38.570325 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.334544    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-112.ec2.internal" event="NodeHasSufficientMemory"
Jul 06 14:19:38.570325 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.334544    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-112.ec2.internal" event="NodeHasSufficientMemory"
Jul 06 14:19:38.570325 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.334589    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-112.ec2.internal" event="NodeHasNoDiskPressure"
Jul 06 14:19:38.570325 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.334605    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-112.ec2.internal" event="NodeHasSufficientPID"
Jul 06 14:19:38.570325 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.334629    2559 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-133-112.ec2.internal"
Jul 06 14:19:38.570325 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.334604    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-112.ec2.internal" event="NodeHasNoDiskPressure"
Jul 06 14:19:38.570325 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.334752    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-112.ec2.internal" event="NodeHasSufficientPID"
Jul 06 14:19:38.570325 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.345616    2559 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-133-112.ec2.internal"
Jul 06 14:19:38.570325 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:19:38.345640    2559 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-133-112.ec2.internal\": node \"ip-10-0-133-112.ec2.internal\" not found"
Jul 06 14:19:38.570325 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:19:38.382315    2559 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-112.ec2.internal\" not found" node="ip-10-0-133-112.ec2.internal"
Jul 06 14:19:38.570325 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.390227    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/112772d63ce9100ee913123b6eaccc48-config\") pod \"kube-apiserver-proxy-ip-10-0-133-112.ec2.internal\" (UID: \"112772d63ce9100ee913123b6eaccc48\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-112.ec2.internal"
Jul 06 14:19:38.570325 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.390253    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/1ce6bbc5e87ed1b7fdb9af3b5ae20a50-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-112.ec2.internal\" (UID: \"1ce6bbc5e87ed1b7fdb9af3b5ae20a50\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-112.ec2.internal"
Jul 06 14:19:38.570325 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.390270    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1ce6bbc5e87ed1b7fdb9af3b5ae20a50-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-112.ec2.internal\" (UID: \"1ce6bbc5e87ed1b7fdb9af3b5ae20a50\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-112.ec2.internal"
Jul 06 14:19:38.570325 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:19:38.390440    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-112.ec2.internal\" not found"
Jul 06 14:19:38.570325 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:19:38.394055    2559 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-112.ec2.internal\" not found" node="ip-10-0-133-112.ec2.internal"
Jul 06 14:19:38.570325 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.490417    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/1ce6bbc5e87ed1b7fdb9af3b5ae20a50-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-112.ec2.internal\" (UID: \"1ce6bbc5e87ed1b7fdb9af3b5ae20a50\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-112.ec2.internal"
Jul 06 14:19:38.570325 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.490444    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1ce6bbc5e87ed1b7fdb9af3b5ae20a50-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-112.ec2.internal\" (UID: \"1ce6bbc5e87ed1b7fdb9af3b5ae20a50\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-112.ec2.internal"
Jul 06 14:19:38.570325 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.490462    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/112772d63ce9100ee913123b6eaccc48-config\") pod \"kube-apiserver-proxy-ip-10-0-133-112.ec2.internal\" (UID: \"112772d63ce9100ee913123b6eaccc48\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-112.ec2.internal"
Jul 06 14:19:38.570325 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:19:38.490483    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-112.ec2.internal\" not found"
Jul 06 14:19:38.570325 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.490500    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/1ce6bbc5e87ed1b7fdb9af3b5ae20a50-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-112.ec2.internal\" (UID: \"1ce6bbc5e87ed1b7fdb9af3b5ae20a50\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-112.ec2.internal"
Jul 06 14:19:38.570325 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.490515    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1ce6bbc5e87ed1b7fdb9af3b5ae20a50-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-112.ec2.internal\" (UID: \"1ce6bbc5e87ed1b7fdb9af3b5ae20a50\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-112.ec2.internal"
Jul 06 14:19:38.570325 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.490505    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/112772d63ce9100ee913123b6eaccc48-config\") pod \"kube-apiserver-proxy-ip-10-0-133-112.ec2.internal\" (UID: \"112772d63ce9100ee913123b6eaccc48\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-112.ec2.internal"
Jul 06 14:19:38.591360 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:19:38.591331    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-112.ec2.internal\" not found"
Jul 06 14:19:38.686562 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.686366    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-112.ec2.internal"
Jul 06 14:19:38.692107 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:19:38.691627    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-112.ec2.internal\" not found"
Jul 06 14:19:38.697959 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.697938    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-112.ec2.internal"
Jul 06 14:19:38.792156 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:19:38.792053    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-112.ec2.internal\" not found"
Jul 06 14:19:38.894771 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:19:38.894738    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-112.ec2.internal\" not found"
Jul 06 14:19:38.906962 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:38.906934    2559 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 06 14:19:38.995302 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:19:38.995267    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-112.ec2.internal\" not found"
Jul 06 14:19:39.027677 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:39.027653    2559 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 06 14:19:39.027826 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:39.027808    2559 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 06 14:19:39.027882 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:39.027819    2559 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 06 14:19:39.095515 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:19:39.095437    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-112.ec2.internal\" not found"
Jul 06 14:19:39.101005 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:39.100990    2559 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 06 14:19:39.134770 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:39.134695    2559 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-05 14:14:39 +0000 UTC" deadline="2028-03-27 16:23:23.419582728 +0000 UTC"
Jul 06 14:19:39.134770 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:39.134765    2559 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15122h3m44.284823512s"
Jul 06 14:19:39.192277 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:39.192247    2559 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-112.ec2.internal"
Jul 06 14:19:39.208878 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:39.208848    2559 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 06 14:19:39.211354 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:39.211331    2559 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-112.ec2.internal"
Jul 06 14:19:39.221139 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:39.221108    2559 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 06 14:19:39.569657 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:39.569627    2559 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 06 14:19:39.583186 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:39.583153    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ce6bbc5e87ed1b7fdb9af3b5ae20a50.slice/crio-369a40e4a20a4cfa7d1eeac13a75c6d8b6533ae9e83acc3a02118b2ab6ffe143 WatchSource:0}: Error finding container 369a40e4a20a4cfa7d1eeac13a75c6d8b6533ae9e83acc3a02118b2ab6ffe143: Status 404 returned error can't find the container with id 369a40e4a20a4cfa7d1eeac13a75c6d8b6533ae9e83acc3a02118b2ab6ffe143
Jul 06 14:19:39.589142 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:39.589122    2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 06 14:19:39.594977 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:39.594953    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod112772d63ce9100ee913123b6eaccc48.slice/crio-bfc0071fa7fda1cfca592c08f8899a552f128da3239b1e9e866eb6de9b92bd80 WatchSource:0}: Error finding container bfc0071fa7fda1cfca592c08f8899a552f128da3239b1e9e866eb6de9b92bd80: Status 404 returned error can't find the container with id bfc0071fa7fda1cfca592c08f8899a552f128da3239b1e9e866eb6de9b92bd80
Jul 06 14:19:40.071595 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.071546    2559 apiserver.go:52] "Watching apiserver"
Jul 06 14:19:40.083768 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.083738    2559 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 06 14:19:40.084090 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.084068    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-operator/iptables-alerter-b9rzt","openshift-ovn-kubernetes/ovnkube-node-prjxn","kube-system/global-pull-secret-syncer-qjsqm","kube-system/kube-apiserver-proxy-ip-10-0-133-112.ec2.internal","openshift-cluster-node-tuning-operator/tuned-8br6l","openshift-image-registry/node-ca-pr7f9","openshift-multus/multus-additional-cni-plugins-7b8n5","openshift-multus/multus-jqxvb","kube-system/konnectivity-agent-qr4tr","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-75cxd","openshift-dns/node-resolver-9tndp","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-112.ec2.internal","openshift-multus/network-metrics-daemon-v94gg","openshift-network-diagnostics/network-check-target-l87kx"]
Jul 06 14:19:40.086978 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.086959    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-75cxd"
Jul 06 14:19:40.089228 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.089204    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v94gg"
Jul 06 14:19:40.089228 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.089225    2559 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 06 14:19:40.089399 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:19:40.089281    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v94gg" podUID="1c05cc5f-44d6-46b6-947a-9d221f282365"
Jul 06 14:19:40.089706 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.089686    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 06 14:19:40.089986 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.089969    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-rznq7\""
Jul 06 14:19:40.090689 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.090657    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 06 14:19:40.091126 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.091109    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 06 14:19:40.092018 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.092000    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qjsqm"
Jul 06 14:19:40.092098 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:19:40.092057    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qjsqm" podUID="ab514f6f-ea23-486a-b065-aac1b24a5035"
Jul 06 14:19:40.094382 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.094360    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l87kx"
Jul 06 14:19:40.094469 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:19:40.094435    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l87kx" podUID="98382dda-aacf-4546-8a91-e6bd00632d71"
Jul 06 14:19:40.096618 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.096601    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-8br6l"
Jul 06 14:19:40.098897 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.098881    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-pr7f9"
Jul 06 14:19:40.099353 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.099333    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 06 14:19:40.099449 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.099433    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 06 14:19:40.099786 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.099769    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-rtkq9\""
Jul 06 14:19:40.100617 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.100600    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d0bff3f4-b4c2-4c2a-ad95-dc24f3b8c92d-registration-dir\") pod \"aws-ebs-csi-driver-node-75cxd\" (UID: \"d0bff3f4-b4c2-4c2a-ad95-dc24f3b8c92d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-75cxd"
Jul 06 14:19:40.100714 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.100624    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d0bff3f4-b4c2-4c2a-ad95-dc24f3b8c92d-device-dir\") pod \"aws-ebs-csi-driver-node-75cxd\" (UID: \"d0bff3f4-b4c2-4c2a-ad95-dc24f3b8c92d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-75cxd"
Jul 06 14:19:40.100714 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.100642    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d0bff3f4-b4c2-4c2a-ad95-dc24f3b8c92d-etc-selinux\") pod \"aws-ebs-csi-driver-node-75cxd\" (UID: \"d0bff3f4-b4c2-4c2a-ad95-dc24f3b8c92d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-75cxd"
Jul 06 14:19:40.100714 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.100681    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ab514f6f-ea23-486a-b065-aac1b24a5035-dbus\") pod \"global-pull-secret-syncer-qjsqm\" (UID: \"ab514f6f-ea23-486a-b065-aac1b24a5035\") " pod="kube-system/global-pull-secret-syncer-qjsqm"
Jul 06 14:19:40.100834 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.100745    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d0bff3f4-b4c2-4c2a-ad95-dc24f3b8c92d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-75cxd\" (UID: \"d0bff3f4-b4c2-4c2a-ad95-dc24f3b8c92d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-75cxd"
Jul 06 14:19:40.100834 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.100774    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d0bff3f4-b4c2-4c2a-ad95-dc24f3b8c92d-sys-fs\") pod \"aws-ebs-csi-driver-node-75cxd\" (UID: \"d0bff3f4-b4c2-4c2a-ad95-dc24f3b8c92d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-75cxd"
Jul 06 14:19:40.100834 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.100790    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ab514f6f-ea23-486a-b065-aac1b24a5035-original-pull-secret\") pod \"global-pull-secret-syncer-qjsqm\" (UID: \"ab514f6f-ea23-486a-b065-aac1b24a5035\") " pod="kube-system/global-pull-secret-syncer-qjsqm"
Jul 06 14:19:40.100834 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.100812    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fwx57\" (UniqueName: \"kubernetes.io/projected/d0bff3f4-b4c2-4c2a-ad95-dc24f3b8c92d-kube-api-access-fwx57\") pod \"aws-ebs-csi-driver-node-75cxd\" (UID: \"d0bff3f4-b4c2-4c2a-ad95-dc24f3b8c92d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-75cxd"
Jul 06 14:19:40.100981 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.100858    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1c05cc5f-44d6-46b6-947a-9d221f282365-metrics-certs\") pod \"network-metrics-daemon-v94gg\" (UID: \"1c05cc5f-44d6-46b6-947a-9d221f282365\") " pod="openshift-multus/network-metrics-daemon-v94gg"
Jul 06 14:19:40.100981 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.100908    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kdkph\" (UniqueName: \"kubernetes.io/projected/1c05cc5f-44d6-46b6-947a-9d221f282365-kube-api-access-kdkph\") pod \"network-metrics-daemon-v94gg\" (UID: \"1c05cc5f-44d6-46b6-947a-9d221f282365\") " pod="openshift-multus/network-metrics-daemon-v94gg"
Jul 06 14:19:40.100981 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.100929    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ab514f6f-ea23-486a-b065-aac1b24a5035-kubelet-config\") pod \"global-pull-secret-syncer-qjsqm\" (UID: \"ab514f6f-ea23-486a-b065-aac1b24a5035\") " pod="kube-system/global-pull-secret-syncer-qjsqm"
Jul 06 14:19:40.100981 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.100947    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d0bff3f4-b4c2-4c2a-ad95-dc24f3b8c92d-socket-dir\") pod \"aws-ebs-csi-driver-node-75cxd\" (UID: \"d0bff3f4-b4c2-4c2a-ad95-dc24f3b8c92d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-75cxd"
Jul 06 14:19:40.101188 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.101175    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-qr4tr"
Jul 06 14:19:40.101667 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.101629    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 06 14:19:40.101667 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.101645    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 06 14:19:40.101785 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.101639    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 06 14:19:40.101785 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.101629    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-zvkbn\""
Jul 06 14:19:40.103625 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.103606    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 06 14:19:40.103723 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.103656    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 06 14:19:40.103796 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.103776    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-fttd5\""
Jul 06 14:19:40.104010 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.103938    2559 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 06 14:19:40.104010 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.103994    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-7b8n5"
Jul 06 14:19:40.106128 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.106107    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 06 14:19:40.106526 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.106503    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 06 14:19:40.106638 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.106625    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 06 14:19:40.106854 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.106835    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-prjxn"
Jul 06 14:19:40.106939 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.106876    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 06 14:19:40.106939 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.106917    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 06 14:19:40.107056 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.106840    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-d4x8g\""
Jul 06 14:19:40.109340 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.109322    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-b9rzt"
Jul 06 14:19:40.109813 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.109794    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 06 14:19:40.109907 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.109843    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 06 14:19:40.109907 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.109861    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-jt2tj\""
Jul 06 14:19:40.110206 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.110189    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 06 14:19:40.110286 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.110226    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 06 14:19:40.111620 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.111565    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 06 14:19:40.111782 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.111764    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 06 14:19:40.112143 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.112122    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-9tndp"
Jul 06 14:19:40.112266 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.112248    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 06 14:19:40.112360 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.112341    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 06 14:19:40.112959 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.112937    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-2qlgb\""
Jul 06 14:19:40.113385 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.113352    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 06 14:19:40.115515 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.115486    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 06 14:19:40.115619 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.115601    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-4z4hr\""
Jul 06 14:19:40.115709 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.115676    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 06 14:19:40.116447 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.116431    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-jqxvb"
Jul 06 14:19:40.118709 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.118693    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-9cg22\""
Jul 06 14:19:40.118787 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.118695    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 06 14:19:40.132003 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.131985    2559 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-vd9nh"
Jul 06 14:19:40.133442 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.133427    2559 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 06 14:19:40.146389 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.146367    2559 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-vd9nh"
Jul 06 14:19:40.190226 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.190200    2559 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 06 14:19:40.202027 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.201993    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ef2ced35-fb83-46cb-9e31-2c0814bd5db2-serviceca\") pod \"node-ca-pr7f9\" (UID: \"ef2ced35-fb83-46cb-9e31-2c0814bd5db2\") " pod="openshift-image-registry/node-ca-pr7f9"
Jul 06 14:19:40.202027 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.202031    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pbmnn\" (UniqueName: \"kubernetes.io/projected/ef2ced35-fb83-46cb-9e31-2c0814bd5db2-kube-api-access-pbmnn\") pod \"node-ca-pr7f9\" (UID: \"ef2ced35-fb83-46cb-9e31-2c0814bd5db2\") " pod="openshift-image-registry/node-ca-pr7f9"
Jul 06 14:19:40.202702 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.202055    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7282da78-ea16-410d-902a-f908b4d5b1fc-cnibin\") pod \"multus-additional-cni-plugins-7b8n5\" (UID: \"7282da78-ea16-410d-902a-f908b4d5b1fc\") " pod="openshift-multus/multus-additional-cni-plugins-7b8n5"
Jul 06 14:19:40.202702 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.202119    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7282da78-ea16-410d-902a-f908b4d5b1fc-tuning-conf-dir\") pod \"multus-additional-cni-plugins-7b8n5\" (UID: \"7282da78-ea16-410d-902a-f908b4d5b1fc\") " pod="openshift-multus/multus-additional-cni-plugins-7b8n5"
Jul 06 14:19:40.202702 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.202157    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ca517334-9c07-4288-a0af-df1a8fc631dd-var-lib-kubelet\") pod \"tuned-8br6l\" (UID: \"ca517334-9c07-4288-a0af-df1a8fc631dd\") " pod="openshift-cluster-node-tuning-operator/tuned-8br6l"
Jul 06 14:19:40.202702 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.202189    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/ca517334-9c07-4288-a0af-df1a8fc631dd-run\") pod \"tuned-8br6l\" (UID: \"ca517334-9c07-4288-a0af-df1a8fc631dd\") " pod="openshift-cluster-node-tuning-operator/tuned-8br6l"
Jul 06 14:19:40.202702 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.202215    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ca517334-9c07-4288-a0af-df1a8fc631dd-tmp\") pod \"tuned-8br6l\" (UID: \"ca517334-9c07-4288-a0af-df1a8fc631dd\") " pod="openshift-cluster-node-tuning-operator/tuned-8br6l"
Jul 06 14:19:40.202702 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.202241    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a830d1ec-471c-4a9f-acd1-134c31193ad1-multus-cni-dir\") pod \"multus-jqxvb\" (UID: \"a830d1ec-471c-4a9f-acd1-134c31193ad1\") " pod="openshift-multus/multus-jqxvb"
Jul 06 14:19:40.202702 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.202324    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a830d1ec-471c-4a9f-acd1-134c31193ad1-host-run-netns\") pod \"multus-jqxvb\" (UID: \"a830d1ec-471c-4a9f-acd1-134c31193ad1\") " pod="openshift-multus/multus-jqxvb"
Jul 06 14:19:40.202702 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.202358    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ca452469-23ef-4ec3-a6f1-f3ca00cf6a9e-systemd-units\") pod \"ovnkube-node-prjxn\" (UID: \"ca452469-23ef-4ec3-a6f1-f3ca00cf6a9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-prjxn"
Jul 06 14:19:40.202702 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.202375    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ca452469-23ef-4ec3-a6f1-f3ca00cf6a9e-log-socket\") pod \"ovnkube-node-prjxn\" (UID: \"ca452469-23ef-4ec3-a6f1-f3ca00cf6a9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-prjxn"
Jul 06 14:19:40.202702 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.202394    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d0bff3f4-b4c2-4c2a-ad95-dc24f3b8c92d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-75cxd\" (UID: \"d0bff3f4-b4c2-4c2a-ad95-dc24f3b8c92d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-75cxd"
Jul 06 14:19:40.202702 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.202446    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ab514f6f-ea23-486a-b065-aac1b24a5035-original-pull-secret\") pod \"global-pull-secret-syncer-qjsqm\" (UID: \"ab514f6f-ea23-486a-b065-aac1b24a5035\") " pod="kube-system/global-pull-secret-syncer-qjsqm"
Jul 06 14:19:40.202702 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.202509    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/ca517334-9c07-4288-a0af-df1a8fc631dd-etc-sysconfig\") pod \"tuned-8br6l\" (UID: \"ca517334-9c07-4288-a0af-df1a8fc631dd\") " pod="openshift-cluster-node-tuning-operator/tuned-8br6l"
Jul 06 14:19:40.202702 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.202536    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d0bff3f4-b4c2-4c2a-ad95-dc24f3b8c92d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-75cxd\" (UID: \"d0bff3f4-b4c2-4c2a-ad95-dc24f3b8c92d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-75cxd"
Jul 06 14:19:40.202702 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:19:40.202551    2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 06 14:19:40.202702 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.202556    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/7282da78-ea16-410d-902a-f908b4d5b1fc-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-7b8n5\" (UID: \"7282da78-ea16-410d-902a-f908b4d5b1fc\") " pod="openshift-multus/multus-additional-cni-plugins-7b8n5"
Jul 06 14:19:40.202702 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.202605    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a830d1ec-471c-4a9f-acd1-134c31193ad1-host-var-lib-cni-multus\") pod \"multus-jqxvb\" (UID: \"a830d1ec-471c-4a9f-acd1-134c31193ad1\") " pod="openshift-multus/multus-jqxvb"
Jul 06 14:19:40.203508 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:19:40.202650    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ab514f6f-ea23-486a-b065-aac1b24a5035-original-pull-secret podName:ab514f6f-ea23-486a-b065-aac1b24a5035 nodeName:}" failed. No retries permitted until 2026-07-06 14:19:40.702613709 +0000 UTC m=+3.009905771 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ab514f6f-ea23-486a-b065-aac1b24a5035-original-pull-secret") pod "global-pull-secret-syncer-qjsqm" (UID: "ab514f6f-ea23-486a-b065-aac1b24a5035") : object "kube-system"/"original-pull-secret" not registered
Jul 06 14:19:40.203508 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.202692    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a830d1ec-471c-4a9f-acd1-134c31193ad1-os-release\") pod \"multus-jqxvb\" (UID: \"a830d1ec-471c-4a9f-acd1-134c31193ad1\") " pod="openshift-multus/multus-jqxvb"
Jul 06 14:19:40.203508 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.202714    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a830d1ec-471c-4a9f-acd1-134c31193ad1-multus-socket-dir-parent\") pod \"multus-jqxvb\" (UID: \"a830d1ec-471c-4a9f-acd1-134c31193ad1\") " pod="openshift-multus/multus-jqxvb"
Jul 06 14:19:40.203508 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.202732    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ca452469-23ef-4ec3-a6f1-f3ca00cf6a9e-run-openvswitch\") pod \"ovnkube-node-prjxn\" (UID: \"ca452469-23ef-4ec3-a6f1-f3ca00cf6a9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-prjxn"
Jul 06 14:19:40.203508 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.202756    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ca452469-23ef-4ec3-a6f1-f3ca00cf6a9e-host-cni-bin\") pod \"ovnkube-node-prjxn\" (UID: \"ca452469-23ef-4ec3-a6f1-f3ca00cf6a9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-prjxn"
Jul 06 14:19:40.203508 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.202776    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ca452469-23ef-4ec3-a6f1-f3ca00cf6a9e-ovnkube-config\") pod \"ovnkube-node-prjxn\" (UID: \"ca452469-23ef-4ec3-a6f1-f3ca00cf6a9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-prjxn"
Jul 06 14:19:40.203508 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.202825    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ca452469-23ef-4ec3-a6f1-f3ca00cf6a9e-ovnkube-script-lib\") pod \"ovnkube-node-prjxn\" (UID: \"ca452469-23ef-4ec3-a6f1-f3ca00cf6a9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-prjxn"
Jul 06 14:19:40.203508 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.202853    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d0bff3f4-b4c2-4c2a-ad95-dc24f3b8c92d-socket-dir\") pod \"aws-ebs-csi-driver-node-75cxd\" (UID: \"d0bff3f4-b4c2-4c2a-ad95-dc24f3b8c92d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-75cxd"
Jul 06 14:19:40.203508 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.202878    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7282da78-ea16-410d-902a-f908b4d5b1fc-os-release\") pod \"multus-additional-cni-plugins-7b8n5\" (UID: \"7282da78-ea16-410d-902a-f908b4d5b1fc\") " pod="openshift-multus/multus-additional-cni-plugins-7b8n5"
Jul 06 14:19:40.203508 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.202902    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f07d461c-9de3-43c8-b501-f50c0cd463eb-tmp-dir\") pod \"node-resolver-9tndp\" (UID: \"f07d461c-9de3-43c8-b501-f50c0cd463eb\") " pod="openshift-dns/node-resolver-9tndp"
Jul 06 14:19:40.203508 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.202956    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a830d1ec-471c-4a9f-acd1-134c31193ad1-hostroot\") pod \"multus-jqxvb\" (UID: \"a830d1ec-471c-4a9f-acd1-134c31193ad1\") " pod="openshift-multus/multus-jqxvb"
Jul 06 14:19:40.203508 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.203027    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d0bff3f4-b4c2-4c2a-ad95-dc24f3b8c92d-etc-selinux\") pod \"aws-ebs-csi-driver-node-75cxd\" (UID: \"d0bff3f4-b4c2-4c2a-ad95-dc24f3b8c92d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-75cxd"
Jul 06 14:19:40.203508 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.203042    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d0bff3f4-b4c2-4c2a-ad95-dc24f3b8c92d-socket-dir\") pod \"aws-ebs-csi-driver-node-75cxd\" (UID: \"d0bff3f4-b4c2-4c2a-ad95-dc24f3b8c92d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-75cxd"
Jul 06 14:19:40.203508 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.203073    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ab514f6f-ea23-486a-b065-aac1b24a5035-dbus\") pod \"global-pull-secret-syncer-qjsqm\" (UID: \"ab514f6f-ea23-486a-b065-aac1b24a5035\") " pod="kube-system/global-pull-secret-syncer-qjsqm"
Jul 06 14:19:40.203508 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.203112    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7282da78-ea16-410d-902a-f908b4d5b1fc-system-cni-dir\") pod \"multus-additional-cni-plugins-7b8n5\" (UID: \"7282da78-ea16-410d-902a-f908b4d5b1fc\") " pod="openshift-multus/multus-additional-cni-plugins-7b8n5"
Jul 06 14:19:40.203508 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.203153    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7282da78-ea16-410d-902a-f908b4d5b1fc-cni-binary-copy\") pod \"multus-additional-cni-plugins-7b8n5\" (UID: \"7282da78-ea16-410d-902a-f908b4d5b1fc\") " pod="openshift-multus/multus-additional-cni-plugins-7b8n5"
Jul 06 14:19:40.204273 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.203175    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d0bff3f4-b4c2-4c2a-ad95-dc24f3b8c92d-etc-selinux\") pod \"aws-ebs-csi-driver-node-75cxd\" (UID: \"d0bff3f4-b4c2-4c2a-ad95-dc24f3b8c92d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-75cxd"
Jul 06 14:19:40.204273 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.203179    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ca452469-23ef-4ec3-a6f1-f3ca00cf6a9e-env-overrides\") pod \"ovnkube-node-prjxn\" (UID: \"ca452469-23ef-4ec3-a6f1-f3ca00cf6a9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-prjxn"
Jul 06 14:19:40.204273 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.203222    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/5cfea822-c324-4a7d-ab24-c8a48da58bcf-iptables-alerter-script\") pod \"iptables-alerter-b9rzt\" (UID: \"5cfea822-c324-4a7d-ab24-c8a48da58bcf\") " pod="openshift-network-operator/iptables-alerter-b9rzt"
Jul 06 14:19:40.204273 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.203257    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ca452469-23ef-4ec3-a6f1-f3ca00cf6a9e-var-lib-openvswitch\") pod \"ovnkube-node-prjxn\" (UID: \"ca452469-23ef-4ec3-a6f1-f3ca00cf6a9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-prjxn"
Jul 06 14:19:40.204273 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.203274    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ab514f6f-ea23-486a-b065-aac1b24a5035-dbus\") pod \"global-pull-secret-syncer-qjsqm\" (UID: \"ab514f6f-ea23-486a-b065-aac1b24a5035\") " pod="kube-system/global-pull-secret-syncer-qjsqm"
Jul 06 14:19:40.204273 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.203282    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wpf65\" (UniqueName: \"kubernetes.io/projected/5cfea822-c324-4a7d-ab24-c8a48da58bcf-kube-api-access-wpf65\") pod \"iptables-alerter-b9rzt\" (UID: \"5cfea822-c324-4a7d-ab24-c8a48da58bcf\") " pod="openshift-network-operator/iptables-alerter-b9rzt"
Jul 06 14:19:40.204273 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.203334    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fwx57\" (UniqueName: \"kubernetes.io/projected/d0bff3f4-b4c2-4c2a-ad95-dc24f3b8c92d-kube-api-access-fwx57\") pod \"aws-ebs-csi-driver-node-75cxd\" (UID: \"d0bff3f4-b4c2-4c2a-ad95-dc24f3b8c92d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-75cxd"
Jul 06 14:19:40.204273 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.203364    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a830d1ec-471c-4a9f-acd1-134c31193ad1-multus-daemon-config\") pod \"multus-jqxvb\" (UID: \"a830d1ec-471c-4a9f-acd1-134c31193ad1\") " pod="openshift-multus/multus-jqxvb"
Jul 06 14:19:40.204273 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.203381    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mn7d6\" (UniqueName: \"kubernetes.io/projected/a830d1ec-471c-4a9f-acd1-134c31193ad1-kube-api-access-mn7d6\") pod \"multus-jqxvb\" (UID: \"a830d1ec-471c-4a9f-acd1-134c31193ad1\") " pod="openshift-multus/multus-jqxvb"
Jul 06 14:19:40.204273 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.203439    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ca452469-23ef-4ec3-a6f1-f3ca00cf6a9e-run-systemd\") pod \"ovnkube-node-prjxn\" (UID: \"ca452469-23ef-4ec3-a6f1-f3ca00cf6a9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-prjxn"
Jul 06 14:19:40.204273 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.203467    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d0bff3f4-b4c2-4c2a-ad95-dc24f3b8c92d-registration-dir\") pod \"aws-ebs-csi-driver-node-75cxd\" (UID: \"d0bff3f4-b4c2-4c2a-ad95-dc24f3b8c92d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-75cxd"
Jul 06 14:19:40.204273 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.203486    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/ca517334-9c07-4288-a0af-df1a8fc631dd-etc-tuned\") pod \"tuned-8br6l\" (UID: \"ca517334-9c07-4288-a0af-df1a8fc631dd\") " pod="openshift-cluster-node-tuning-operator/tuned-8br6l"
Jul 06 14:19:40.204273 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.203511    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ca452469-23ef-4ec3-a6f1-f3ca00cf6a9e-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-prjxn\" (UID: \"ca452469-23ef-4ec3-a6f1-f3ca00cf6a9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-prjxn"
Jul 06 14:19:40.204273 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.203530    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ca452469-23ef-4ec3-a6f1-f3ca00cf6a9e-ovn-node-metrics-cert\") pod \"ovnkube-node-prjxn\" (UID: \"ca452469-23ef-4ec3-a6f1-f3ca00cf6a9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-prjxn"
Jul 06 14:19:40.204273 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.203546    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a830d1ec-471c-4a9f-acd1-134c31193ad1-multus-conf-dir\") pod \"multus-jqxvb\" (UID: \"a830d1ec-471c-4a9f-acd1-134c31193ad1\") " pod="openshift-multus/multus-jqxvb"
Jul 06 14:19:40.204273 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.203569    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zx8rz\" (UniqueName: \"kubernetes.io/projected/ca517334-9c07-4288-a0af-df1a8fc631dd-kube-api-access-zx8rz\") pod \"tuned-8br6l\" (UID: \"ca517334-9c07-4288-a0af-df1a8fc631dd\") " pod="openshift-cluster-node-tuning-operator/tuned-8br6l"
Jul 06 14:19:40.205088 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.203609    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/f07d461c-9de3-43c8-b501-f50c0cd463eb-hosts-file\") pod \"node-resolver-9tndp\" (UID: \"f07d461c-9de3-43c8-b501-f50c0cd463eb\") " pod="openshift-dns/node-resolver-9tndp"
Jul 06 14:19:40.205088 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.203625    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nc82b\" (UniqueName: \"kubernetes.io/projected/f07d461c-9de3-43c8-b501-f50c0cd463eb-kube-api-access-nc82b\") pod \"node-resolver-9tndp\" (UID: \"f07d461c-9de3-43c8-b501-f50c0cd463eb\") " pod="openshift-dns/node-resolver-9tndp"
Jul 06 14:19:40.205088 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.203641    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a830d1ec-471c-4a9f-acd1-134c31193ad1-cnibin\") pod \"multus-jqxvb\" (UID: \"a830d1ec-471c-4a9f-acd1-134c31193ad1\") " pod="openshift-multus/multus-jqxvb"
Jul 06 14:19:40.205088 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.203657    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a830d1ec-471c-4a9f-acd1-134c31193ad1-host-var-lib-kubelet\") pod \"multus-jqxvb\" (UID: \"a830d1ec-471c-4a9f-acd1-134c31193ad1\") " pod="openshift-multus/multus-jqxvb"
Jul 06 14:19:40.205088 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.203678    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a830d1ec-471c-4a9f-acd1-134c31193ad1-etc-kubernetes\") pod \"multus-jqxvb\" (UID: \"a830d1ec-471c-4a9f-acd1-134c31193ad1\") " pod="openshift-multus/multus-jqxvb"
Jul 06 14:19:40.205088 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.203698    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/6a513fd4-439d-4806-afbc-6462321a6d29-konnectivity-ca\") pod \"konnectivity-agent-qr4tr\" (UID: \"6a513fd4-439d-4806-afbc-6462321a6d29\") " pod="kube-system/konnectivity-agent-qr4tr"
Jul 06 14:19:40.205088 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.203716    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a830d1ec-471c-4a9f-acd1-134c31193ad1-cni-binary-copy\") pod \"multus-jqxvb\" (UID: \"a830d1ec-471c-4a9f-acd1-134c31193ad1\") " pod="openshift-multus/multus-jqxvb"
Jul 06 14:19:40.205088 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.203732    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a830d1ec-471c-4a9f-acd1-134c31193ad1-host-run-multus-certs\") pod \"multus-jqxvb\" (UID: \"a830d1ec-471c-4a9f-acd1-134c31193ad1\") " pod="openshift-multus/multus-jqxvb"
Jul 06 14:19:40.205088 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.203783    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ca452469-23ef-4ec3-a6f1-f3ca00cf6a9e-host-kubelet\") pod \"ovnkube-node-prjxn\" (UID: \"ca452469-23ef-4ec3-a6f1-f3ca00cf6a9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-prjxn"
Jul 06 14:19:40.205088 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.203802    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ab514f6f-ea23-486a-b065-aac1b24a5035-kubelet-config\") pod \"global-pull-secret-syncer-qjsqm\" (UID: \"ab514f6f-ea23-486a-b065-aac1b24a5035\") " pod="kube-system/global-pull-secret-syncer-qjsqm"
Jul 06 14:19:40.205088 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.203822    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/ca517334-9c07-4288-a0af-df1a8fc631dd-etc-modprobe-d\") pod \"tuned-8br6l\" (UID: \"ca517334-9c07-4288-a0af-df1a8fc631dd\") " pod="openshift-cluster-node-tuning-operator/tuned-8br6l"
Jul 06 14:19:40.205088 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.203855    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ca517334-9c07-4288-a0af-df1a8fc631dd-etc-kubernetes\") pod \"tuned-8br6l\" (UID: \"ca517334-9c07-4288-a0af-df1a8fc631dd\") " pod="openshift-cluster-node-tuning-operator/tuned-8br6l"
Jul 06 14:19:40.205088 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.203875    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kp8rb\" (UniqueName: \"kubernetes.io/projected/7282da78-ea16-410d-902a-f908b4d5b1fc-kube-api-access-kp8rb\") pod \"multus-additional-cni-plugins-7b8n5\" (UID: \"7282da78-ea16-410d-902a-f908b4d5b1fc\") " pod="openshift-multus/multus-additional-cni-plugins-7b8n5"
Jul 06 14:19:40.205088 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.203914    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ca452469-23ef-4ec3-a6f1-f3ca00cf6a9e-host-run-netns\") pod \"ovnkube-node-prjxn\" (UID: \"ca452469-23ef-4ec3-a6f1-f3ca00cf6a9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-prjxn"
Jul 06 14:19:40.205088 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.203959    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ca452469-23ef-4ec3-a6f1-f3ca00cf6a9e-run-ovn\") pod \"ovnkube-node-prjxn\" (UID: \"ca452469-23ef-4ec3-a6f1-f3ca00cf6a9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-prjxn"
Jul 06 14:19:40.205088 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.204016    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d0bff3f4-b4c2-4c2a-ad95-dc24f3b8c92d-registration-dir\") pod \"aws-ebs-csi-driver-node-75cxd\" (UID: \"d0bff3f4-b4c2-4c2a-ad95-dc24f3b8c92d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-75cxd"
Jul 06 14:19:40.205088 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.204043    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ab514f6f-ea23-486a-b065-aac1b24a5035-kubelet-config\") pod \"global-pull-secret-syncer-qjsqm\" (UID: \"ab514f6f-ea23-486a-b065-aac1b24a5035\") " pod="kube-system/global-pull-secret-syncer-qjsqm"
Jul 06 14:19:40.205732 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.204089    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ca517334-9c07-4288-a0af-df1a8fc631dd-sys\") pod \"tuned-8br6l\" (UID: \"ca517334-9c07-4288-a0af-df1a8fc631dd\") " pod="openshift-cluster-node-tuning-operator/tuned-8br6l"
Jul 06 14:19:40.205732 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.204125    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a830d1ec-471c-4a9f-acd1-134c31193ad1-system-cni-dir\") pod \"multus-jqxvb\" (UID: \"a830d1ec-471c-4a9f-acd1-134c31193ad1\") " pod="openshift-multus/multus-jqxvb"
Jul 06 14:19:40.205732 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.204166    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a830d1ec-471c-4a9f-acd1-134c31193ad1-host-run-k8s-cni-cncf-io\") pod \"multus-jqxvb\" (UID: \"a830d1ec-471c-4a9f-acd1-134c31193ad1\") " pod="openshift-multus/multus-jqxvb"
Jul 06 14:19:40.205732 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.204199    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2r8z4\" (UniqueName: \"kubernetes.io/projected/98382dda-aacf-4546-8a91-e6bd00632d71-kube-api-access-2r8z4\") pod \"network-check-target-l87kx\" (UID: \"98382dda-aacf-4546-8a91-e6bd00632d71\") " pod="openshift-network-diagnostics/network-check-target-l87kx"
Jul 06 14:19:40.205732 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.204246    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/6a513fd4-439d-4806-afbc-6462321a6d29-agent-certs\") pod \"konnectivity-agent-qr4tr\" (UID: \"6a513fd4-439d-4806-afbc-6462321a6d29\") " pod="kube-system/konnectivity-agent-qr4tr"
Jul 06 14:19:40.205732 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.204269    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d0bff3f4-b4c2-4c2a-ad95-dc24f3b8c92d-device-dir\") pod \"aws-ebs-csi-driver-node-75cxd\" (UID: \"d0bff3f4-b4c2-4c2a-ad95-dc24f3b8c92d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-75cxd"
Jul 06 14:19:40.205732 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.204293    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/ca517334-9c07-4288-a0af-df1a8fc631dd-etc-sysctl-conf\") pod \"tuned-8br6l\" (UID: \"ca517334-9c07-4288-a0af-df1a8fc631dd\") " pod="openshift-cluster-node-tuning-operator/tuned-8br6l"
Jul 06 14:19:40.205732 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.204319    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/ca517334-9c07-4288-a0af-df1a8fc631dd-etc-systemd\") pod \"tuned-8br6l\" (UID: \"ca517334-9c07-4288-a0af-df1a8fc631dd\") " pod="openshift-cluster-node-tuning-operator/tuned-8br6l"
Jul 06 14:19:40.205732 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.204350    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ca517334-9c07-4288-a0af-df1a8fc631dd-lib-modules\") pod \"tuned-8br6l\" (UID: \"ca517334-9c07-4288-a0af-df1a8fc631dd\") " pod="openshift-cluster-node-tuning-operator/tuned-8br6l"
Jul 06 14:19:40.205732 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.204377    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ca517334-9c07-4288-a0af-df1a8fc631dd-host\") pod \"tuned-8br6l\" (UID: \"ca517334-9c07-4288-a0af-df1a8fc631dd\") " pod="openshift-cluster-node-tuning-operator/tuned-8br6l"
Jul 06 14:19:40.205732 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.204379    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d0bff3f4-b4c2-4c2a-ad95-dc24f3b8c92d-device-dir\") pod \"aws-ebs-csi-driver-node-75cxd\" (UID: \"d0bff3f4-b4c2-4c2a-ad95-dc24f3b8c92d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-75cxd"
Jul 06 14:19:40.205732 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.204402    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/7282da78-ea16-410d-902a-f908b4d5b1fc-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-7b8n5\" (UID: \"7282da78-ea16-410d-902a-f908b4d5b1fc\") " pod="openshift-multus/multus-additional-cni-plugins-7b8n5"
Jul 06 14:19:40.205732 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.204436    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a830d1ec-471c-4a9f-acd1-134c31193ad1-host-var-lib-cni-bin\") pod \"multus-jqxvb\" (UID: \"a830d1ec-471c-4a9f-acd1-134c31193ad1\") " pod="openshift-multus/multus-jqxvb"
Jul 06 14:19:40.205732 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.204476    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ca452469-23ef-4ec3-a6f1-f3ca00cf6a9e-host-run-ovn-kubernetes\") pod \"ovnkube-node-prjxn\" (UID: \"ca452469-23ef-4ec3-a6f1-f3ca00cf6a9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-prjxn"
Jul 06 14:19:40.205732 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.204511    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d0bff3f4-b4c2-4c2a-ad95-dc24f3b8c92d-sys-fs\") pod \"aws-ebs-csi-driver-node-75cxd\" (UID: \"d0bff3f4-b4c2-4c2a-ad95-dc24f3b8c92d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-75cxd"
Jul 06 14:19:40.205732 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.204540    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ef2ced35-fb83-46cb-9e31-2c0814bd5db2-host\") pod \"node-ca-pr7f9\" (UID: \"ef2ced35-fb83-46cb-9e31-2c0814bd5db2\") " pod="openshift-image-registry/node-ca-pr7f9"
Jul 06 14:19:40.205732 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.204564    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ca452469-23ef-4ec3-a6f1-f3ca00cf6a9e-host-slash\") pod \"ovnkube-node-prjxn\" (UID: \"ca452469-23ef-4ec3-a6f1-f3ca00cf6a9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-prjxn"
Jul 06 14:19:40.206298 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.204610    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ca452469-23ef-4ec3-a6f1-f3ca00cf6a9e-etc-openvswitch\") pod \"ovnkube-node-prjxn\" (UID: \"ca452469-23ef-4ec3-a6f1-f3ca00cf6a9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-prjxn"
Jul 06 14:19:40.206298 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.204641    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d0bff3f4-b4c2-4c2a-ad95-dc24f3b8c92d-sys-fs\") pod \"aws-ebs-csi-driver-node-75cxd\" (UID: \"d0bff3f4-b4c2-4c2a-ad95-dc24f3b8c92d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-75cxd"
Jul 06 14:19:40.206298 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.204701    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ca452469-23ef-4ec3-a6f1-f3ca00cf6a9e-node-log\") pod \"ovnkube-node-prjxn\" (UID: \"ca452469-23ef-4ec3-a6f1-f3ca00cf6a9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-prjxn"
Jul 06 14:19:40.206298 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.204738    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ca452469-23ef-4ec3-a6f1-f3ca00cf6a9e-host-cni-netd\") pod \"ovnkube-node-prjxn\" (UID: \"ca452469-23ef-4ec3-a6f1-f3ca00cf6a9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-prjxn"
Jul 06 14:19:40.206298 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.204765    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gl4bd\" (UniqueName: \"kubernetes.io/projected/ca452469-23ef-4ec3-a6f1-f3ca00cf6a9e-kube-api-access-gl4bd\") pod \"ovnkube-node-prjxn\" (UID: \"ca452469-23ef-4ec3-a6f1-f3ca00cf6a9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-prjxn"
Jul 06 14:19:40.206298 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.204792    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5cfea822-c324-4a7d-ab24-c8a48da58bcf-host-slash\") pod \"iptables-alerter-b9rzt\" (UID: \"5cfea822-c324-4a7d-ab24-c8a48da58bcf\") " pod="openshift-network-operator/iptables-alerter-b9rzt"
Jul 06 14:19:40.206298 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.204834    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1c05cc5f-44d6-46b6-947a-9d221f282365-metrics-certs\") pod \"network-metrics-daemon-v94gg\" (UID: \"1c05cc5f-44d6-46b6-947a-9d221f282365\") " pod="openshift-multus/network-metrics-daemon-v94gg"
Jul 06 14:19:40.206298 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.204861    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kdkph\" (UniqueName: \"kubernetes.io/projected/1c05cc5f-44d6-46b6-947a-9d221f282365-kube-api-access-kdkph\") pod \"network-metrics-daemon-v94gg\" (UID: \"1c05cc5f-44d6-46b6-947a-9d221f282365\") " pod="openshift-multus/network-metrics-daemon-v94gg"
Jul 06 14:19:40.206298 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.204890    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/ca517334-9c07-4288-a0af-df1a8fc631dd-etc-sysctl-d\") pod \"tuned-8br6l\" (UID: \"ca517334-9c07-4288-a0af-df1a8fc631dd\") " pod="openshift-cluster-node-tuning-operator/tuned-8br6l"
Jul 06 14:19:40.206298 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:19:40.204931    2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 14:19:40.206298 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:19:40.204980    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1c05cc5f-44d6-46b6-947a-9d221f282365-metrics-certs podName:1c05cc5f-44d6-46b6-947a-9d221f282365 nodeName:}" failed. No retries permitted until 2026-07-06 14:19:40.704966602 +0000 UTC m=+3.012258665 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1c05cc5f-44d6-46b6-947a-9d221f282365-metrics-certs") pod "network-metrics-daemon-v94gg" (UID: "1c05cc5f-44d6-46b6-947a-9d221f282365") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 14:19:40.214193 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.214165    2559 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 06 14:19:40.218328 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.218303    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kdkph\" (UniqueName: \"kubernetes.io/projected/1c05cc5f-44d6-46b6-947a-9d221f282365-kube-api-access-kdkph\") pod \"network-metrics-daemon-v94gg\" (UID: \"1c05cc5f-44d6-46b6-947a-9d221f282365\") " pod="openshift-multus/network-metrics-daemon-v94gg"
Jul 06 14:19:40.218449 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.218333    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fwx57\" (UniqueName: \"kubernetes.io/projected/d0bff3f4-b4c2-4c2a-ad95-dc24f3b8c92d-kube-api-access-fwx57\") pod \"aws-ebs-csi-driver-node-75cxd\" (UID: \"d0bff3f4-b4c2-4c2a-ad95-dc24f3b8c92d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-75cxd"
Jul 06 14:19:40.228565 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.228508    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-112.ec2.internal" event={"ID":"112772d63ce9100ee913123b6eaccc48","Type":"ContainerStarted","Data":"bfc0071fa7fda1cfca592c08f8899a552f128da3239b1e9e866eb6de9b92bd80"}
Jul 06 14:19:40.230082 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.230054    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-112.ec2.internal" event={"ID":"1ce6bbc5e87ed1b7fdb9af3b5ae20a50","Type":"ContainerStarted","Data":"369a40e4a20a4cfa7d1eeac13a75c6d8b6533ae9e83acc3a02118b2ab6ffe143"}
Jul 06 14:19:40.305446 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.305153    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/ca517334-9c07-4288-a0af-df1a8fc631dd-etc-tuned\") pod \"tuned-8br6l\" (UID: \"ca517334-9c07-4288-a0af-df1a8fc631dd\") " pod="openshift-cluster-node-tuning-operator/tuned-8br6l"
Jul 06 14:19:40.305446 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.305206    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ca452469-23ef-4ec3-a6f1-f3ca00cf6a9e-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-prjxn\" (UID: \"ca452469-23ef-4ec3-a6f1-f3ca00cf6a9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-prjxn"
Jul 06 14:19:40.305446 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.305237    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ca452469-23ef-4ec3-a6f1-f3ca00cf6a9e-ovn-node-metrics-cert\") pod \"ovnkube-node-prjxn\" (UID: \"ca452469-23ef-4ec3-a6f1-f3ca00cf6a9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-prjxn"
Jul 06 14:19:40.305446 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.305255    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a830d1ec-471c-4a9f-acd1-134c31193ad1-multus-conf-dir\") pod \"multus-jqxvb\" (UID: \"a830d1ec-471c-4a9f-acd1-134c31193ad1\") " pod="openshift-multus/multus-jqxvb"
Jul 06 14:19:40.305446 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.305281    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zx8rz\" (UniqueName: \"kubernetes.io/projected/ca517334-9c07-4288-a0af-df1a8fc631dd-kube-api-access-zx8rz\") pod \"tuned-8br6l\" (UID: \"ca517334-9c07-4288-a0af-df1a8fc631dd\") " pod="openshift-cluster-node-tuning-operator/tuned-8br6l"
Jul 06 14:19:40.305446 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.305298    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/f07d461c-9de3-43c8-b501-f50c0cd463eb-hosts-file\") pod \"node-resolver-9tndp\" (UID: \"f07d461c-9de3-43c8-b501-f50c0cd463eb\") " pod="openshift-dns/node-resolver-9tndp"
Jul 06 14:19:40.305446 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.305313    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nc82b\" (UniqueName: \"kubernetes.io/projected/f07d461c-9de3-43c8-b501-f50c0cd463eb-kube-api-access-nc82b\") pod \"node-resolver-9tndp\" (UID: \"f07d461c-9de3-43c8-b501-f50c0cd463eb\") " pod="openshift-dns/node-resolver-9tndp"
Jul 06 14:19:40.305446 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.305329    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a830d1ec-471c-4a9f-acd1-134c31193ad1-cnibin\") pod \"multus-jqxvb\" (UID: \"a830d1ec-471c-4a9f-acd1-134c31193ad1\") " pod="openshift-multus/multus-jqxvb"
Jul 06 14:19:40.305446 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.305353    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a830d1ec-471c-4a9f-acd1-134c31193ad1-host-var-lib-kubelet\") pod \"multus-jqxvb\" (UID: \"a830d1ec-471c-4a9f-acd1-134c31193ad1\") " pod="openshift-multus/multus-jqxvb"
Jul 06 14:19:40.305446 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.305386    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a830d1ec-471c-4a9f-acd1-134c31193ad1-etc-kubernetes\") pod \"multus-jqxvb\" (UID: \"a830d1ec-471c-4a9f-acd1-134c31193ad1\") " pod="openshift-multus/multus-jqxvb"
Jul 06 14:19:40.305446 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.305410    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/6a513fd4-439d-4806-afbc-6462321a6d29-konnectivity-ca\") pod \"konnectivity-agent-qr4tr\" (UID: \"6a513fd4-439d-4806-afbc-6462321a6d29\") " pod="kube-system/konnectivity-agent-qr4tr"
Jul 06 14:19:40.305446 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.305436    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a830d1ec-471c-4a9f-acd1-134c31193ad1-cni-binary-copy\") pod \"multus-jqxvb\" (UID: \"a830d1ec-471c-4a9f-acd1-134c31193ad1\") " pod="openshift-multus/multus-jqxvb"
Jul 06 14:19:40.306239 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.305470    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a830d1ec-471c-4a9f-acd1-134c31193ad1-host-run-multus-certs\") pod \"multus-jqxvb\" (UID: \"a830d1ec-471c-4a9f-acd1-134c31193ad1\") " pod="openshift-multus/multus-jqxvb"
Jul 06 14:19:40.306239 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.305503    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ca452469-23ef-4ec3-a6f1-f3ca00cf6a9e-host-kubelet\") pod \"ovnkube-node-prjxn\" (UID: \"ca452469-23ef-4ec3-a6f1-f3ca00cf6a9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-prjxn"
Jul 06 14:19:40.306239 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.305533    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/ca517334-9c07-4288-a0af-df1a8fc631dd-etc-modprobe-d\") pod \"tuned-8br6l\" (UID: \"ca517334-9c07-4288-a0af-df1a8fc631dd\") " pod="openshift-cluster-node-tuning-operator/tuned-8br6l"
Jul 06 14:19:40.306239 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.305558    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ca517334-9c07-4288-a0af-df1a8fc631dd-etc-kubernetes\") pod \"tuned-8br6l\" (UID: \"ca517334-9c07-4288-a0af-df1a8fc631dd\") " pod="openshift-cluster-node-tuning-operator/tuned-8br6l"
Jul 06 14:19:40.306239 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.305562    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a830d1ec-471c-4a9f-acd1-134c31193ad1-multus-conf-dir\") pod \"multus-jqxvb\" (UID: \"a830d1ec-471c-4a9f-acd1-134c31193ad1\") " pod="openshift-multus/multus-jqxvb"
Jul 06 14:19:40.306239 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.305582    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kp8rb\" (UniqueName: \"kubernetes.io/projected/7282da78-ea16-410d-902a-f908b4d5b1fc-kube-api-access-kp8rb\") pod \"multus-additional-cni-plugins-7b8n5\" (UID: \"7282da78-ea16-410d-902a-f908b4d5b1fc\") " pod="openshift-multus/multus-additional-cni-plugins-7b8n5"
Jul 06 14:19:40.306239 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.305705    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a830d1ec-471c-4a9f-acd1-134c31193ad1-cnibin\") pod \"multus-jqxvb\" (UID: \"a830d1ec-471c-4a9f-acd1-134c31193ad1\") " pod="openshift-multus/multus-jqxvb"
Jul 06 14:19:40.306239 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.305763    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a830d1ec-471c-4a9f-acd1-134c31193ad1-host-var-lib-kubelet\") pod \"multus-jqxvb\" (UID: \"a830d1ec-471c-4a9f-acd1-134c31193ad1\") " pod="openshift-multus/multus-jqxvb"
Jul 06 14:19:40.306239 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.305763    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/f07d461c-9de3-43c8-b501-f50c0cd463eb-hosts-file\") pod \"node-resolver-9tndp\" (UID: \"f07d461c-9de3-43c8-b501-f50c0cd463eb\") " pod="openshift-dns/node-resolver-9tndp"
Jul 06 14:19:40.306239 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.305803    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a830d1ec-471c-4a9f-acd1-134c31193ad1-etc-kubernetes\") pod \"multus-jqxvb\" (UID: \"a830d1ec-471c-4a9f-acd1-134c31193ad1\") " pod="openshift-multus/multus-jqxvb"
Jul 06 14:19:40.306239 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.305855    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ca452469-23ef-4ec3-a6f1-f3ca00cf6a9e-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-prjxn\" (UID: \"ca452469-23ef-4ec3-a6f1-f3ca00cf6a9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-prjxn"
Jul 06 14:19:40.306239 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.305904    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ca517334-9c07-4288-a0af-df1a8fc631dd-etc-kubernetes\") pod \"tuned-8br6l\" (UID: \"ca517334-9c07-4288-a0af-df1a8fc631dd\") " pod="openshift-cluster-node-tuning-operator/tuned-8br6l"
Jul 06 14:19:40.306239 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.305911    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/ca517334-9c07-4288-a0af-df1a8fc631dd-etc-modprobe-d\") pod \"tuned-8br6l\" (UID: \"ca517334-9c07-4288-a0af-df1a8fc631dd\") " pod="openshift-cluster-node-tuning-operator/tuned-8br6l"
Jul 06 14:19:40.306239 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.305946    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a830d1ec-471c-4a9f-acd1-134c31193ad1-host-run-multus-certs\") pod \"multus-jqxvb\" (UID: \"a830d1ec-471c-4a9f-acd1-134c31193ad1\") " pod="openshift-multus/multus-jqxvb"
Jul 06 14:19:40.306239 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.305947    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ca452469-23ef-4ec3-a6f1-f3ca00cf6a9e-host-run-netns\") pod \"ovnkube-node-prjxn\" (UID: \"ca452469-23ef-4ec3-a6f1-f3ca00cf6a9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-prjxn"
Jul 06 14:19:40.306239 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.305974    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ca452469-23ef-4ec3-a6f1-f3ca00cf6a9e-host-kubelet\") pod \"ovnkube-node-prjxn\" (UID: \"ca452469-23ef-4ec3-a6f1-f3ca00cf6a9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-prjxn"
Jul 06 14:19:40.306239 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.305985    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ca452469-23ef-4ec3-a6f1-f3ca00cf6a9e-run-ovn\") pod \"ovnkube-node-prjxn\" (UID: \"ca452469-23ef-4ec3-a6f1-f3ca00cf6a9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-prjxn"
Jul 06 14:19:40.306239 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.306015    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ca517334-9c07-4288-a0af-df1a8fc631dd-sys\") pod \"tuned-8br6l\" (UID: \"ca517334-9c07-4288-a0af-df1a8fc631dd\") " pod="openshift-cluster-node-tuning-operator/tuned-8br6l"
Jul 06 14:19:40.307150 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.306021    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ca452469-23ef-4ec3-a6f1-f3ca00cf6a9e-host-run-netns\") pod \"ovnkube-node-prjxn\" (UID: \"ca452469-23ef-4ec3-a6f1-f3ca00cf6a9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-prjxn"
Jul 06 14:19:40.307150 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.306039    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a830d1ec-471c-4a9f-acd1-134c31193ad1-system-cni-dir\") pod \"multus-jqxvb\" (UID: \"a830d1ec-471c-4a9f-acd1-134c31193ad1\") " pod="openshift-multus/multus-jqxvb"
Jul 06 14:19:40.307150 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.306064    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ca452469-23ef-4ec3-a6f1-f3ca00cf6a9e-run-ovn\") pod \"ovnkube-node-prjxn\" (UID: \"ca452469-23ef-4ec3-a6f1-f3ca00cf6a9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-prjxn"
Jul 06 14:19:40.307150 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.306064    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a830d1ec-471c-4a9f-acd1-134c31193ad1-host-run-k8s-cni-cncf-io\") pod \"multus-jqxvb\" (UID: \"a830d1ec-471c-4a9f-acd1-134c31193ad1\") " pod="openshift-multus/multus-jqxvb"
Jul 06 14:19:40.307150 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.306120    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ca517334-9c07-4288-a0af-df1a8fc631dd-sys\") pod \"tuned-8br6l\" (UID: \"ca517334-9c07-4288-a0af-df1a8fc631dd\") " pod="openshift-cluster-node-tuning-operator/tuned-8br6l"
Jul 06 14:19:40.307150 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.306127    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a830d1ec-471c-4a9f-acd1-134c31193ad1-system-cni-dir\") pod \"multus-jqxvb\" (UID: \"a830d1ec-471c-4a9f-acd1-134c31193ad1\") " pod="openshift-multus/multus-jqxvb"
Jul 06 14:19:40.307150 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.306126    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2r8z4\" (UniqueName: \"kubernetes.io/projected/98382dda-aacf-4546-8a91-e6bd00632d71-kube-api-access-2r8z4\") pod \"network-check-target-l87kx\" (UID: \"98382dda-aacf-4546-8a91-e6bd00632d71\") " pod="openshift-network-diagnostics/network-check-target-l87kx"
Jul 06 14:19:40.307150 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.306159    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a830d1ec-471c-4a9f-acd1-134c31193ad1-host-run-k8s-cni-cncf-io\") pod \"multus-jqxvb\" (UID: \"a830d1ec-471c-4a9f-acd1-134c31193ad1\") " pod="openshift-multus/multus-jqxvb"
Jul 06 14:19:40.307150 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.306165    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/6a513fd4-439d-4806-afbc-6462321a6d29-agent-certs\") pod \"konnectivity-agent-qr4tr\" (UID: \"6a513fd4-439d-4806-afbc-6462321a6d29\") " pod="kube-system/konnectivity-agent-qr4tr"
Jul 06 14:19:40.307150 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.306196    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/ca517334-9c07-4288-a0af-df1a8fc631dd-etc-sysctl-conf\") pod \"tuned-8br6l\" (UID: \"ca517334-9c07-4288-a0af-df1a8fc631dd\") " pod="openshift-cluster-node-tuning-operator/tuned-8br6l"
Jul 06 14:19:40.307150 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.306220    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/ca517334-9c07-4288-a0af-df1a8fc631dd-etc-systemd\") pod \"tuned-8br6l\" (UID: \"ca517334-9c07-4288-a0af-df1a8fc631dd\") " pod="openshift-cluster-node-tuning-operator/tuned-8br6l"
Jul 06 14:19:40.307150 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.306275    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/ca517334-9c07-4288-a0af-df1a8fc631dd-etc-systemd\") pod \"tuned-8br6l\" (UID: \"ca517334-9c07-4288-a0af-df1a8fc631dd\") " pod="openshift-cluster-node-tuning-operator/tuned-8br6l"
Jul 06 14:19:40.307150 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.306309    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ca517334-9c07-4288-a0af-df1a8fc631dd-lib-modules\") pod \"tuned-8br6l\" (UID: \"ca517334-9c07-4288-a0af-df1a8fc631dd\") " pod="openshift-cluster-node-tuning-operator/tuned-8br6l"
Jul 06 14:19:40.307150 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.306336    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ca517334-9c07-4288-a0af-df1a8fc631dd-host\") pod \"tuned-8br6l\" (UID: \"ca517334-9c07-4288-a0af-df1a8fc631dd\") " pod="openshift-cluster-node-tuning-operator/tuned-8br6l"
Jul 06 14:19:40.307150 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.306345    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/ca517334-9c07-4288-a0af-df1a8fc631dd-etc-sysctl-conf\") pod \"tuned-8br6l\" (UID: \"ca517334-9c07-4288-a0af-df1a8fc631dd\") " pod="openshift-cluster-node-tuning-operator/tuned-8br6l"
Jul 06 14:19:40.307150 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.306364    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/7282da78-ea16-410d-902a-f908b4d5b1fc-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-7b8n5\" (UID: \"7282da78-ea16-410d-902a-f908b4d5b1fc\") " pod="openshift-multus/multus-additional-cni-plugins-7b8n5"
Jul 06 14:19:40.307150 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.306393    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a830d1ec-471c-4a9f-acd1-134c31193ad1-host-var-lib-cni-bin\") pod \"multus-jqxvb\" (UID: \"a830d1ec-471c-4a9f-acd1-134c31193ad1\") " pod="openshift-multus/multus-jqxvb"
Jul 06 14:19:40.307150 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.306406    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ca517334-9c07-4288-a0af-df1a8fc631dd-host\") pod \"tuned-8br6l\" (UID: \"ca517334-9c07-4288-a0af-df1a8fc631dd\") " pod="openshift-cluster-node-tuning-operator/tuned-8br6l"
Jul 06 14:19:40.308083 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.306404    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/6a513fd4-439d-4806-afbc-6462321a6d29-konnectivity-ca\") pod \"konnectivity-agent-qr4tr\" (UID: \"6a513fd4-439d-4806-afbc-6462321a6d29\") " pod="kube-system/konnectivity-agent-qr4tr"
Jul 06 14:19:40.308083 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.306419    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ca452469-23ef-4ec3-a6f1-f3ca00cf6a9e-host-run-ovn-kubernetes\") pod \"ovnkube-node-prjxn\" (UID: \"ca452469-23ef-4ec3-a6f1-f3ca00cf6a9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-prjxn"
Jul 06 14:19:40.308083 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.306460    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ef2ced35-fb83-46cb-9e31-2c0814bd5db2-host\") pod \"node-ca-pr7f9\" (UID: \"ef2ced35-fb83-46cb-9e31-2c0814bd5db2\") " pod="openshift-image-registry/node-ca-pr7f9"
Jul 06 14:19:40.308083 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.306463    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ca517334-9c07-4288-a0af-df1a8fc631dd-lib-modules\") pod \"tuned-8br6l\" (UID: \"ca517334-9c07-4288-a0af-df1a8fc631dd\") " pod="openshift-cluster-node-tuning-operator/tuned-8br6l"
Jul 06 14:19:40.308083 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.306487    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ca452469-23ef-4ec3-a6f1-f3ca00cf6a9e-host-slash\") pod \"ovnkube-node-prjxn\" (UID: \"ca452469-23ef-4ec3-a6f1-f3ca00cf6a9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-prjxn"
Jul 06 14:19:40.308083 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.306497    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a830d1ec-471c-4a9f-acd1-134c31193ad1-cni-binary-copy\") pod \"multus-jqxvb\" (UID: \"a830d1ec-471c-4a9f-acd1-134c31193ad1\") " pod="openshift-multus/multus-jqxvb"
Jul 06 14:19:40.308083 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.306503    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a830d1ec-471c-4a9f-acd1-134c31193ad1-host-var-lib-cni-bin\") pod \"multus-jqxvb\" (UID: \"a830d1ec-471c-4a9f-acd1-134c31193ad1\") " pod="openshift-multus/multus-jqxvb"
Jul 06 14:19:40.308083 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.306513    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ca452469-23ef-4ec3-a6f1-f3ca00cf6a9e-etc-openvswitch\") pod \"ovnkube-node-prjxn\" (UID: \"ca452469-23ef-4ec3-a6f1-f3ca00cf6a9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-prjxn"
Jul 06 14:19:40.308083 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.306521    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ef2ced35-fb83-46cb-9e31-2c0814bd5db2-host\") pod \"node-ca-pr7f9\" (UID: \"ef2ced35-fb83-46cb-9e31-2c0814bd5db2\") " pod="openshift-image-registry/node-ca-pr7f9"
Jul 06 14:19:40.308083 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.306547    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ca452469-23ef-4ec3-a6f1-f3ca00cf6a9e-node-log\") pod \"ovnkube-node-prjxn\" (UID: \"ca452469-23ef-4ec3-a6f1-f3ca00cf6a9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-prjxn"
Jul 06 14:19:40.308083 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.306552    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ca452469-23ef-4ec3-a6f1-f3ca00cf6a9e-host-slash\") pod \"ovnkube-node-prjxn\" (UID: \"ca452469-23ef-4ec3-a6f1-f3ca00cf6a9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-prjxn"
Jul 06 14:19:40.308083 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.306460    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ca452469-23ef-4ec3-a6f1-f3ca00cf6a9e-host-run-ovn-kubernetes\") pod \"ovnkube-node-prjxn\" (UID: \"ca452469-23ef-4ec3-a6f1-f3ca00cf6a9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-prjxn"
Jul 06 14:19:40.308083 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.306572    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ca452469-23ef-4ec3-a6f1-f3ca00cf6a9e-etc-openvswitch\") pod \"ovnkube-node-prjxn\" (UID: \"ca452469-23ef-4ec3-a6f1-f3ca00cf6a9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-prjxn"
Jul 06 14:19:40.308083 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.306573    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ca452469-23ef-4ec3-a6f1-f3ca00cf6a9e-host-cni-netd\") pod \"ovnkube-node-prjxn\" (UID: \"ca452469-23ef-4ec3-a6f1-f3ca00cf6a9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-prjxn"
Jul 06 14:19:40.308083 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.306617    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gl4bd\" (UniqueName: \"kubernetes.io/projected/ca452469-23ef-4ec3-a6f1-f3ca00cf6a9e-kube-api-access-gl4bd\") pod \"ovnkube-node-prjxn\" (UID: \"ca452469-23ef-4ec3-a6f1-f3ca00cf6a9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-prjxn"
Jul 06 14:19:40.308083 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.306603    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ca452469-23ef-4ec3-a6f1-f3ca00cf6a9e-node-log\") pod \"ovnkube-node-prjxn\" (UID: \"ca452469-23ef-4ec3-a6f1-f3ca00cf6a9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-prjxn"
Jul 06 14:19:40.308083 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.306628    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ca452469-23ef-4ec3-a6f1-f3ca00cf6a9e-host-cni-netd\") pod \"ovnkube-node-prjxn\" (UID: \"ca452469-23ef-4ec3-a6f1-f3ca00cf6a9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-prjxn"
Jul 06 14:19:40.308083 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.306689    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5cfea822-c324-4a7d-ab24-c8a48da58bcf-host-slash\") pod \"iptables-alerter-b9rzt\" (UID: \"5cfea822-c324-4a7d-ab24-c8a48da58bcf\") " pod="openshift-network-operator/iptables-alerter-b9rzt"
Jul 06 14:19:40.309018 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.306742    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/ca517334-9c07-4288-a0af-df1a8fc631dd-etc-sysctl-d\") pod \"tuned-8br6l\" (UID: \"ca517334-9c07-4288-a0af-df1a8fc631dd\") " pod="openshift-cluster-node-tuning-operator/tuned-8br6l"
Jul 06 14:19:40.309018 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.306764    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5cfea822-c324-4a7d-ab24-c8a48da58bcf-host-slash\") pod \"iptables-alerter-b9rzt\" (UID: \"5cfea822-c324-4a7d-ab24-c8a48da58bcf\") " pod="openshift-network-operator/iptables-alerter-b9rzt"
Jul 06 14:19:40.309018 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.306768    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ef2ced35-fb83-46cb-9e31-2c0814bd5db2-serviceca\") pod \"node-ca-pr7f9\" (UID: \"ef2ced35-fb83-46cb-9e31-2c0814bd5db2\") " pod="openshift-image-registry/node-ca-pr7f9"
Jul 06 14:19:40.309018 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.306807    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pbmnn\" (UniqueName: \"kubernetes.io/projected/ef2ced35-fb83-46cb-9e31-2c0814bd5db2-kube-api-access-pbmnn\") pod \"node-ca-pr7f9\" (UID: \"ef2ced35-fb83-46cb-9e31-2c0814bd5db2\") " pod="openshift-image-registry/node-ca-pr7f9"
Jul 06 14:19:40.309018 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.306832    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7282da78-ea16-410d-902a-f908b4d5b1fc-cnibin\") pod \"multus-additional-cni-plugins-7b8n5\" (UID: \"7282da78-ea16-410d-902a-f908b4d5b1fc\") " pod="openshift-multus/multus-additional-cni-plugins-7b8n5"
Jul 06 14:19:40.309018 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.306854    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7282da78-ea16-410d-902a-f908b4d5b1fc-tuning-conf-dir\") pod \"multus-additional-cni-plugins-7b8n5\" (UID: \"7282da78-ea16-410d-902a-f908b4d5b1fc\") " pod="openshift-multus/multus-additional-cni-plugins-7b8n5"
Jul 06 14:19:40.309018 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.306878    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ca517334-9c07-4288-a0af-df1a8fc631dd-var-lib-kubelet\") pod \"tuned-8br6l\" (UID: \"ca517334-9c07-4288-a0af-df1a8fc631dd\") " pod="openshift-cluster-node-tuning-operator/tuned-8br6l"
Jul 06 14:19:40.309018 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.306896    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/ca517334-9c07-4288-a0af-df1a8fc631dd-etc-sysctl-d\") pod \"tuned-8br6l\" (UID: \"ca517334-9c07-4288-a0af-df1a8fc631dd\") " pod="openshift-cluster-node-tuning-operator/tuned-8br6l"
Jul 06 14:19:40.309018 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.306902    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/ca517334-9c07-4288-a0af-df1a8fc631dd-run\") pod \"tuned-8br6l\" (UID: \"ca517334-9c07-4288-a0af-df1a8fc631dd\") " pod="openshift-cluster-node-tuning-operator/tuned-8br6l"
Jul 06 14:19:40.309018 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.306921    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7282da78-ea16-410d-902a-f908b4d5b1fc-cnibin\") pod \"multus-additional-cni-plugins-7b8n5\" (UID: \"7282da78-ea16-410d-902a-f908b4d5b1fc\") " pod="openshift-multus/multus-additional-cni-plugins-7b8n5"
Jul 06 14:19:40.309018 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.306934    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ca517334-9c07-4288-a0af-df1a8fc631dd-tmp\") pod \"tuned-8br6l\" (UID: \"ca517334-9c07-4288-a0af-df1a8fc631dd\") " pod="openshift-cluster-node-tuning-operator/tuned-8br6l"
Jul 06 14:19:40.309018 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.306946    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/ca517334-9c07-4288-a0af-df1a8fc631dd-run\") pod \"tuned-8br6l\" (UID: \"ca517334-9c07-4288-a0af-df1a8fc631dd\") " pod="openshift-cluster-node-tuning-operator/tuned-8br6l"
Jul 06 14:19:40.309018 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.306960    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a830d1ec-471c-4a9f-acd1-134c31193ad1-multus-cni-dir\") pod \"multus-jqxvb\" (UID: \"a830d1ec-471c-4a9f-acd1-134c31193ad1\") " pod="openshift-multus/multus-jqxvb"
Jul 06 14:19:40.309018 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.306987    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a830d1ec-471c-4a9f-acd1-134c31193ad1-host-run-netns\") pod \"multus-jqxvb\" (UID: \"a830d1ec-471c-4a9f-acd1-134c31193ad1\") " pod="openshift-multus/multus-jqxvb"
Jul 06 14:19:40.309018 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.306991    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ca517334-9c07-4288-a0af-df1a8fc631dd-var-lib-kubelet\") pod \"tuned-8br6l\" (UID: \"ca517334-9c07-4288-a0af-df1a8fc631dd\") " pod="openshift-cluster-node-tuning-operator/tuned-8br6l"
Jul 06 14:19:40.309018 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.306996    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/7282da78-ea16-410d-902a-f908b4d5b1fc-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-7b8n5\" (UID: \"7282da78-ea16-410d-902a-f908b4d5b1fc\") " pod="openshift-multus/multus-additional-cni-plugins-7b8n5"
Jul 06 14:19:40.309018 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.307012    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ca452469-23ef-4ec3-a6f1-f3ca00cf6a9e-systemd-units\") pod \"ovnkube-node-prjxn\" (UID: \"ca452469-23ef-4ec3-a6f1-f3ca00cf6a9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-prjxn"
Jul 06 14:19:40.309018 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.307049    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a830d1ec-471c-4a9f-acd1-134c31193ad1-multus-cni-dir\") pod \"multus-jqxvb\" (UID: \"a830d1ec-471c-4a9f-acd1-134c31193ad1\") " pod="openshift-multus/multus-jqxvb"
Jul 06 14:19:40.310007 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.307049    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ca452469-23ef-4ec3-a6f1-f3ca00cf6a9e-systemd-units\") pod \"ovnkube-node-prjxn\" (UID: \"ca452469-23ef-4ec3-a6f1-f3ca00cf6a9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-prjxn"
Jul 06 14:19:40.310007 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.307055    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ca452469-23ef-4ec3-a6f1-f3ca00cf6a9e-log-socket\") pod \"ovnkube-node-prjxn\" (UID: \"ca452469-23ef-4ec3-a6f1-f3ca00cf6a9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-prjxn"
Jul 06 14:19:40.310007 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.307081    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a830d1ec-471c-4a9f-acd1-134c31193ad1-host-run-netns\") pod \"multus-jqxvb\" (UID: \"a830d1ec-471c-4a9f-acd1-134c31193ad1\") " pod="openshift-multus/multus-jqxvb"
Jul 06 14:19:40.310007 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.307102    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/ca517334-9c07-4288-a0af-df1a8fc631dd-etc-sysconfig\") pod \"tuned-8br6l\" (UID: \"ca517334-9c07-4288-a0af-df1a8fc631dd\") " pod="openshift-cluster-node-tuning-operator/tuned-8br6l"
Jul 06 14:19:40.310007 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.307142    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/7282da78-ea16-410d-902a-f908b4d5b1fc-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-7b8n5\" (UID: \"7282da78-ea16-410d-902a-f908b4d5b1fc\") " pod="openshift-multus/multus-additional-cni-plugins-7b8n5"
Jul 06 14:19:40.310007 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.307147    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ca452469-23ef-4ec3-a6f1-f3ca00cf6a9e-log-socket\") pod \"ovnkube-node-prjxn\" (UID: \"ca452469-23ef-4ec3-a6f1-f3ca00cf6a9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-prjxn"
Jul 06 14:19:40.310007 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.307172    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a830d1ec-471c-4a9f-acd1-134c31193ad1-host-var-lib-cni-multus\") pod \"multus-jqxvb\" (UID: \"a830d1ec-471c-4a9f-acd1-134c31193ad1\") " pod="openshift-multus/multus-jqxvb"
Jul 06 14:19:40.310007 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.307205    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a830d1ec-471c-4a9f-acd1-134c31193ad1-os-release\") pod \"multus-jqxvb\" (UID: \"a830d1ec-471c-4a9f-acd1-134c31193ad1\") " pod="openshift-multus/multus-jqxvb"
Jul 06 14:19:40.310007 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.307208    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/ca517334-9c07-4288-a0af-df1a8fc631dd-etc-sysconfig\") pod \"tuned-8br6l\" (UID: \"ca517334-9c07-4288-a0af-df1a8fc631dd\") " pod="openshift-cluster-node-tuning-operator/tuned-8br6l"
Jul 06 14:19:40.310007 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.307234    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a830d1ec-471c-4a9f-acd1-134c31193ad1-multus-socket-dir-parent\") pod \"multus-jqxvb\" (UID: \"a830d1ec-471c-4a9f-acd1-134c31193ad1\") " pod="openshift-multus/multus-jqxvb"
Jul 06 14:19:40.310007 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.307252    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ef2ced35-fb83-46cb-9e31-2c0814bd5db2-serviceca\") pod \"node-ca-pr7f9\" (UID: \"ef2ced35-fb83-46cb-9e31-2c0814bd5db2\") " pod="openshift-image-registry/node-ca-pr7f9"
Jul 06 14:19:40.310007 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.307259    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a830d1ec-471c-4a9f-acd1-134c31193ad1-host-var-lib-cni-multus\") pod \"multus-jqxvb\" (UID: \"a830d1ec-471c-4a9f-acd1-134c31193ad1\") " pod="openshift-multus/multus-jqxvb"
Jul 06 14:19:40.310007 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.307262    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ca452469-23ef-4ec3-a6f1-f3ca00cf6a9e-run-openvswitch\") pod \"ovnkube-node-prjxn\" (UID: \"ca452469-23ef-4ec3-a6f1-f3ca00cf6a9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-prjxn"
Jul 06 14:19:40.310007 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.307234    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7282da78-ea16-410d-902a-f908b4d5b1fc-tuning-conf-dir\") pod \"multus-additional-cni-plugins-7b8n5\" (UID: \"7282da78-ea16-410d-902a-f908b4d5b1fc\") " pod="openshift-multus/multus-additional-cni-plugins-7b8n5"
Jul 06 14:19:40.310007 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.307334    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a830d1ec-471c-4a9f-acd1-134c31193ad1-os-release\") pod \"multus-jqxvb\" (UID: \"a830d1ec-471c-4a9f-acd1-134c31193ad1\") " pod="openshift-multus/multus-jqxvb"
Jul 06 14:19:40.310007 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.307334    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ca452469-23ef-4ec3-a6f1-f3ca00cf6a9e-run-openvswitch\") pod \"ovnkube-node-prjxn\" (UID: \"ca452469-23ef-4ec3-a6f1-f3ca00cf6a9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-prjxn"
Jul 06 14:19:40.310007 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.307402    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ca452469-23ef-4ec3-a6f1-f3ca00cf6a9e-host-cni-bin\") pod \"ovnkube-node-prjxn\" (UID: \"ca452469-23ef-4ec3-a6f1-f3ca00cf6a9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-prjxn"
Jul 06 14:19:40.310007 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.307437    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ca452469-23ef-4ec3-a6f1-f3ca00cf6a9e-ovnkube-config\") pod \"ovnkube-node-prjxn\" (UID: \"ca452469-23ef-4ec3-a6f1-f3ca00cf6a9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-prjxn"
Jul 06 14:19:40.310874 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.307439    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a830d1ec-471c-4a9f-acd1-134c31193ad1-multus-socket-dir-parent\") pod \"multus-jqxvb\" (UID: \"a830d1ec-471c-4a9f-acd1-134c31193ad1\") " pod="openshift-multus/multus-jqxvb"
Jul 06 14:19:40.310874 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.307475    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ca452469-23ef-4ec3-a6f1-f3ca00cf6a9e-ovnkube-script-lib\") pod \"ovnkube-node-prjxn\" (UID: \"ca452469-23ef-4ec3-a6f1-f3ca00cf6a9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-prjxn"
Jul 06 14:19:40.310874 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.307496    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ca452469-23ef-4ec3-a6f1-f3ca00cf6a9e-host-cni-bin\") pod \"ovnkube-node-prjxn\" (UID: \"ca452469-23ef-4ec3-a6f1-f3ca00cf6a9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-prjxn"
Jul 06 14:19:40.310874 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.307504    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7282da78-ea16-410d-902a-f908b4d5b1fc-os-release\") pod \"multus-additional-cni-plugins-7b8n5\" (UID: \"7282da78-ea16-410d-902a-f908b4d5b1fc\") " pod="openshift-multus/multus-additional-cni-plugins-7b8n5"
Jul 06 14:19:40.310874 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.307530    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f07d461c-9de3-43c8-b501-f50c0cd463eb-tmp-dir\") pod \"node-resolver-9tndp\" (UID: \"f07d461c-9de3-43c8-b501-f50c0cd463eb\") " pod="openshift-dns/node-resolver-9tndp"
Jul 06 14:19:40.310874 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.307554    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a830d1ec-471c-4a9f-acd1-134c31193ad1-hostroot\") pod \"multus-jqxvb\" (UID: \"a830d1ec-471c-4a9f-acd1-134c31193ad1\") " pod="openshift-multus/multus-jqxvb"
Jul 06 14:19:40.310874 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.307582    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7282da78-ea16-410d-902a-f908b4d5b1fc-system-cni-dir\") pod \"multus-additional-cni-plugins-7b8n5\" (UID: \"7282da78-ea16-410d-902a-f908b4d5b1fc\") " pod="openshift-multus/multus-additional-cni-plugins-7b8n5"
Jul 06 14:19:40.310874 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.307610    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7282da78-ea16-410d-902a-f908b4d5b1fc-cni-binary-copy\") pod \"multus-additional-cni-plugins-7b8n5\" (UID: \"7282da78-ea16-410d-902a-f908b4d5b1fc\") " pod="openshift-multus/multus-additional-cni-plugins-7b8n5"
Jul 06 14:19:40.310874 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.307637    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ca452469-23ef-4ec3-a6f1-f3ca00cf6a9e-env-overrides\") pod \"ovnkube-node-prjxn\" (UID: \"ca452469-23ef-4ec3-a6f1-f3ca00cf6a9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-prjxn"
Jul 06 14:19:40.310874 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.307689    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/5cfea822-c324-4a7d-ab24-c8a48da58bcf-iptables-alerter-script\") pod \"iptables-alerter-b9rzt\" (UID: \"5cfea822-c324-4a7d-ab24-c8a48da58bcf\") " pod="openshift-network-operator/iptables-alerter-b9rzt"
Jul 06 14:19:40.310874 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.307707    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/7282da78-ea16-410d-902a-f908b4d5b1fc-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-7b8n5\" (UID: \"7282da78-ea16-410d-902a-f908b4d5b1fc\") " pod="openshift-multus/multus-additional-cni-plugins-7b8n5"
Jul 06 14:19:40.310874 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.307716    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ca452469-23ef-4ec3-a6f1-f3ca00cf6a9e-var-lib-openvswitch\") pod \"ovnkube-node-prjxn\" (UID: \"ca452469-23ef-4ec3-a6f1-f3ca00cf6a9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-prjxn"
Jul 06 14:19:40.310874 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.307746    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wpf65\" (UniqueName: \"kubernetes.io/projected/5cfea822-c324-4a7d-ab24-c8a48da58bcf-kube-api-access-wpf65\") pod \"iptables-alerter-b9rzt\" (UID: \"5cfea822-c324-4a7d-ab24-c8a48da58bcf\") " pod="openshift-network-operator/iptables-alerter-b9rzt"
Jul 06 14:19:40.310874 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.307773    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a830d1ec-471c-4a9f-acd1-134c31193ad1-multus-daemon-config\") pod \"multus-jqxvb\" (UID: \"a830d1ec-471c-4a9f-acd1-134c31193ad1\") " pod="openshift-multus/multus-jqxvb"
Jul 06 14:19:40.310874 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.307803    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mn7d6\" (UniqueName: \"kubernetes.io/projected/a830d1ec-471c-4a9f-acd1-134c31193ad1-kube-api-access-mn7d6\") pod \"multus-jqxvb\" (UID: \"a830d1ec-471c-4a9f-acd1-134c31193ad1\") " pod="openshift-multus/multus-jqxvb"
Jul 06 14:19:40.310874 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.307830    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ca452469-23ef-4ec3-a6f1-f3ca00cf6a9e-run-systemd\") pod \"ovnkube-node-prjxn\" (UID: \"ca452469-23ef-4ec3-a6f1-f3ca00cf6a9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-prjxn"
Jul 06 14:19:40.310874 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.307911    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ca452469-23ef-4ec3-a6f1-f3ca00cf6a9e-run-systemd\") pod \"ovnkube-node-prjxn\" (UID: \"ca452469-23ef-4ec3-a6f1-f3ca00cf6a9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-prjxn"
Jul 06 14:19:40.311623 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.307959    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7282da78-ea16-410d-902a-f908b4d5b1fc-system-cni-dir\") pod \"multus-additional-cni-plugins-7b8n5\" (UID: \"7282da78-ea16-410d-902a-f908b4d5b1fc\") " pod="openshift-multus/multus-additional-cni-plugins-7b8n5"
Jul 06 14:19:40.311623 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.308015    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ca452469-23ef-4ec3-a6f1-f3ca00cf6a9e-var-lib-openvswitch\") pod \"ovnkube-node-prjxn\" (UID: \"ca452469-23ef-4ec3-a6f1-f3ca00cf6a9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-prjxn"
Jul 06 14:19:40.311623 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.308029    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a830d1ec-471c-4a9f-acd1-134c31193ad1-hostroot\") pod \"multus-jqxvb\" (UID: \"a830d1ec-471c-4a9f-acd1-134c31193ad1\") " pod="openshift-multus/multus-jqxvb"
Jul 06 14:19:40.311623 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.308030    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ca452469-23ef-4ec3-a6f1-f3ca00cf6a9e-ovnkube-config\") pod \"ovnkube-node-prjxn\" (UID: \"ca452469-23ef-4ec3-a6f1-f3ca00cf6a9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-prjxn"
Jul 06 14:19:40.311623 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.308185    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ca452469-23ef-4ec3-a6f1-f3ca00cf6a9e-ovnkube-script-lib\") pod \"ovnkube-node-prjxn\" (UID: \"ca452469-23ef-4ec3-a6f1-f3ca00cf6a9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-prjxn"
Jul 06 14:19:40.311623 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.308266    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7282da78-ea16-410d-902a-f908b4d5b1fc-os-release\") pod \"multus-additional-cni-plugins-7b8n5\" (UID: \"7282da78-ea16-410d-902a-f908b4d5b1fc\") " pod="openshift-multus/multus-additional-cni-plugins-7b8n5"
Jul 06 14:19:40.311623 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.308550    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f07d461c-9de3-43c8-b501-f50c0cd463eb-tmp-dir\") pod \"node-resolver-9tndp\" (UID: \"f07d461c-9de3-43c8-b501-f50c0cd463eb\") " pod="openshift-dns/node-resolver-9tndp"
Jul 06 14:19:40.311623 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.308596    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7282da78-ea16-410d-902a-f908b4d5b1fc-cni-binary-copy\") pod \"multus-additional-cni-plugins-7b8n5\" (UID: \"7282da78-ea16-410d-902a-f908b4d5b1fc\") " pod="openshift-multus/multus-additional-cni-plugins-7b8n5"
Jul 06 14:19:40.311623 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.308710    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a830d1ec-471c-4a9f-acd1-134c31193ad1-multus-daemon-config\") pod \"multus-jqxvb\" (UID: \"a830d1ec-471c-4a9f-acd1-134c31193ad1\") " pod="openshift-multus/multus-jqxvb"
Jul 06 14:19:40.311623 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.308821    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/ca517334-9c07-4288-a0af-df1a8fc631dd-etc-tuned\") pod \"tuned-8br6l\" (UID: \"ca517334-9c07-4288-a0af-df1a8fc631dd\") " pod="openshift-cluster-node-tuning-operator/tuned-8br6l"
Jul 06 14:19:40.311623 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.308983    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ca452469-23ef-4ec3-a6f1-f3ca00cf6a9e-ovn-node-metrics-cert\") pod \"ovnkube-node-prjxn\" (UID: \"ca452469-23ef-4ec3-a6f1-f3ca00cf6a9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-prjxn"
Jul 06 14:19:40.311623 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.309099    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ca452469-23ef-4ec3-a6f1-f3ca00cf6a9e-env-overrides\") pod \"ovnkube-node-prjxn\" (UID: \"ca452469-23ef-4ec3-a6f1-f3ca00cf6a9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-prjxn"
Jul 06 14:19:40.311623 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.309149    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/5cfea822-c324-4a7d-ab24-c8a48da58bcf-iptables-alerter-script\") pod \"iptables-alerter-b9rzt\" (UID: \"5cfea822-c324-4a7d-ab24-c8a48da58bcf\") " pod="openshift-network-operator/iptables-alerter-b9rzt"
Jul 06 14:19:40.311623 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.311009    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ca517334-9c07-4288-a0af-df1a8fc631dd-tmp\") pod \"tuned-8br6l\" (UID: \"ca517334-9c07-4288-a0af-df1a8fc631dd\") " pod="openshift-cluster-node-tuning-operator/tuned-8br6l"
Jul 06 14:19:40.311623 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.311223    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/6a513fd4-439d-4806-afbc-6462321a6d29-agent-certs\") pod \"konnectivity-agent-qr4tr\" (UID: \"6a513fd4-439d-4806-afbc-6462321a6d29\") " pod="kube-system/konnectivity-agent-qr4tr"
Jul 06 14:19:40.315055 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:19:40.314820    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 06 14:19:40.315055 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:19:40.314867    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 06 14:19:40.315055 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:19:40.314883    2559 projected.go:196] Error preparing data for projected volume kube-api-access-2r8z4 for pod openshift-network-diagnostics/network-check-target-l87kx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 14:19:40.315055 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:19:40.314958    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/98382dda-aacf-4546-8a91-e6bd00632d71-kube-api-access-2r8z4 podName:98382dda-aacf-4546-8a91-e6bd00632d71 nodeName:}" failed. No retries permitted until 2026-07-06 14:19:40.814938186 +0000 UTC m=+3.122230254 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-2r8z4" (UniqueName: "kubernetes.io/projected/98382dda-aacf-4546-8a91-e6bd00632d71-kube-api-access-2r8z4") pod "network-check-target-l87kx" (UID: "98382dda-aacf-4546-8a91-e6bd00632d71") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 14:19:40.315702 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.315464    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kp8rb\" (UniqueName: \"kubernetes.io/projected/7282da78-ea16-410d-902a-f908b4d5b1fc-kube-api-access-kp8rb\") pod \"multus-additional-cni-plugins-7b8n5\" (UID: \"7282da78-ea16-410d-902a-f908b4d5b1fc\") " pod="openshift-multus/multus-additional-cni-plugins-7b8n5"
Jul 06 14:19:40.317165 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.317138    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nc82b\" (UniqueName: \"kubernetes.io/projected/f07d461c-9de3-43c8-b501-f50c0cd463eb-kube-api-access-nc82b\") pod \"node-resolver-9tndp\" (UID: \"f07d461c-9de3-43c8-b501-f50c0cd463eb\") " pod="openshift-dns/node-resolver-9tndp"
Jul 06 14:19:40.317830 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.317811    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gl4bd\" (UniqueName: \"kubernetes.io/projected/ca452469-23ef-4ec3-a6f1-f3ca00cf6a9e-kube-api-access-gl4bd\") pod \"ovnkube-node-prjxn\" (UID: \"ca452469-23ef-4ec3-a6f1-f3ca00cf6a9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-prjxn"
Jul 06 14:19:40.318223 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.318197    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pbmnn\" (UniqueName: \"kubernetes.io/projected/ef2ced35-fb83-46cb-9e31-2c0814bd5db2-kube-api-access-pbmnn\") pod \"node-ca-pr7f9\" (UID: \"ef2ced35-fb83-46cb-9e31-2c0814bd5db2\") " pod="openshift-image-registry/node-ca-pr7f9"
Jul 06 14:19:40.318601 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.318580    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wpf65\" (UniqueName: \"kubernetes.io/projected/5cfea822-c324-4a7d-ab24-c8a48da58bcf-kube-api-access-wpf65\") pod \"iptables-alerter-b9rzt\" (UID: \"5cfea822-c324-4a7d-ab24-c8a48da58bcf\") " pod="openshift-network-operator/iptables-alerter-b9rzt"
Jul 06 14:19:40.318876 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.318853    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zx8rz\" (UniqueName: \"kubernetes.io/projected/ca517334-9c07-4288-a0af-df1a8fc631dd-kube-api-access-zx8rz\") pod \"tuned-8br6l\" (UID: \"ca517334-9c07-4288-a0af-df1a8fc631dd\") " pod="openshift-cluster-node-tuning-operator/tuned-8br6l"
Jul 06 14:19:40.319079 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.319051    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mn7d6\" (UniqueName: \"kubernetes.io/projected/a830d1ec-471c-4a9f-acd1-134c31193ad1-kube-api-access-mn7d6\") pod \"multus-jqxvb\" (UID: \"a830d1ec-471c-4a9f-acd1-134c31193ad1\") " pod="openshift-multus/multus-jqxvb"
Jul 06 14:19:40.396557 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.396460    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-75cxd"
Jul 06 14:19:40.406706 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.406680    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-8br6l"
Jul 06 14:19:40.408322 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:40.408287    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0bff3f4_b4c2_4c2a_ad95_dc24f3b8c92d.slice/crio-6e7d1e74cb67f82aa384e8f969656ddf05c77f4323f9157be3d701b753aa348e WatchSource:0}: Error finding container 6e7d1e74cb67f82aa384e8f969656ddf05c77f4323f9157be3d701b753aa348e: Status 404 returned error can't find the container with id 6e7d1e74cb67f82aa384e8f969656ddf05c77f4323f9157be3d701b753aa348e
Jul 06 14:19:40.415770 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.415746    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-pr7f9"
Jul 06 14:19:40.416592 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:40.416541    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca517334_9c07_4288_a0af_df1a8fc631dd.slice/crio-ba8c4c4371b30c91286fb33aa6f850ed47a3e60afcf8d78ddd90c53d61fc10aa WatchSource:0}: Error finding container ba8c4c4371b30c91286fb33aa6f850ed47a3e60afcf8d78ddd90c53d61fc10aa: Status 404 returned error can't find the container with id ba8c4c4371b30c91286fb33aa6f850ed47a3e60afcf8d78ddd90c53d61fc10aa
Jul 06 14:19:40.420320 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.420297    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-qr4tr"
Jul 06 14:19:40.426335 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.426314    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-7b8n5"
Jul 06 14:19:40.427236 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:40.427207    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef2ced35_fb83_46cb_9e31_2c0814bd5db2.slice/crio-e54308c36ec47f6f87a9125875d84d7590825ab7ab2aae6b0a01128f15fc9460 WatchSource:0}: Error finding container e54308c36ec47f6f87a9125875d84d7590825ab7ab2aae6b0a01128f15fc9460: Status 404 returned error can't find the container with id e54308c36ec47f6f87a9125875d84d7590825ab7ab2aae6b0a01128f15fc9460
Jul 06 14:19:40.431016 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:40.430956    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a513fd4_439d_4806_afbc_6462321a6d29.slice/crio-979c9cea4d76043aa57d00681f3ad94fa709f06e21e87c70bf27c971f29c7a55 WatchSource:0}: Error finding container 979c9cea4d76043aa57d00681f3ad94fa709f06e21e87c70bf27c971f29c7a55: Status 404 returned error can't find the container with id 979c9cea4d76043aa57d00681f3ad94fa709f06e21e87c70bf27c971f29c7a55
Jul 06 14:19:40.432021 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.432001    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-prjxn"
Jul 06 14:19:40.437361 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:40.437338    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7282da78_ea16_410d_902a_f908b4d5b1fc.slice/crio-50ebc44cf0aa1d20d29b0f9bd093e42ebc65cabe39580dab61f65bbd5623a46e WatchSource:0}: Error finding container 50ebc44cf0aa1d20d29b0f9bd093e42ebc65cabe39580dab61f65bbd5623a46e: Status 404 returned error can't find the container with id 50ebc44cf0aa1d20d29b0f9bd093e42ebc65cabe39580dab61f65bbd5623a46e
Jul 06 14:19:40.438203 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.438182    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-b9rzt"
Jul 06 14:19:40.443498 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:40.443466    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca452469_23ef_4ec3_a6f1_f3ca00cf6a9e.slice/crio-4eaf2772b23ba99d249945f0c386470b4c977b95c33b1be126297424da2aa4cf WatchSource:0}: Error finding container 4eaf2772b23ba99d249945f0c386470b4c977b95c33b1be126297424da2aa4cf: Status 404 returned error can't find the container with id 4eaf2772b23ba99d249945f0c386470b4c977b95c33b1be126297424da2aa4cf
Jul 06 14:19:40.444386 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.444365    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-9tndp"
Jul 06 14:19:40.449758 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.449734    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-jqxvb"
Jul 06 14:19:40.450410 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:40.450375    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5cfea822_c324_4a7d_ab24_c8a48da58bcf.slice/crio-1104859a8cd06aaef71aaf3993059ee53666d7444a948474e6a61b3633c2181b WatchSource:0}: Error finding container 1104859a8cd06aaef71aaf3993059ee53666d7444a948474e6a61b3633c2181b: Status 404 returned error can't find the container with id 1104859a8cd06aaef71aaf3993059ee53666d7444a948474e6a61b3633c2181b
Jul 06 14:19:40.460382 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:40.460175    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf07d461c_9de3_43c8_b501_f50c0cd463eb.slice/crio-51de605360c804f2a32afa312b26321ac60222385d251d20c506d7a96822cb00 WatchSource:0}: Error finding container 51de605360c804f2a32afa312b26321ac60222385d251d20c506d7a96822cb00: Status 404 returned error can't find the container with id 51de605360c804f2a32afa312b26321ac60222385d251d20c506d7a96822cb00
Jul 06 14:19:40.463698 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:19:40.463669    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda830d1ec_471c_4a9f_acd1_134c31193ad1.slice/crio-4634698ba3699a6c280650f19f347438936883eb5bcee57cea4b3c022a948d74 WatchSource:0}: Error finding container 4634698ba3699a6c280650f19f347438936883eb5bcee57cea4b3c022a948d74: Status 404 returned error can't find the container with id 4634698ba3699a6c280650f19f347438936883eb5bcee57cea4b3c022a948d74
Jul 06 14:19:40.711537 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.711494    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1c05cc5f-44d6-46b6-947a-9d221f282365-metrics-certs\") pod \"network-metrics-daemon-v94gg\" (UID: \"1c05cc5f-44d6-46b6-947a-9d221f282365\") " pod="openshift-multus/network-metrics-daemon-v94gg"
Jul 06 14:19:40.711759 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.711566    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ab514f6f-ea23-486a-b065-aac1b24a5035-original-pull-secret\") pod \"global-pull-secret-syncer-qjsqm\" (UID: \"ab514f6f-ea23-486a-b065-aac1b24a5035\") " pod="kube-system/global-pull-secret-syncer-qjsqm"
Jul 06 14:19:40.711759 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:19:40.711644    2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 14:19:40.711986 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:19:40.711959    2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 06 14:19:40.712134 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:19:40.712092    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ab514f6f-ea23-486a-b065-aac1b24a5035-original-pull-secret podName:ab514f6f-ea23-486a-b065-aac1b24a5035 nodeName:}" failed. No retries permitted until 2026-07-06 14:19:41.712052608 +0000 UTC m=+4.019344677 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ab514f6f-ea23-486a-b065-aac1b24a5035-original-pull-secret") pod "global-pull-secret-syncer-qjsqm" (UID: "ab514f6f-ea23-486a-b065-aac1b24a5035") : object "kube-system"/"original-pull-secret" not registered
Jul 06 14:19:40.712134 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:19:40.712118    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1c05cc5f-44d6-46b6-947a-9d221f282365-metrics-certs podName:1c05cc5f-44d6-46b6-947a-9d221f282365 nodeName:}" failed. No retries permitted until 2026-07-06 14:19:41.712105946 +0000 UTC m=+4.019398014 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1c05cc5f-44d6-46b6-947a-9d221f282365-metrics-certs") pod "network-metrics-daemon-v94gg" (UID: "1c05cc5f-44d6-46b6-947a-9d221f282365") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 14:19:40.913473 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:40.913437    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2r8z4\" (UniqueName: \"kubernetes.io/projected/98382dda-aacf-4546-8a91-e6bd00632d71-kube-api-access-2r8z4\") pod \"network-check-target-l87kx\" (UID: \"98382dda-aacf-4546-8a91-e6bd00632d71\") " pod="openshift-network-diagnostics/network-check-target-l87kx"
Jul 06 14:19:40.913682 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:19:40.913575    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 06 14:19:40.913682 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:19:40.913594    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 06 14:19:40.913682 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:19:40.913625    2559 projected.go:196] Error preparing data for projected volume kube-api-access-2r8z4 for pod openshift-network-diagnostics/network-check-target-l87kx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 14:19:40.913682 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:19:40.913683    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/98382dda-aacf-4546-8a91-e6bd00632d71-kube-api-access-2r8z4 podName:98382dda-aacf-4546-8a91-e6bd00632d71 nodeName:}" failed. No retries permitted until 2026-07-06 14:19:41.913666077 +0000 UTC m=+4.220958146 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-2r8z4" (UniqueName: "kubernetes.io/projected/98382dda-aacf-4546-8a91-e6bd00632d71-kube-api-access-2r8z4") pod "network-check-target-l87kx" (UID: "98382dda-aacf-4546-8a91-e6bd00632d71") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 14:19:41.150145 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:41.150024    2559 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-05 14:14:40 +0000 UTC" deadline="2028-02-29 06:41:30.961259253 +0000 UTC"
Jul 06 14:19:41.150145 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:41.150083    2559 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14464h21m49.811181722s"
Jul 06 14:19:41.220782 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:41.220744    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qjsqm"
Jul 06 14:19:41.221490 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:19:41.220892    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qjsqm" podUID="ab514f6f-ea23-486a-b065-aac1b24a5035"
Jul 06 14:19:41.221490 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:41.220961    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l87kx"
Jul 06 14:19:41.221490 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:19:41.221073    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l87kx" podUID="98382dda-aacf-4546-8a91-e6bd00632d71"
Jul 06 14:19:41.233806 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:41.233768    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-9tndp" event={"ID":"f07d461c-9de3-43c8-b501-f50c0cd463eb","Type":"ContainerStarted","Data":"51de605360c804f2a32afa312b26321ac60222385d251d20c506d7a96822cb00"}
Jul 06 14:19:41.235078 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:41.235041    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7b8n5" event={"ID":"7282da78-ea16-410d-902a-f908b4d5b1fc","Type":"ContainerStarted","Data":"50ebc44cf0aa1d20d29b0f9bd093e42ebc65cabe39580dab61f65bbd5623a46e"}
Jul 06 14:19:41.236406 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:41.236366    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-qr4tr" event={"ID":"6a513fd4-439d-4806-afbc-6462321a6d29","Type":"ContainerStarted","Data":"979c9cea4d76043aa57d00681f3ad94fa709f06e21e87c70bf27c971f29c7a55"}
Jul 06 14:19:41.237713 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:41.237679    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-pr7f9" event={"ID":"ef2ced35-fb83-46cb-9e31-2c0814bd5db2","Type":"ContainerStarted","Data":"e54308c36ec47f6f87a9125875d84d7590825ab7ab2aae6b0a01128f15fc9460"}
Jul 06 14:19:41.239068 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:41.239043    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-8br6l" event={"ID":"ca517334-9c07-4288-a0af-df1a8fc631dd","Type":"ContainerStarted","Data":"ba8c4c4371b30c91286fb33aa6f850ed47a3e60afcf8d78ddd90c53d61fc10aa"}
Jul 06 14:19:41.240411 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:41.240380    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-75cxd" event={"ID":"d0bff3f4-b4c2-4c2a-ad95-dc24f3b8c92d","Type":"ContainerStarted","Data":"6e7d1e74cb67f82aa384e8f969656ddf05c77f4323f9157be3d701b753aa348e"}
Jul 06 14:19:41.242557 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:41.242532    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-jqxvb" event={"ID":"a830d1ec-471c-4a9f-acd1-134c31193ad1","Type":"ContainerStarted","Data":"4634698ba3699a6c280650f19f347438936883eb5bcee57cea4b3c022a948d74"}
Jul 06 14:19:41.244319 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:41.244267    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-b9rzt" event={"ID":"5cfea822-c324-4a7d-ab24-c8a48da58bcf","Type":"ContainerStarted","Data":"1104859a8cd06aaef71aaf3993059ee53666d7444a948474e6a61b3633c2181b"}
Jul 06 14:19:41.246076 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:41.246051    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-prjxn" event={"ID":"ca452469-23ef-4ec3-a6f1-f3ca00cf6a9e","Type":"ContainerStarted","Data":"4eaf2772b23ba99d249945f0c386470b4c977b95c33b1be126297424da2aa4cf"}
Jul 06 14:19:41.299068 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:41.299034    2559 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 06 14:19:41.721468 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:41.721428    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ab514f6f-ea23-486a-b065-aac1b24a5035-original-pull-secret\") pod \"global-pull-secret-syncer-qjsqm\" (UID: \"ab514f6f-ea23-486a-b065-aac1b24a5035\") " pod="kube-system/global-pull-secret-syncer-qjsqm"
Jul 06 14:19:41.721711 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:41.721538    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1c05cc5f-44d6-46b6-947a-9d221f282365-metrics-certs\") pod \"network-metrics-daemon-v94gg\" (UID: \"1c05cc5f-44d6-46b6-947a-9d221f282365\") " pod="openshift-multus/network-metrics-daemon-v94gg"
Jul 06 14:19:41.721711 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:19:41.721577    2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 06 14:19:41.721711 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:19:41.721670    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ab514f6f-ea23-486a-b065-aac1b24a5035-original-pull-secret podName:ab514f6f-ea23-486a-b065-aac1b24a5035 nodeName:}" failed. No retries permitted until 2026-07-06 14:19:43.721648282 +0000 UTC m=+6.028940344 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ab514f6f-ea23-486a-b065-aac1b24a5035-original-pull-secret") pod "global-pull-secret-syncer-qjsqm" (UID: "ab514f6f-ea23-486a-b065-aac1b24a5035") : object "kube-system"/"original-pull-secret" not registered
Jul 06 14:19:41.721711 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:19:41.721689    2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 14:19:41.721906 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:19:41.721747    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1c05cc5f-44d6-46b6-947a-9d221f282365-metrics-certs podName:1c05cc5f-44d6-46b6-947a-9d221f282365 nodeName:}" failed. No retries permitted until 2026-07-06 14:19:43.721730058 +0000 UTC m=+6.029022122 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1c05cc5f-44d6-46b6-947a-9d221f282365-metrics-certs") pod "network-metrics-daemon-v94gg" (UID: "1c05cc5f-44d6-46b6-947a-9d221f282365") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 14:19:41.924287 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:41.924244    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2r8z4\" (UniqueName: \"kubernetes.io/projected/98382dda-aacf-4546-8a91-e6bd00632d71-kube-api-access-2r8z4\") pod \"network-check-target-l87kx\" (UID: \"98382dda-aacf-4546-8a91-e6bd00632d71\") " pod="openshift-network-diagnostics/network-check-target-l87kx"
Jul 06 14:19:41.924487 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:19:41.924426    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 06 14:19:41.924487 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:19:41.924456    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 06 14:19:41.924487 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:19:41.924471    2559 projected.go:196] Error preparing data for projected volume kube-api-access-2r8z4 for pod openshift-network-diagnostics/network-check-target-l87kx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 14:19:41.924679 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:19:41.924542    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/98382dda-aacf-4546-8a91-e6bd00632d71-kube-api-access-2r8z4 podName:98382dda-aacf-4546-8a91-e6bd00632d71 nodeName:}" failed. No retries permitted until 2026-07-06 14:19:43.924522082 +0000 UTC m=+6.231814149 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-2r8z4" (UniqueName: "kubernetes.io/projected/98382dda-aacf-4546-8a91-e6bd00632d71-kube-api-access-2r8z4") pod "network-check-target-l87kx" (UID: "98382dda-aacf-4546-8a91-e6bd00632d71") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 14:19:42.151334 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:42.151242    2559 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-05 14:14:40 +0000 UTC" deadline="2027-12-08 07:36:29.548933297 +0000 UTC"
Jul 06 14:19:42.151334 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:42.151286    2559 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12473h16m47.39765155s"
Jul 06 14:19:42.223001 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:42.222967    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v94gg"
Jul 06 14:19:42.223524 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:19:42.223123    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v94gg" podUID="1c05cc5f-44d6-46b6-947a-9d221f282365"
Jul 06 14:19:43.221706 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:43.221660    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l87kx"
Jul 06 14:19:43.221920 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:19:43.221823    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l87kx" podUID="98382dda-aacf-4546-8a91-e6bd00632d71"
Jul 06 14:19:43.222636 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:43.222612    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qjsqm"
Jul 06 14:19:43.222787 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:19:43.222745    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qjsqm" podUID="ab514f6f-ea23-486a-b065-aac1b24a5035"
Jul 06 14:19:43.741254 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:43.741150    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ab514f6f-ea23-486a-b065-aac1b24a5035-original-pull-secret\") pod \"global-pull-secret-syncer-qjsqm\" (UID: \"ab514f6f-ea23-486a-b065-aac1b24a5035\") " pod="kube-system/global-pull-secret-syncer-qjsqm"
Jul 06 14:19:43.742044 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:43.741262    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1c05cc5f-44d6-46b6-947a-9d221f282365-metrics-certs\") pod \"network-metrics-daemon-v94gg\" (UID: \"1c05cc5f-44d6-46b6-947a-9d221f282365\") " pod="openshift-multus/network-metrics-daemon-v94gg"
Jul 06 14:19:43.742044 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:19:43.741347    2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 06 14:19:43.742044 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:19:43.741394    2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 14:19:43.742044 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:19:43.741431    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ab514f6f-ea23-486a-b065-aac1b24a5035-original-pull-secret podName:ab514f6f-ea23-486a-b065-aac1b24a5035 nodeName:}" failed. No retries permitted until 2026-07-06 14:19:47.741409848 +0000 UTC m=+10.048701914 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ab514f6f-ea23-486a-b065-aac1b24a5035-original-pull-secret") pod "global-pull-secret-syncer-qjsqm" (UID: "ab514f6f-ea23-486a-b065-aac1b24a5035") : object "kube-system"/"original-pull-secret" not registered
Jul 06 14:19:43.742044 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:19:43.741453    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1c05cc5f-44d6-46b6-947a-9d221f282365-metrics-certs podName:1c05cc5f-44d6-46b6-947a-9d221f282365 nodeName:}" failed. No retries permitted until 2026-07-06 14:19:47.741441926 +0000 UTC m=+10.048733995 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1c05cc5f-44d6-46b6-947a-9d221f282365-metrics-certs") pod "network-metrics-daemon-v94gg" (UID: "1c05cc5f-44d6-46b6-947a-9d221f282365") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 14:19:43.943431 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:43.943394    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2r8z4\" (UniqueName: \"kubernetes.io/projected/98382dda-aacf-4546-8a91-e6bd00632d71-kube-api-access-2r8z4\") pod \"network-check-target-l87kx\" (UID: \"98382dda-aacf-4546-8a91-e6bd00632d71\") " pod="openshift-network-diagnostics/network-check-target-l87kx"
Jul 06 14:19:43.943689 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:19:43.943591    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 06 14:19:43.943689 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:19:43.943627    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 06 14:19:43.943689 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:19:43.943643    2559 projected.go:196] Error preparing data for projected volume kube-api-access-2r8z4 for pod openshift-network-diagnostics/network-check-target-l87kx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 14:19:43.943885 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:19:43.943723    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/98382dda-aacf-4546-8a91-e6bd00632d71-kube-api-access-2r8z4 podName:98382dda-aacf-4546-8a91-e6bd00632d71 nodeName:}" failed. No retries permitted until 2026-07-06 14:19:47.943701973 +0000 UTC m=+10.250994038 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-2r8z4" (UniqueName: "kubernetes.io/projected/98382dda-aacf-4546-8a91-e6bd00632d71-kube-api-access-2r8z4") pod "network-check-target-l87kx" (UID: "98382dda-aacf-4546-8a91-e6bd00632d71") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 14:19:44.221126 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:44.221083    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v94gg"
Jul 06 14:19:44.221332 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:19:44.221239    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v94gg" podUID="1c05cc5f-44d6-46b6-947a-9d221f282365"
Jul 06 14:19:45.221524 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:45.221422    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qjsqm"
Jul 06 14:19:45.222047 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:19:45.221567    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qjsqm" podUID="ab514f6f-ea23-486a-b065-aac1b24a5035"
Jul 06 14:19:45.222047 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:45.222035    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l87kx"
Jul 06 14:19:45.222166 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:19:45.222131    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l87kx" podUID="98382dda-aacf-4546-8a91-e6bd00632d71"
Jul 06 14:19:46.221121 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:46.221078    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v94gg"
Jul 06 14:19:46.221316 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:19:46.221255    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v94gg" podUID="1c05cc5f-44d6-46b6-947a-9d221f282365"
Jul 06 14:19:47.220906 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:47.220870    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qjsqm"
Jul 06 14:19:47.221395 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:47.220881    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l87kx"
Jul 06 14:19:47.221395 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:19:47.221029    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qjsqm" podUID="ab514f6f-ea23-486a-b065-aac1b24a5035"
Jul 06 14:19:47.221395 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:19:47.221064    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l87kx" podUID="98382dda-aacf-4546-8a91-e6bd00632d71"
Jul 06 14:19:47.775833 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:47.775798    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1c05cc5f-44d6-46b6-947a-9d221f282365-metrics-certs\") pod \"network-metrics-daemon-v94gg\" (UID: \"1c05cc5f-44d6-46b6-947a-9d221f282365\") " pod="openshift-multus/network-metrics-daemon-v94gg"
Jul 06 14:19:47.776041 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:47.775858    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ab514f6f-ea23-486a-b065-aac1b24a5035-original-pull-secret\") pod \"global-pull-secret-syncer-qjsqm\" (UID: \"ab514f6f-ea23-486a-b065-aac1b24a5035\") " pod="kube-system/global-pull-secret-syncer-qjsqm"
Jul 06 14:19:47.776041 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:19:47.775984    2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 14:19:47.776153 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:19:47.776067    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1c05cc5f-44d6-46b6-947a-9d221f282365-metrics-certs podName:1c05cc5f-44d6-46b6-947a-9d221f282365 nodeName:}" failed. No retries permitted until 2026-07-06 14:19:55.7760462 +0000 UTC m=+18.083338266 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1c05cc5f-44d6-46b6-947a-9d221f282365-metrics-certs") pod "network-metrics-daemon-v94gg" (UID: "1c05cc5f-44d6-46b6-947a-9d221f282365") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 14:19:47.776153 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:19:47.775993    2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 06 14:19:47.776153 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:19:47.776141    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ab514f6f-ea23-486a-b065-aac1b24a5035-original-pull-secret podName:ab514f6f-ea23-486a-b065-aac1b24a5035 nodeName:}" failed. No retries permitted until 2026-07-06 14:19:55.776122239 +0000 UTC m=+18.083414307 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ab514f6f-ea23-486a-b065-aac1b24a5035-original-pull-secret") pod "global-pull-secret-syncer-qjsqm" (UID: "ab514f6f-ea23-486a-b065-aac1b24a5035") : object "kube-system"/"original-pull-secret" not registered
Jul 06 14:19:47.978203 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:47.978157    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2r8z4\" (UniqueName: \"kubernetes.io/projected/98382dda-aacf-4546-8a91-e6bd00632d71-kube-api-access-2r8z4\") pod \"network-check-target-l87kx\" (UID: \"98382dda-aacf-4546-8a91-e6bd00632d71\") " pod="openshift-network-diagnostics/network-check-target-l87kx"
Jul 06 14:19:47.978373 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:19:47.978335    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 06 14:19:47.978373 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:19:47.978357    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 06 14:19:47.978373 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:19:47.978371    2559 projected.go:196] Error preparing data for projected volume kube-api-access-2r8z4 for pod openshift-network-diagnostics/network-check-target-l87kx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 14:19:47.978548 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:19:47.978438    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/98382dda-aacf-4546-8a91-e6bd00632d71-kube-api-access-2r8z4 podName:98382dda-aacf-4546-8a91-e6bd00632d71 nodeName:}" failed. No retries permitted until 2026-07-06 14:19:55.978417761 +0000 UTC m=+18.285709823 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-2r8z4" (UniqueName: "kubernetes.io/projected/98382dda-aacf-4546-8a91-e6bd00632d71-kube-api-access-2r8z4") pod "network-check-target-l87kx" (UID: "98382dda-aacf-4546-8a91-e6bd00632d71") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 14:19:48.227694 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:48.224429    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v94gg"
Jul 06 14:19:48.227694 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:19:48.224604    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v94gg" podUID="1c05cc5f-44d6-46b6-947a-9d221f282365"
Jul 06 14:19:49.221242 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:49.221202    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l87kx"
Jul 06 14:19:49.221453 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:49.221212    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qjsqm"
Jul 06 14:19:49.221453 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:19:49.221331    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l87kx" podUID="98382dda-aacf-4546-8a91-e6bd00632d71"
Jul 06 14:19:49.221577 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:19:49.221482    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qjsqm" podUID="ab514f6f-ea23-486a-b065-aac1b24a5035"
Jul 06 14:19:50.225749 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:50.225496    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v94gg"
Jul 06 14:19:50.225749 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:19:50.225660    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v94gg" podUID="1c05cc5f-44d6-46b6-947a-9d221f282365"
Jul 06 14:19:51.221356 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:51.221320    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qjsqm"
Jul 06 14:19:51.221527 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:19:51.221448    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qjsqm" podUID="ab514f6f-ea23-486a-b065-aac1b24a5035"
Jul 06 14:19:51.221613 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:51.221535    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l87kx"
Jul 06 14:19:51.221675 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:19:51.221641    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l87kx" podUID="98382dda-aacf-4546-8a91-e6bd00632d71"
Jul 06 14:19:51.275726 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:51.275019    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-112.ec2.internal" event={"ID":"112772d63ce9100ee913123b6eaccc48","Type":"ContainerStarted","Data":"fa28be3e668a2fc9ac5faa46447df0f0ed046cb3209ef419f69ce9f97e8a4eb6"}
Jul 06 14:19:51.278243 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:51.278207    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-8br6l" event={"ID":"ca517334-9c07-4288-a0af-df1a8fc631dd","Type":"ContainerStarted","Data":"d54cf0d5ae7df942cd81df785f04daa08e216d0673c954b1aa07fc25ed432550"}
Jul 06 14:19:51.293683 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:51.293630    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-112.ec2.internal" podStartSLOduration=11.293614651 podStartE2EDuration="11.293614651s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 14:19:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:19:51.292642697 +0000 UTC m=+13.599934781" watchObservedRunningTime="2026-07-06 14:19:51.293614651 +0000 UTC m=+13.600906737"
Jul 06 14:19:51.311998 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:51.311917    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-8br6l" podStartSLOduration=1.667157542 podStartE2EDuration="12.31190547s" totalImagesPullingTime="10.644747928s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 14:19:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:19:51.311503817 +0000 UTC m=+13.618795899" watchObservedRunningTime="2026-07-06 14:19:51.31190547 +0000 UTC m=+13.619197548"
Jul 06 14:19:52.221407 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:52.221372    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v94gg"
Jul 06 14:19:52.221557 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:19:52.221519    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v94gg" podUID="1c05cc5f-44d6-46b6-947a-9d221f282365"
Jul 06 14:19:52.281923 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:52.281885    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-9tndp" event={"ID":"f07d461c-9de3-43c8-b501-f50c0cd463eb","Type":"ContainerStarted","Data":"ad47823c3f81ff94cb76cfab83cdc855a18698c2cd6dd9480420c6d11c1545ac"}
Jul 06 14:19:52.283684 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:52.283647    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7b8n5" event={"ID":"7282da78-ea16-410d-902a-f908b4d5b1fc","Type":"ContainerStarted","Data":"edd85212b61a5579dccc1b151b76391f3c83c91930c5cf8291f99232956d222f"}
Jul 06 14:19:52.285246 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:52.285211    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-qr4tr" event={"ID":"6a513fd4-439d-4806-afbc-6462321a6d29","Type":"ContainerStarted","Data":"020389628bf7cf8dbd25603d083d6a5dc9a31ef3b0475a429fb5cd879a0875f3"}
Jul 06 14:19:52.286782 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:52.286739    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-pr7f9" event={"ID":"ef2ced35-fb83-46cb-9e31-2c0814bd5db2","Type":"ContainerStarted","Data":"26ca3a1235787cbc7584a9ad143b0b3a071fa26c994c6445e2c1c3f59aa9b2ef"}
Jul 06 14:19:52.288177 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:52.288150    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-75cxd" event={"ID":"d0bff3f4-b4c2-4c2a-ad95-dc24f3b8c92d","Type":"ContainerStarted","Data":"9c0f2648d0fb931c83e5206b6f14c48bb1fa7095c877b0edcd82f80a960bc792"}
Jul 06 14:19:52.289669 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:52.289644    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-112.ec2.internal" event={"ID":"1ce6bbc5e87ed1b7fdb9af3b5ae20a50","Type":"ContainerStarted","Data":"46d47c1e120b13b6043ee5f2c5d3d43aac94ca80cda207514e65c4a6605a4acc"}
Jul 06 14:19:52.301491 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:52.301441    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-9tndp" podStartSLOduration=2.959817852 podStartE2EDuration="13.301422136s" totalImagesPullingTime="10.341604284s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 14:19:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:19:52.300234632 +0000 UTC m=+14.607526714" watchObservedRunningTime="2026-07-06 14:19:52.301422136 +0000 UTC m=+14.608714219"
Jul 06 14:19:52.314561 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:52.314485    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-pr7f9" podStartSLOduration=7.675945875 podStartE2EDuration="13.314471346s" totalImagesPullingTime="5.638525471s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 14:19:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:19:52.313939142 +0000 UTC m=+14.621231228" watchObservedRunningTime="2026-07-06 14:19:52.314471346 +0000 UTC m=+14.621763446"
Jul 06 14:19:52.328484 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:52.328437    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-qr4tr" podStartSLOduration=2.956788489 podStartE2EDuration="13.328418953s" totalImagesPullingTime="10.371630464s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:19:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:19:52.327908031 +0000 UTC m=+14.635200113" watchObservedRunningTime="2026-07-06 14:19:52.328418953 +0000 UTC m=+14.635711035"
Jul 06 14:19:53.144048 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:53.143799    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-qr4tr"
Jul 06 14:19:53.144433 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:53.144410    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-qr4tr"
Jul 06 14:19:53.221024 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:53.220898    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l87kx"
Jul 06 14:19:53.221214 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:19:53.221030    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l87kx" podUID="98382dda-aacf-4546-8a91-e6bd00632d71"
Jul 06 14:19:53.221214 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:53.221191    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qjsqm"
Jul 06 14:19:53.221326 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:19:53.221310    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qjsqm" podUID="ab514f6f-ea23-486a-b065-aac1b24a5035"
Jul 06 14:19:53.294697 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:53.294663    2559 generic.go:356] "Generic (PLEG): container finished" podID="7282da78-ea16-410d-902a-f908b4d5b1fc" containerID="edd85212b61a5579dccc1b151b76391f3c83c91930c5cf8291f99232956d222f" exitCode=0
Jul 06 14:19:53.295161 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:53.294752    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7b8n5" event={"ID":"7282da78-ea16-410d-902a-f908b4d5b1fc","Type":"ContainerDied","Data":"edd85212b61a5579dccc1b151b76391f3c83c91930c5cf8291f99232956d222f"}
Jul 06 14:19:53.298684 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:53.297912    2559 generic.go:356] "Generic (PLEG): container finished" podID="1ce6bbc5e87ed1b7fdb9af3b5ae20a50" containerID="46d47c1e120b13b6043ee5f2c5d3d43aac94ca80cda207514e65c4a6605a4acc" exitCode=0
Jul 06 14:19:53.298684 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:53.297990    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-112.ec2.internal" event={"ID":"1ce6bbc5e87ed1b7fdb9af3b5ae20a50","Type":"ContainerDied","Data":"46d47c1e120b13b6043ee5f2c5d3d43aac94ca80cda207514e65c4a6605a4acc"}
Jul 06 14:19:53.306953 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:53.305893    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-b9rzt" event={"ID":"5cfea822-c324-4a7d-ab24-c8a48da58bcf","Type":"ContainerStarted","Data":"39c0f37451a77ba5816fb9d34d7484b39db1e7034c517c088c576c4419b13277"}
Jul 06 14:19:53.306953 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:53.306877    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-qr4tr"
Jul 06 14:19:53.307651 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:53.307608    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-qr4tr"
Jul 06 14:19:53.337297 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:53.337235    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-b9rzt" podStartSLOduration=3.986229984 podStartE2EDuration="14.337215526s" totalImagesPullingTime="10.350985542s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 14:19:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:19:53.336938764 +0000 UTC m=+15.644230882" watchObservedRunningTime="2026-07-06 14:19:53.337215526 +0000 UTC m=+15.644507609"
Jul 06 14:19:53.598921 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:53.598873    2559 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 06 14:19:54.225371 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:54.224965    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v94gg"
Jul 06 14:19:54.225371 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:19:54.225102    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v94gg" podUID="1c05cc5f-44d6-46b6-947a-9d221f282365"
Jul 06 14:19:54.243825 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:54.243257    2559 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-06T14:19:53.598892786Z","UUID":"3baa6404-40f7-4230-ba4d-2f54d413a45d","Handler":null,"Name":"","Endpoint":""}
Jul 06 14:19:54.245114 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:54.245090    2559 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 06 14:19:54.245255 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:54.245181    2559 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 06 14:19:54.310029 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:54.309990    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-75cxd" event={"ID":"d0bff3f4-b4c2-4c2a-ad95-dc24f3b8c92d","Type":"ContainerStarted","Data":"4f68acf47a70fa3f1785365295e2e72d8c5a2fb0c22a10fd52cec884750392d1"}
Jul 06 14:19:54.314998 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:54.314969    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-112.ec2.internal" event={"ID":"1ce6bbc5e87ed1b7fdb9af3b5ae20a50","Type":"ContainerStarted","Data":"ef91f57a397b6d80b438712543120e78ed961ebd764ea368135717c17f2cceb8"}
Jul 06 14:19:54.336414 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:54.336356    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-112.ec2.internal" podStartSLOduration=14.336335884 podStartE2EDuration="14.336335884s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 14:19:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:19:54.336335696 +0000 UTC m=+16.643627783" watchObservedRunningTime="2026-07-06 14:19:54.336335884 +0000 UTC m=+16.643627969"
Jul 06 14:19:55.221379 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:55.221339    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qjsqm"
Jul 06 14:19:55.221379 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:55.221379    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l87kx"
Jul 06 14:19:55.221674 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:19:55.221458    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qjsqm" podUID="ab514f6f-ea23-486a-b065-aac1b24a5035"
Jul 06 14:19:55.221674 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:19:55.221629    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l87kx" podUID="98382dda-aacf-4546-8a91-e6bd00632d71"
Jul 06 14:19:55.843541 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:55.843509    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1c05cc5f-44d6-46b6-947a-9d221f282365-metrics-certs\") pod \"network-metrics-daemon-v94gg\" (UID: \"1c05cc5f-44d6-46b6-947a-9d221f282365\") " pod="openshift-multus/network-metrics-daemon-v94gg"
Jul 06 14:19:55.844157 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:55.843562    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ab514f6f-ea23-486a-b065-aac1b24a5035-original-pull-secret\") pod \"global-pull-secret-syncer-qjsqm\" (UID: \"ab514f6f-ea23-486a-b065-aac1b24a5035\") " pod="kube-system/global-pull-secret-syncer-qjsqm"
Jul 06 14:19:55.844157 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:19:55.843698    2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 06 14:19:55.844157 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:19:55.843699    2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 14:19:55.844157 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:19:55.843772    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ab514f6f-ea23-486a-b065-aac1b24a5035-original-pull-secret podName:ab514f6f-ea23-486a-b065-aac1b24a5035 nodeName:}" failed. No retries permitted until 2026-07-06 14:20:11.843747511 +0000 UTC m=+34.151039571 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ab514f6f-ea23-486a-b065-aac1b24a5035-original-pull-secret") pod "global-pull-secret-syncer-qjsqm" (UID: "ab514f6f-ea23-486a-b065-aac1b24a5035") : object "kube-system"/"original-pull-secret" not registered
Jul 06 14:19:55.844157 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:19:55.843787    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1c05cc5f-44d6-46b6-947a-9d221f282365-metrics-certs podName:1c05cc5f-44d6-46b6-947a-9d221f282365 nodeName:}" failed. No retries permitted until 2026-07-06 14:20:11.84378034 +0000 UTC m=+34.151072400 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1c05cc5f-44d6-46b6-947a-9d221f282365-metrics-certs") pod "network-metrics-daemon-v94gg" (UID: "1c05cc5f-44d6-46b6-947a-9d221f282365") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 14:19:56.045078 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:56.045042    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2r8z4\" (UniqueName: \"kubernetes.io/projected/98382dda-aacf-4546-8a91-e6bd00632d71-kube-api-access-2r8z4\") pod \"network-check-target-l87kx\" (UID: \"98382dda-aacf-4546-8a91-e6bd00632d71\") " pod="openshift-network-diagnostics/network-check-target-l87kx"
Jul 06 14:19:56.045303 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:19:56.045281    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 06 14:19:56.045390 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:19:56.045311    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 06 14:19:56.045390 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:19:56.045324    2559 projected.go:196] Error preparing data for projected volume kube-api-access-2r8z4 for pod openshift-network-diagnostics/network-check-target-l87kx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 14:19:56.045390 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:19:56.045387    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/98382dda-aacf-4546-8a91-e6bd00632d71-kube-api-access-2r8z4 podName:98382dda-aacf-4546-8a91-e6bd00632d71 nodeName:}" failed. No retries permitted until 2026-07-06 14:20:12.045369058 +0000 UTC m=+34.352661122 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-2r8z4" (UniqueName: "kubernetes.io/projected/98382dda-aacf-4546-8a91-e6bd00632d71-kube-api-access-2r8z4") pod "network-check-target-l87kx" (UID: "98382dda-aacf-4546-8a91-e6bd00632d71") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 14:19:56.220982 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:56.220919    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v94gg"
Jul 06 14:19:56.221337 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:19:56.221284    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v94gg" podUID="1c05cc5f-44d6-46b6-947a-9d221f282365"
Jul 06 14:19:57.220959 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:57.220921    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l87kx"
Jul 06 14:19:57.221516 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:57.220929    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qjsqm"
Jul 06 14:19:57.221516 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:19:57.221054    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l87kx" podUID="98382dda-aacf-4546-8a91-e6bd00632d71"
Jul 06 14:19:57.221516 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:19:57.221138    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qjsqm" podUID="ab514f6f-ea23-486a-b065-aac1b24a5035"
Jul 06 14:19:58.224468 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:58.224437    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v94gg"
Jul 06 14:19:58.224923 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:19:58.224564    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v94gg" podUID="1c05cc5f-44d6-46b6-947a-9d221f282365"
Jul 06 14:19:59.221138 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:59.221102    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l87kx"
Jul 06 14:19:59.221315 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:19:59.221120    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qjsqm"
Jul 06 14:19:59.221315 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:19:59.221222    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l87kx" podUID="98382dda-aacf-4546-8a91-e6bd00632d71"
Jul 06 14:19:59.221416 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:19:59.221315    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qjsqm" podUID="ab514f6f-ea23-486a-b065-aac1b24a5035"
Jul 06 14:20:00.224703 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:00.224677    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v94gg"
Jul 06 14:20:00.225196 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:20:00.224787    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v94gg" podUID="1c05cc5f-44d6-46b6-947a-9d221f282365"
Jul 06 14:20:01.221466 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:01.221426    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qjsqm"
Jul 06 14:20:01.221666 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:20:01.221546    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qjsqm" podUID="ab514f6f-ea23-486a-b065-aac1b24a5035"
Jul 06 14:20:01.221666 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:01.221605    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l87kx"
Jul 06 14:20:01.221776 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:20:01.221712    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l87kx" podUID="98382dda-aacf-4546-8a91-e6bd00632d71"
Jul 06 14:20:02.224040 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:02.224010    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v94gg"
Jul 06 14:20:02.224504 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:20:02.224121    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v94gg" podUID="1c05cc5f-44d6-46b6-947a-9d221f282365"
Jul 06 14:20:03.220945 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:03.220908    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qjsqm"
Jul 06 14:20:03.221137 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:20:03.221035    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qjsqm" podUID="ab514f6f-ea23-486a-b065-aac1b24a5035"
Jul 06 14:20:03.221137 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:03.221081    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l87kx"
Jul 06 14:20:03.221254 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:20:03.221202    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l87kx" podUID="98382dda-aacf-4546-8a91-e6bd00632d71"
Jul 06 14:20:04.221073 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:04.220986    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v94gg"
Jul 06 14:20:04.221514 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:20:04.221131    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v94gg" podUID="1c05cc5f-44d6-46b6-947a-9d221f282365"
Jul 06 14:20:05.221710 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:05.221517    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l87kx"
Jul 06 14:20:05.222120 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:05.221515    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qjsqm"
Jul 06 14:20:05.222120 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:20:05.221786    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l87kx" podUID="98382dda-aacf-4546-8a91-e6bd00632d71"
Jul 06 14:20:05.222120 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:20:05.221851    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qjsqm" podUID="ab514f6f-ea23-486a-b065-aac1b24a5035"
Jul 06 14:20:05.339044 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:05.339011    2559 generic.go:356] "Generic (PLEG): container finished" podID="7282da78-ea16-410d-902a-f908b4d5b1fc" containerID="6764e7f5ad85c18d47a7f6b36a8243f1028a7958ed18105d7e89d3824a2b3420" exitCode=0
Jul 06 14:20:05.339255 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:05.339127    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7b8n5" event={"ID":"7282da78-ea16-410d-902a-f908b4d5b1fc","Type":"ContainerDied","Data":"6764e7f5ad85c18d47a7f6b36a8243f1028a7958ed18105d7e89d3824a2b3420"}
Jul 06 14:20:05.341293 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:05.341259    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-75cxd" event={"ID":"d0bff3f4-b4c2-4c2a-ad95-dc24f3b8c92d","Type":"ContainerStarted","Data":"2fdfd25b5eaf4ecf56bdde84b43d3abcfa6f8cfaa98f5d1a5f5345f7ce365d0c"}
Jul 06 14:20:05.342729 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:05.342705    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-jqxvb" event={"ID":"a830d1ec-471c-4a9f-acd1-134c31193ad1","Type":"ContainerStarted","Data":"af3ae72679cecb9082f3828b9f863cb6d1bdd34a6365929cc2f7c1cb49544217"}
Jul 06 14:20:05.345760 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:05.345741    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-prjxn" event={"ID":"ca452469-23ef-4ec3-a6f1-f3ca00cf6a9e","Type":"ContainerStarted","Data":"7f8a97e2b66e165bea835d8e2a2c04c29d5400b1d71ad6b8fe428406e36f1a69"}
Jul 06 14:20:05.345858 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:05.345798    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-prjxn" event={"ID":"ca452469-23ef-4ec3-a6f1-f3ca00cf6a9e","Type":"ContainerStarted","Data":"c879e02b02bc6726451d5155cc04fdfdb5791ff79c4b3611f9479a50a4477cec"}
Jul 06 14:20:05.345858 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:05.345830    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-prjxn" event={"ID":"ca452469-23ef-4ec3-a6f1-f3ca00cf6a9e","Type":"ContainerStarted","Data":"43f32a216c5ea6ee4f3d84ed60d614c94d67916db4b36dadf7f9378ccaa8326c"}
Jul 06 14:20:05.345926 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:05.345858    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-prjxn" event={"ID":"ca452469-23ef-4ec3-a6f1-f3ca00cf6a9e","Type":"ContainerStarted","Data":"3e53d7688e3e498b474af885ae1a4462e9f3de89859aea95814417f722a39737"}
Jul 06 14:20:05.345926 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:05.345891    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-prjxn" event={"ID":"ca452469-23ef-4ec3-a6f1-f3ca00cf6a9e","Type":"ContainerStarted","Data":"f5470b4dd17877e4c208e31cae891c6a6b5fa23d6bd738fe1b615f8f9bde8161"}
Jul 06 14:20:05.345926 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:05.345920    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-prjxn" event={"ID":"ca452469-23ef-4ec3-a6f1-f3ca00cf6a9e","Type":"ContainerStarted","Data":"fcb4405234d0bce185407fcfd29ea00a886d904687169cd27e5da7c0b2af02ec"}
Jul 06 14:20:05.398241 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:05.398198    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-jqxvb" podStartSLOduration=2.61321464 podStartE2EDuration="26.398184818s" totalImagesPullingTime="23.784970178s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 14:19:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:20:05.397042836 +0000 UTC m=+27.704334919" watchObservedRunningTime="2026-07-06 14:20:05.398184818 +0000 UTC m=+27.705476899"
Jul 06 14:20:06.221262 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:06.221234    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v94gg"
Jul 06 14:20:06.221492 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:20:06.221338    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v94gg" podUID="1c05cc5f-44d6-46b6-947a-9d221f282365"
Jul 06 14:20:07.221327 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:07.221248    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qjsqm"
Jul 06 14:20:07.221883 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:20:07.221344    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qjsqm" podUID="ab514f6f-ea23-486a-b065-aac1b24a5035"
Jul 06 14:20:07.221883 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:07.221369    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l87kx"
Jul 06 14:20:07.221883 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:20:07.221414    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l87kx" podUID="98382dda-aacf-4546-8a91-e6bd00632d71"
Jul 06 14:20:07.353633 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:07.353596    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-prjxn" event={"ID":"ca452469-23ef-4ec3-a6f1-f3ca00cf6a9e","Type":"ContainerStarted","Data":"5502a9571c61ee9b2e8d80bcfe3c6c2af41d53b5d5c00bdbc792b684a8ce4e6a"}
Jul 06 14:20:07.355632 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:07.355602    2559 generic.go:356] "Generic (PLEG): container finished" podID="7282da78-ea16-410d-902a-f908b4d5b1fc" containerID="6f56cf52e705e93268dd6ea6448110fd8e00445ec261ee600391af2da6b6f385" exitCode=0
Jul 06 14:20:07.355769 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:07.355638    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7b8n5" event={"ID":"7282da78-ea16-410d-902a-f908b4d5b1fc","Type":"ContainerDied","Data":"6f56cf52e705e93268dd6ea6448110fd8e00445ec261ee600391af2da6b6f385"}
Jul 06 14:20:07.380604 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:07.380539    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-75cxd" podStartSLOduration=6.0061023 podStartE2EDuration="28.380525857s" totalImagesPullingTime="22.374423557s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 14:19:39 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:20:05.420259548 +0000 UTC m=+27.727551630" watchObservedRunningTime="2026-07-06 14:20:07.380525857 +0000 UTC m=+29.687817939"
Jul 06 14:20:08.221369 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:08.221337    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v94gg"
Jul 06 14:20:08.221968 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:20:08.221445    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v94gg" podUID="1c05cc5f-44d6-46b6-947a-9d221f282365"
Jul 06 14:20:09.221486 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:09.221450    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qjsqm"
Jul 06 14:20:09.221982 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:09.221450    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l87kx"
Jul 06 14:20:09.221982 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:20:09.221558    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qjsqm" podUID="ab514f6f-ea23-486a-b065-aac1b24a5035"
Jul 06 14:20:09.221982 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:20:09.221645    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l87kx" podUID="98382dda-aacf-4546-8a91-e6bd00632d71"
Jul 06 14:20:10.221173 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:10.220979    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v94gg"
Jul 06 14:20:10.221381 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:20:10.221250    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v94gg" podUID="1c05cc5f-44d6-46b6-947a-9d221f282365"
Jul 06 14:20:10.364863 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:10.364822    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-prjxn" event={"ID":"ca452469-23ef-4ec3-a6f1-f3ca00cf6a9e","Type":"ContainerStarted","Data":"93b92ec5a09d566ea9afae24b9ea5e29b700925ad6c2343cd1fb560f8899a0bf"}
Jul 06 14:20:10.365262 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:10.365066    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-prjxn"
Jul 06 14:20:10.365262 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:10.365129    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-prjxn"
Jul 06 14:20:10.367326 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:10.367302    2559 generic.go:356] "Generic (PLEG): container finished" podID="7282da78-ea16-410d-902a-f908b4d5b1fc" containerID="0fcbd7920e0823ab788d9dc972066f84f2d092052fa379951c048c20f27f6c03" exitCode=0
Jul 06 14:20:10.367500 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:10.367337    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7b8n5" event={"ID":"7282da78-ea16-410d-902a-f908b4d5b1fc","Type":"ContainerDied","Data":"0fcbd7920e0823ab788d9dc972066f84f2d092052fa379951c048c20f27f6c03"}
Jul 06 14:20:10.381322 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:10.381294    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-prjxn"
Jul 06 14:20:10.395176 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:10.395132    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-prjxn" podStartSLOduration=7.590521931 podStartE2EDuration="31.395119639s" totalImagesPullingTime="23.804597708s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 14:19:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:20:10.393678959 +0000 UTC m=+32.700971035" watchObservedRunningTime="2026-07-06 14:20:10.395119639 +0000 UTC m=+32.702411720"
Jul 06 14:20:11.221643 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:11.221604    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qjsqm"
Jul 06 14:20:11.221812 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:11.221649    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l87kx"
Jul 06 14:20:11.221812 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:20:11.221714    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qjsqm" podUID="ab514f6f-ea23-486a-b065-aac1b24a5035"
Jul 06 14:20:11.221812 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:20:11.221788    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l87kx" podUID="98382dda-aacf-4546-8a91-e6bd00632d71"
Jul 06 14:20:11.370421 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:11.370381    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-prjxn"
Jul 06 14:20:11.388668 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:11.388631    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-prjxn"
Jul 06 14:20:11.612931 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:11.612855    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-qjsqm"]
Jul 06 14:20:11.613096 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:11.612968    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qjsqm"
Jul 06 14:20:11.613096 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:20:11.613086    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qjsqm" podUID="ab514f6f-ea23-486a-b065-aac1b24a5035"
Jul 06 14:20:11.613446 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:11.613428    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-l87kx"]
Jul 06 14:20:11.613514 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:11.613503    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l87kx"
Jul 06 14:20:11.613612 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:20:11.613595    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l87kx" podUID="98382dda-aacf-4546-8a91-e6bd00632d71"
Jul 06 14:20:11.627311 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:11.627282    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-v94gg"]
Jul 06 14:20:11.627417 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:11.627382    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v94gg"
Jul 06 14:20:11.627477 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:20:11.627462    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v94gg" podUID="1c05cc5f-44d6-46b6-947a-9d221f282365"
Jul 06 14:20:11.862613 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:11.862554    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1c05cc5f-44d6-46b6-947a-9d221f282365-metrics-certs\") pod \"network-metrics-daemon-v94gg\" (UID: \"1c05cc5f-44d6-46b6-947a-9d221f282365\") " pod="openshift-multus/network-metrics-daemon-v94gg"
Jul 06 14:20:11.862613 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:11.862621    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ab514f6f-ea23-486a-b065-aac1b24a5035-original-pull-secret\") pod \"global-pull-secret-syncer-qjsqm\" (UID: \"ab514f6f-ea23-486a-b065-aac1b24a5035\") " pod="kube-system/global-pull-secret-syncer-qjsqm"
Jul 06 14:20:11.862826 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:20:11.862696    2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 14:20:11.862826 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:20:11.862707    2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 06 14:20:11.862826 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:20:11.862757    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ab514f6f-ea23-486a-b065-aac1b24a5035-original-pull-secret podName:ab514f6f-ea23-486a-b065-aac1b24a5035 nodeName:}" failed. No retries permitted until 2026-07-06 14:20:43.86274395 +0000 UTC m=+66.170036010 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ab514f6f-ea23-486a-b065-aac1b24a5035-original-pull-secret") pod "global-pull-secret-syncer-qjsqm" (UID: "ab514f6f-ea23-486a-b065-aac1b24a5035") : object "kube-system"/"original-pull-secret" not registered
Jul 06 14:20:11.862826 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:20:11.862771    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1c05cc5f-44d6-46b6-947a-9d221f282365-metrics-certs podName:1c05cc5f-44d6-46b6-947a-9d221f282365 nodeName:}" failed. No retries permitted until 2026-07-06 14:20:43.862764589 +0000 UTC m=+66.170056649 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1c05cc5f-44d6-46b6-947a-9d221f282365-metrics-certs") pod "network-metrics-daemon-v94gg" (UID: "1c05cc5f-44d6-46b6-947a-9d221f282365") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 14:20:12.064260 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:12.064224    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2r8z4\" (UniqueName: \"kubernetes.io/projected/98382dda-aacf-4546-8a91-e6bd00632d71-kube-api-access-2r8z4\") pod \"network-check-target-l87kx\" (UID: \"98382dda-aacf-4546-8a91-e6bd00632d71\") " pod="openshift-network-diagnostics/network-check-target-l87kx"
Jul 06 14:20:12.064432 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:20:12.064384    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 06 14:20:12.064432 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:20:12.064404    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 06 14:20:12.064432 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:20:12.064414    2559 projected.go:196] Error preparing data for projected volume kube-api-access-2r8z4 for pod openshift-network-diagnostics/network-check-target-l87kx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 14:20:12.064532 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:20:12.064462    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/98382dda-aacf-4546-8a91-e6bd00632d71-kube-api-access-2r8z4 podName:98382dda-aacf-4546-8a91-e6bd00632d71 nodeName:}" failed. No retries permitted until 2026-07-06 14:20:44.06444879 +0000 UTC m=+66.371740850 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-2r8z4" (UniqueName: "kubernetes.io/projected/98382dda-aacf-4546-8a91-e6bd00632d71-kube-api-access-2r8z4") pod "network-check-target-l87kx" (UID: "98382dda-aacf-4546-8a91-e6bd00632d71") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 14:20:13.221555 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:13.221514    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l87kx"
Jul 06 14:20:13.222005 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:13.221560    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qjsqm"
Jul 06 14:20:13.222005 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:13.221643    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v94gg"
Jul 06 14:20:13.222005 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:20:13.221658    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l87kx" podUID="98382dda-aacf-4546-8a91-e6bd00632d71"
Jul 06 14:20:13.222005 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:20:13.221782    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v94gg" podUID="1c05cc5f-44d6-46b6-947a-9d221f282365"
Jul 06 14:20:13.222005 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:20:13.221862    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qjsqm" podUID="ab514f6f-ea23-486a-b065-aac1b24a5035"
Jul 06 14:20:15.220896 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:15.220666    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qjsqm"
Jul 06 14:20:15.221232 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:15.220728    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v94gg"
Jul 06 14:20:15.221232 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:20:15.220928    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qjsqm" podUID="ab514f6f-ea23-486a-b065-aac1b24a5035"
Jul 06 14:20:15.221232 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:20:15.221013    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v94gg" podUID="1c05cc5f-44d6-46b6-947a-9d221f282365"
Jul 06 14:20:15.221232 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:15.220728    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l87kx"
Jul 06 14:20:15.221232 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:20:15.221094    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l87kx" podUID="98382dda-aacf-4546-8a91-e6bd00632d71"
Jul 06 14:20:16.951697 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:16.951665    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-112.ec2.internal" event="NodeReady"
Jul 06 14:20:16.952142 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:16.951826    2559 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 06 14:20:17.024351 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:17.024320    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-kfvv5"]
Jul 06 14:20:17.037096 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:17.037071    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-kfvv5"
Jul 06 14:20:17.038740 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:17.038709    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-7r8pk"]
Jul 06 14:20:17.046026 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:17.046005    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 06 14:20:17.046136 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:17.046005    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 06 14:20:17.046352 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:17.046339    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-tng92\""
Jul 06 14:20:17.050511 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:17.050491    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-7r8pk"
Jul 06 14:20:17.051358 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:17.051339    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-kfvv5"]
Jul 06 14:20:17.055830 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:20:17.055801    2559 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"default-dockercfg-9fk4j\" is forbidden: User \"system:node:ip-10-0-133-112.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-ingress-canary\": no relationship found between node 'ip-10-0-133-112.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-9fk4j\"" type="*v1.Secret"
Jul 06 14:20:17.055830 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:20:17.055801    2559 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"kube-root-ca.crt\" is forbidden: User \"system:node:ip-10-0-133-112.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-ingress-canary\": no relationship found between node 'ip-10-0-133-112.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap"
Jul 06 14:20:17.056028 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:20:17.056000    2559 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"canary-serving-cert\" is forbidden: User \"system:node:ip-10-0-133-112.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-ingress-canary\": no relationship found between node 'ip-10-0-133-112.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" type="*v1.Secret"
Jul 06 14:20:17.056096 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:20:17.056020    2559 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"openshift-service-ca.crt\" is forbidden: User \"system:node:ip-10-0-133-112.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-ingress-canary\": no relationship found between node 'ip-10-0-133-112.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap"
Jul 06 14:20:17.065451 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:17.065426    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-7r8pk"]
Jul 06 14:20:17.204455 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:17.204378    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/e1c99840-57b6-4e0b-8e6d-80bca474168d-config-volume\") pod \"dns-default-kfvv5\" (UID: \"e1c99840-57b6-4e0b-8e6d-80bca474168d\") " pod="openshift-dns/dns-default-kfvv5"
Jul 06 14:20:17.204455 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:17.204417    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e1c99840-57b6-4e0b-8e6d-80bca474168d-metrics-tls\") pod \"dns-default-kfvv5\" (UID: \"e1c99840-57b6-4e0b-8e6d-80bca474168d\") " pod="openshift-dns/dns-default-kfvv5"
Jul 06 14:20:17.204455 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:17.204437    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e1c99840-57b6-4e0b-8e6d-80bca474168d-tmp-dir\") pod \"dns-default-kfvv5\" (UID: \"e1c99840-57b6-4e0b-8e6d-80bca474168d\") " pod="openshift-dns/dns-default-kfvv5"
Jul 06 14:20:17.204455 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:17.204453    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5j69d\" (UniqueName: \"kubernetes.io/projected/e1c99840-57b6-4e0b-8e6d-80bca474168d-kube-api-access-5j69d\") pod \"dns-default-kfvv5\" (UID: \"e1c99840-57b6-4e0b-8e6d-80bca474168d\") " pod="openshift-dns/dns-default-kfvv5"
Jul 06 14:20:17.204742 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:17.204552    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e19cda1c-b504-43fa-8dfb-8f53092dd8d8-cert\") pod \"ingress-canary-7r8pk\" (UID: \"e19cda1c-b504-43fa-8dfb-8f53092dd8d8\") " pod="openshift-ingress-canary/ingress-canary-7r8pk"
Jul 06 14:20:17.204742 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:17.204613    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pq7dt\" (UniqueName: \"kubernetes.io/projected/e19cda1c-b504-43fa-8dfb-8f53092dd8d8-kube-api-access-pq7dt\") pod \"ingress-canary-7r8pk\" (UID: \"e19cda1c-b504-43fa-8dfb-8f53092dd8d8\") " pod="openshift-ingress-canary/ingress-canary-7r8pk"
Jul 06 14:20:17.221650 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:17.221624    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v94gg"
Jul 06 14:20:17.221730 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:17.221624    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l87kx"
Jul 06 14:20:17.221904 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:17.221624    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qjsqm"
Jul 06 14:20:17.225827 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:17.225806    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 06 14:20:17.226113 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:17.226097    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 06 14:20:17.226294 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:17.226282    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 06 14:20:17.226362 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:17.226347    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-nfstm\""
Jul 06 14:20:17.226417 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:17.226403    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 06 14:20:17.226465 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:17.226450    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-74wbt\""
Jul 06 14:20:17.305326 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:17.305297    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pq7dt\" (UniqueName: \"kubernetes.io/projected/e19cda1c-b504-43fa-8dfb-8f53092dd8d8-kube-api-access-pq7dt\") pod \"ingress-canary-7r8pk\" (UID: \"e19cda1c-b504-43fa-8dfb-8f53092dd8d8\") " pod="openshift-ingress-canary/ingress-canary-7r8pk"
Jul 06 14:20:17.305445 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:17.305350    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/e1c99840-57b6-4e0b-8e6d-80bca474168d-config-volume\") pod \"dns-default-kfvv5\" (UID: \"e1c99840-57b6-4e0b-8e6d-80bca474168d\") " pod="openshift-dns/dns-default-kfvv5"
Jul 06 14:20:17.305445 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:17.305381    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e1c99840-57b6-4e0b-8e6d-80bca474168d-metrics-tls\") pod \"dns-default-kfvv5\" (UID: \"e1c99840-57b6-4e0b-8e6d-80bca474168d\") " pod="openshift-dns/dns-default-kfvv5"
Jul 06 14:20:17.305523 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:20:17.305469    2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 06 14:20:17.305523 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:17.305488    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e1c99840-57b6-4e0b-8e6d-80bca474168d-tmp-dir\") pod \"dns-default-kfvv5\" (UID: \"e1c99840-57b6-4e0b-8e6d-80bca474168d\") " pod="openshift-dns/dns-default-kfvv5"
Jul 06 14:20:17.305628 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:20:17.305534    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e1c99840-57b6-4e0b-8e6d-80bca474168d-metrics-tls podName:e1c99840-57b6-4e0b-8e6d-80bca474168d nodeName:}" failed. No retries permitted until 2026-07-06 14:20:17.805514389 +0000 UTC m=+40.112806450 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e1c99840-57b6-4e0b-8e6d-80bca474168d-metrics-tls") pod "dns-default-kfvv5" (UID: "e1c99840-57b6-4e0b-8e6d-80bca474168d") : secret "dns-default-metrics-tls" not found
Jul 06 14:20:17.305628 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:17.305555    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5j69d\" (UniqueName: \"kubernetes.io/projected/e1c99840-57b6-4e0b-8e6d-80bca474168d-kube-api-access-5j69d\") pod \"dns-default-kfvv5\" (UID: \"e1c99840-57b6-4e0b-8e6d-80bca474168d\") " pod="openshift-dns/dns-default-kfvv5"
Jul 06 14:20:17.305734 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:17.305654    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e19cda1c-b504-43fa-8dfb-8f53092dd8d8-cert\") pod \"ingress-canary-7r8pk\" (UID: \"e19cda1c-b504-43fa-8dfb-8f53092dd8d8\") " pod="openshift-ingress-canary/ingress-canary-7r8pk"
Jul 06 14:20:17.305827 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:17.305805    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e1c99840-57b6-4e0b-8e6d-80bca474168d-tmp-dir\") pod \"dns-default-kfvv5\" (UID: \"e1c99840-57b6-4e0b-8e6d-80bca474168d\") " pod="openshift-dns/dns-default-kfvv5"
Jul 06 14:20:17.305911 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:17.305892    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/e1c99840-57b6-4e0b-8e6d-80bca474168d-config-volume\") pod \"dns-default-kfvv5\" (UID: \"e1c99840-57b6-4e0b-8e6d-80bca474168d\") " pod="openshift-dns/dns-default-kfvv5"
Jul 06 14:20:17.317858 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:17.317832    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5j69d\" (UniqueName: \"kubernetes.io/projected/e1c99840-57b6-4e0b-8e6d-80bca474168d-kube-api-access-5j69d\") pod \"dns-default-kfvv5\" (UID: \"e1c99840-57b6-4e0b-8e6d-80bca474168d\") " pod="openshift-dns/dns-default-kfvv5"
Jul 06 14:20:17.810107 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:17.810015    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e1c99840-57b6-4e0b-8e6d-80bca474168d-metrics-tls\") pod \"dns-default-kfvv5\" (UID: \"e1c99840-57b6-4e0b-8e6d-80bca474168d\") " pod="openshift-dns/dns-default-kfvv5"
Jul 06 14:20:17.810278 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:20:17.810167    2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 06 14:20:17.810278 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:20:17.810234    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e1c99840-57b6-4e0b-8e6d-80bca474168d-metrics-tls podName:e1c99840-57b6-4e0b-8e6d-80bca474168d nodeName:}" failed. No retries permitted until 2026-07-06 14:20:18.810218398 +0000 UTC m=+41.117510458 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e1c99840-57b6-4e0b-8e6d-80bca474168d-metrics-tls") pod "dns-default-kfvv5" (UID: "e1c99840-57b6-4e0b-8e6d-80bca474168d") : secret "dns-default-metrics-tls" not found
Jul 06 14:20:18.117929 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:18.117848    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 06 14:20:18.128488 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:18.128453    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e19cda1c-b504-43fa-8dfb-8f53092dd8d8-cert\") pod \"ingress-canary-7r8pk\" (UID: \"e19cda1c-b504-43fa-8dfb-8f53092dd8d8\") " pod="openshift-ingress-canary/ingress-canary-7r8pk"
Jul 06 14:20:18.194060 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:18.194026    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-9fk4j\""
Jul 06 14:20:18.314417 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:20:18.314367    2559 projected.go:291] Couldn't get configMap openshift-ingress-canary/kube-root-ca.crt: failed to sync configmap cache: timed out waiting for the condition
Jul 06 14:20:18.390428 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:18.390342    2559 generic.go:356] "Generic (PLEG): container finished" podID="7282da78-ea16-410d-902a-f908b4d5b1fc" containerID="014a99d75a6c02179a48dae3c58f164ee95ce586edf942f9ecb9bc38955da01d" exitCode=0
Jul 06 14:20:18.390428 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:18.390404    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7b8n5" event={"ID":"7282da78-ea16-410d-902a-f908b4d5b1fc","Type":"ContainerDied","Data":"014a99d75a6c02179a48dae3c58f164ee95ce586edf942f9ecb9bc38955da01d"}
Jul 06 14:20:18.410962 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:18.410930    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 06 14:20:18.415121 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:20:18.415100    2559 projected.go:196] Error preparing data for projected volume kube-api-access-pq7dt for pod openshift-ingress-canary/ingress-canary-7r8pk: failed to sync configmap cache: timed out waiting for the condition
Jul 06 14:20:18.415202 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:20:18.415191    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e19cda1c-b504-43fa-8dfb-8f53092dd8d8-kube-api-access-pq7dt podName:e19cda1c-b504-43fa-8dfb-8f53092dd8d8 nodeName:}" failed. No retries permitted until 2026-07-06 14:20:18.91517357 +0000 UTC m=+41.222465649 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-pq7dt" (UniqueName: "kubernetes.io/projected/e19cda1c-b504-43fa-8dfb-8f53092dd8d8-kube-api-access-pq7dt") pod "ingress-canary-7r8pk" (UID: "e19cda1c-b504-43fa-8dfb-8f53092dd8d8") : failed to sync configmap cache: timed out waiting for the condition
Jul 06 14:20:18.455738 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:18.455703    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 06 14:20:18.818718 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:18.818677    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e1c99840-57b6-4e0b-8e6d-80bca474168d-metrics-tls\") pod \"dns-default-kfvv5\" (UID: \"e1c99840-57b6-4e0b-8e6d-80bca474168d\") " pod="openshift-dns/dns-default-kfvv5"
Jul 06 14:20:18.818871 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:20:18.818829    2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 06 14:20:18.818944 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:20:18.818900    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e1c99840-57b6-4e0b-8e6d-80bca474168d-metrics-tls podName:e1c99840-57b6-4e0b-8e6d-80bca474168d nodeName:}" failed. No retries permitted until 2026-07-06 14:20:20.818885037 +0000 UTC m=+43.126177097 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e1c99840-57b6-4e0b-8e6d-80bca474168d-metrics-tls") pod "dns-default-kfvv5" (UID: "e1c99840-57b6-4e0b-8e6d-80bca474168d") : secret "dns-default-metrics-tls" not found
Jul 06 14:20:18.919958 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:18.919917    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pq7dt\" (UniqueName: \"kubernetes.io/projected/e19cda1c-b504-43fa-8dfb-8f53092dd8d8-kube-api-access-pq7dt\") pod \"ingress-canary-7r8pk\" (UID: \"e19cda1c-b504-43fa-8dfb-8f53092dd8d8\") " pod="openshift-ingress-canary/ingress-canary-7r8pk"
Jul 06 14:20:18.922414 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:18.922388    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pq7dt\" (UniqueName: \"kubernetes.io/projected/e19cda1c-b504-43fa-8dfb-8f53092dd8d8-kube-api-access-pq7dt\") pod \"ingress-canary-7r8pk\" (UID: \"e19cda1c-b504-43fa-8dfb-8f53092dd8d8\") " pod="openshift-ingress-canary/ingress-canary-7r8pk"
Jul 06 14:20:19.158752 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:19.158657    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-7r8pk"
Jul 06 14:20:19.347284 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:19.347071    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-7r8pk"]
Jul 06 14:20:19.358234 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:20:19.358204    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode19cda1c_b504_43fa_8dfb_8f53092dd8d8.slice/crio-3268f92af88d0e2aaa224d1e44ae1f94d614a85fe7b43ef2a5ef48c9714a89f1 WatchSource:0}: Error finding container 3268f92af88d0e2aaa224d1e44ae1f94d614a85fe7b43ef2a5ef48c9714a89f1: Status 404 returned error can't find the container with id 3268f92af88d0e2aaa224d1e44ae1f94d614a85fe7b43ef2a5ef48c9714a89f1
Jul 06 14:20:19.396078 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:19.396046    2559 generic.go:356] "Generic (PLEG): container finished" podID="7282da78-ea16-410d-902a-f908b4d5b1fc" containerID="a4973b4f797b5859dd7fbc4e09b5844f28949ec07ac677db43130f305df32648" exitCode=0
Jul 06 14:20:19.396267 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:19.396141    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7b8n5" event={"ID":"7282da78-ea16-410d-902a-f908b4d5b1fc","Type":"ContainerDied","Data":"a4973b4f797b5859dd7fbc4e09b5844f28949ec07ac677db43130f305df32648"}
Jul 06 14:20:19.397460 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:19.397437    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-7r8pk" event={"ID":"e19cda1c-b504-43fa-8dfb-8f53092dd8d8","Type":"ContainerStarted","Data":"3268f92af88d0e2aaa224d1e44ae1f94d614a85fe7b43ef2a5ef48c9714a89f1"}
Jul 06 14:20:20.403991 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:20.403957    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7b8n5" event={"ID":"7282da78-ea16-410d-902a-f908b4d5b1fc","Type":"ContainerStarted","Data":"03d08432c494d74ae91ea95895994466411dd4c03c8b62c4a8bdf079974add2b"}
Jul 06 14:20:20.431706 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:20.431653    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-7b8n5" podStartSLOduration=9.990296558 podStartE2EDuration="41.431634284s" totalImagesPullingTime="31.441337726s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 14:19:39 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:20:20.430271982 +0000 UTC m=+42.737564064" watchObservedRunningTime="2026-07-06 14:20:20.431634284 +0000 UTC m=+42.738926367"
Jul 06 14:20:20.834124 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:20.834088    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e1c99840-57b6-4e0b-8e6d-80bca474168d-metrics-tls\") pod \"dns-default-kfvv5\" (UID: \"e1c99840-57b6-4e0b-8e6d-80bca474168d\") " pod="openshift-dns/dns-default-kfvv5"
Jul 06 14:20:20.834299 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:20:20.834256    2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 06 14:20:20.834364 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:20:20.834351    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e1c99840-57b6-4e0b-8e6d-80bca474168d-metrics-tls podName:e1c99840-57b6-4e0b-8e6d-80bca474168d nodeName:}" failed. No retries permitted until 2026-07-06 14:20:24.834330156 +0000 UTC m=+47.141622219 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e1c99840-57b6-4e0b-8e6d-80bca474168d-metrics-tls") pod "dns-default-kfvv5" (UID: "e1c99840-57b6-4e0b-8e6d-80bca474168d") : secret "dns-default-metrics-tls" not found
Jul 06 14:20:22.411755 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:22.411721    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-7r8pk" event={"ID":"e19cda1c-b504-43fa-8dfb-8f53092dd8d8","Type":"ContainerStarted","Data":"96876eda3f8080741525c4a4e05e2a0d661337befc9de236874a17326a6c1992"}
Jul 06 14:20:22.431120 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:22.431062    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-7r8pk" podStartSLOduration=2.607174891 podStartE2EDuration="5.431045053s" totalImagesPullingTime="2.823870162s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:20:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:20:22.430543571 +0000 UTC m=+44.737835653" watchObservedRunningTime="2026-07-06 14:20:22.431045053 +0000 UTC m=+44.738337134"
Jul 06 14:20:24.862617 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:24.862554    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e1c99840-57b6-4e0b-8e6d-80bca474168d-metrics-tls\") pod \"dns-default-kfvv5\" (UID: \"e1c99840-57b6-4e0b-8e6d-80bca474168d\") " pod="openshift-dns/dns-default-kfvv5"
Jul 06 14:20:24.863147 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:20:24.862698    2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 06 14:20:24.863147 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:20:24.862765    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e1c99840-57b6-4e0b-8e6d-80bca474168d-metrics-tls podName:e1c99840-57b6-4e0b-8e6d-80bca474168d nodeName:}" failed. No retries permitted until 2026-07-06 14:20:32.862749725 +0000 UTC m=+55.170041786 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e1c99840-57b6-4e0b-8e6d-80bca474168d-metrics-tls") pod "dns-default-kfvv5" (UID: "e1c99840-57b6-4e0b-8e6d-80bca474168d") : secret "dns-default-metrics-tls" not found
Jul 06 14:20:32.917846 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:32.917785    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e1c99840-57b6-4e0b-8e6d-80bca474168d-metrics-tls\") pod \"dns-default-kfvv5\" (UID: \"e1c99840-57b6-4e0b-8e6d-80bca474168d\") " pod="openshift-dns/dns-default-kfvv5"
Jul 06 14:20:32.918337 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:20:32.917952    2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 06 14:20:32.918337 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:20:32.918034    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e1c99840-57b6-4e0b-8e6d-80bca474168d-metrics-tls podName:e1c99840-57b6-4e0b-8e6d-80bca474168d nodeName:}" failed. No retries permitted until 2026-07-06 14:20:48.918016948 +0000 UTC m=+71.225309008 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e1c99840-57b6-4e0b-8e6d-80bca474168d-metrics-tls") pod "dns-default-kfvv5" (UID: "e1c99840-57b6-4e0b-8e6d-80bca474168d") : secret "dns-default-metrics-tls" not found
Jul 06 14:20:41.385744 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:41.385712    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-q7t4k"]
Jul 06 14:20:41.388059 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:41.388035    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-q7t4k"
Jul 06 14:20:41.388853 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:41.388829    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6b894bb7d5-8nmsf"]
Jul 06 14:20:41.390384 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:41.390303    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6b894bb7d5-8nmsf"
Jul 06 14:20:41.390763 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:41.390745    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Jul 06 14:20:41.390841 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:41.390802    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Jul 06 14:20:41.390900 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:41.390803    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Jul 06 14:20:41.390900 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:41.390807    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Jul 06 14:20:41.390900 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:41.390884    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-sth46\""
Jul 06 14:20:41.393428 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:41.393393    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 06 14:20:41.393428 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:41.393404    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 06 14:20:41.393614 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:41.393466    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 06 14:20:41.393762 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:41.393746    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-xn2mh\""
Jul 06 14:20:41.398606 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:41.398587    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 06 14:20:41.400625 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:41.400602    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-q7t4k"]
Jul 06 14:20:41.404897 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:41.404877    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6b894bb7d5-8nmsf"]
Jul 06 14:20:41.484766 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:41.484734    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/606a9ffc-10e9-4114-ad7b-752ab0272a0d-bound-sa-token\") pod \"image-registry-6b894bb7d5-8nmsf\" (UID: \"606a9ffc-10e9-4114-ad7b-752ab0272a0d\") " pod="openshift-image-registry/image-registry-6b894bb7d5-8nmsf"
Jul 06 14:20:41.484766 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:41.484768    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/606a9ffc-10e9-4114-ad7b-752ab0272a0d-image-registry-private-configuration\") pod \"image-registry-6b894bb7d5-8nmsf\" (UID: \"606a9ffc-10e9-4114-ad7b-752ab0272a0d\") " pod="openshift-image-registry/image-registry-6b894bb7d5-8nmsf"
Jul 06 14:20:41.484999 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:41.484787    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/606a9ffc-10e9-4114-ad7b-752ab0272a0d-trusted-ca\") pod \"image-registry-6b894bb7d5-8nmsf\" (UID: \"606a9ffc-10e9-4114-ad7b-752ab0272a0d\") " pod="openshift-image-registry/image-registry-6b894bb7d5-8nmsf"
Jul 06 14:20:41.484999 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:41.484828    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-js2fr\" (UniqueName: \"kubernetes.io/projected/606a9ffc-10e9-4114-ad7b-752ab0272a0d-kube-api-access-js2fr\") pod \"image-registry-6b894bb7d5-8nmsf\" (UID: \"606a9ffc-10e9-4114-ad7b-752ab0272a0d\") " pod="openshift-image-registry/image-registry-6b894bb7d5-8nmsf"
Jul 06 14:20:41.484999 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:41.484878    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/aecb0b51-89db-4843-9c3c-960ac76a1697-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-q7t4k\" (UID: \"aecb0b51-89db-4843-9c3c-960ac76a1697\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-q7t4k"
Jul 06 14:20:41.484999 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:41.484895    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8l6xb\" (UniqueName: \"kubernetes.io/projected/aecb0b51-89db-4843-9c3c-960ac76a1697-kube-api-access-8l6xb\") pod \"kube-storage-version-migrator-operator-846bf668f7-q7t4k\" (UID: \"aecb0b51-89db-4843-9c3c-960ac76a1697\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-q7t4k"
Jul 06 14:20:41.484999 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:41.484919    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/606a9ffc-10e9-4114-ad7b-752ab0272a0d-registry-certificates\") pod \"image-registry-6b894bb7d5-8nmsf\" (UID: \"606a9ffc-10e9-4114-ad7b-752ab0272a0d\") " pod="openshift-image-registry/image-registry-6b894bb7d5-8nmsf"
Jul 06 14:20:41.484999 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:41.484939    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/606a9ffc-10e9-4114-ad7b-752ab0272a0d-installation-pull-secrets\") pod \"image-registry-6b894bb7d5-8nmsf\" (UID: \"606a9ffc-10e9-4114-ad7b-752ab0272a0d\") " pod="openshift-image-registry/image-registry-6b894bb7d5-8nmsf"
Jul 06 14:20:41.484999 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:41.484959    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/aecb0b51-89db-4843-9c3c-960ac76a1697-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-q7t4k\" (UID: \"aecb0b51-89db-4843-9c3c-960ac76a1697\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-q7t4k"
Jul 06 14:20:41.484999 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:41.484988    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/606a9ffc-10e9-4114-ad7b-752ab0272a0d-registry-tls\") pod \"image-registry-6b894bb7d5-8nmsf\" (UID: \"606a9ffc-10e9-4114-ad7b-752ab0272a0d\") " pod="openshift-image-registry/image-registry-6b894bb7d5-8nmsf"
Jul 06 14:20:41.485252 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:41.485004    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/606a9ffc-10e9-4114-ad7b-752ab0272a0d-ca-trust-extracted\") pod \"image-registry-6b894bb7d5-8nmsf\" (UID: \"606a9ffc-10e9-4114-ad7b-752ab0272a0d\") " pod="openshift-image-registry/image-registry-6b894bb7d5-8nmsf"
Jul 06 14:20:41.488151 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:41.488111    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-5t8dp"]
Jul 06 14:20:41.490641 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:41.490618    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-5t8dp"
Jul 06 14:20:41.492870 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:41.492852    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-7b2g8\""
Jul 06 14:20:41.510304 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:41.510273    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-5t8dp"]
Jul 06 14:20:41.585941 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:41.585909    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/606a9ffc-10e9-4114-ad7b-752ab0272a0d-bound-sa-token\") pod \"image-registry-6b894bb7d5-8nmsf\" (UID: \"606a9ffc-10e9-4114-ad7b-752ab0272a0d\") " pod="openshift-image-registry/image-registry-6b894bb7d5-8nmsf"
Jul 06 14:20:41.586106 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:41.585948    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/606a9ffc-10e9-4114-ad7b-752ab0272a0d-image-registry-private-configuration\") pod \"image-registry-6b894bb7d5-8nmsf\" (UID: \"606a9ffc-10e9-4114-ad7b-752ab0272a0d\") " pod="openshift-image-registry/image-registry-6b894bb7d5-8nmsf"
Jul 06 14:20:41.586106 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:41.585968    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/606a9ffc-10e9-4114-ad7b-752ab0272a0d-trusted-ca\") pod \"image-registry-6b894bb7d5-8nmsf\" (UID: \"606a9ffc-10e9-4114-ad7b-752ab0272a0d\") " pod="openshift-image-registry/image-registry-6b894bb7d5-8nmsf"
Jul 06 14:20:41.586106 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:41.586003    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-js2fr\" (UniqueName: \"kubernetes.io/projected/606a9ffc-10e9-4114-ad7b-752ab0272a0d-kube-api-access-js2fr\") pod \"image-registry-6b894bb7d5-8nmsf\" (UID: \"606a9ffc-10e9-4114-ad7b-752ab0272a0d\") " pod="openshift-image-registry/image-registry-6b894bb7d5-8nmsf"
Jul 06 14:20:41.586106 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:41.586053    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/aecb0b51-89db-4843-9c3c-960ac76a1697-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-q7t4k\" (UID: \"aecb0b51-89db-4843-9c3c-960ac76a1697\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-q7t4k"
Jul 06 14:20:41.586106 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:41.586081    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nmfm6\" (UniqueName: \"kubernetes.io/projected/fdb626d0-4fcb-42aa-8069-c1d7c8a76975-kube-api-access-nmfm6\") pod \"network-check-source-989f8b8dc-5t8dp\" (UID: \"fdb626d0-4fcb-42aa-8069-c1d7c8a76975\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-5t8dp"
Jul 06 14:20:41.586329 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:41.586145    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8l6xb\" (UniqueName: \"kubernetes.io/projected/aecb0b51-89db-4843-9c3c-960ac76a1697-kube-api-access-8l6xb\") pod \"kube-storage-version-migrator-operator-846bf668f7-q7t4k\" (UID: \"aecb0b51-89db-4843-9c3c-960ac76a1697\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-q7t4k"
Jul 06 14:20:41.586329 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:41.586291    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/606a9ffc-10e9-4114-ad7b-752ab0272a0d-registry-certificates\") pod \"image-registry-6b894bb7d5-8nmsf\" (UID: \"606a9ffc-10e9-4114-ad7b-752ab0272a0d\") " pod="openshift-image-registry/image-registry-6b894bb7d5-8nmsf"
Jul 06 14:20:41.586429 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:41.586336    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/606a9ffc-10e9-4114-ad7b-752ab0272a0d-installation-pull-secrets\") pod \"image-registry-6b894bb7d5-8nmsf\" (UID: \"606a9ffc-10e9-4114-ad7b-752ab0272a0d\") " pod="openshift-image-registry/image-registry-6b894bb7d5-8nmsf"
Jul 06 14:20:41.586429 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:41.586364    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/aecb0b51-89db-4843-9c3c-960ac76a1697-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-q7t4k\" (UID: \"aecb0b51-89db-4843-9c3c-960ac76a1697\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-q7t4k"
Jul 06 14:20:41.586429 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:41.586390    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/606a9ffc-10e9-4114-ad7b-752ab0272a0d-registry-tls\") pod \"image-registry-6b894bb7d5-8nmsf\" (UID: \"606a9ffc-10e9-4114-ad7b-752ab0272a0d\") " pod="openshift-image-registry/image-registry-6b894bb7d5-8nmsf"
Jul 06 14:20:41.586429 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:41.586416    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/606a9ffc-10e9-4114-ad7b-752ab0272a0d-ca-trust-extracted\") pod \"image-registry-6b894bb7d5-8nmsf\" (UID: \"606a9ffc-10e9-4114-ad7b-752ab0272a0d\") " pod="openshift-image-registry/image-registry-6b894bb7d5-8nmsf"
Jul 06 14:20:41.586812 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:41.586784    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/606a9ffc-10e9-4114-ad7b-752ab0272a0d-ca-trust-extracted\") pod \"image-registry-6b894bb7d5-8nmsf\" (UID: \"606a9ffc-10e9-4114-ad7b-752ab0272a0d\") " pod="openshift-image-registry/image-registry-6b894bb7d5-8nmsf"
Jul 06 14:20:41.586933 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:20:41.586889    2559 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 06 14:20:41.586933 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:20:41.586903    2559 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6b894bb7d5-8nmsf: secret "image-registry-tls" not found
Jul 06 14:20:41.587028 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:41.586931    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/606a9ffc-10e9-4114-ad7b-752ab0272a0d-registry-certificates\") pod \"image-registry-6b894bb7d5-8nmsf\" (UID: \"606a9ffc-10e9-4114-ad7b-752ab0272a0d\") " pod="openshift-image-registry/image-registry-6b894bb7d5-8nmsf"
Jul 06 14:20:41.587028 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:20:41.586958    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/606a9ffc-10e9-4114-ad7b-752ab0272a0d-registry-tls podName:606a9ffc-10e9-4114-ad7b-752ab0272a0d nodeName:}" failed. No retries permitted until 2026-07-06 14:20:42.086937937 +0000 UTC m=+64.394230016 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/606a9ffc-10e9-4114-ad7b-752ab0272a0d-registry-tls") pod "image-registry-6b894bb7d5-8nmsf" (UID: "606a9ffc-10e9-4114-ad7b-752ab0272a0d") : secret "image-registry-tls" not found
Jul 06 14:20:41.587122 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:41.587020    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/aecb0b51-89db-4843-9c3c-960ac76a1697-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-q7t4k\" (UID: \"aecb0b51-89db-4843-9c3c-960ac76a1697\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-q7t4k"
Jul 06 14:20:41.587210 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:41.587188    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/606a9ffc-10e9-4114-ad7b-752ab0272a0d-trusted-ca\") pod \"image-registry-6b894bb7d5-8nmsf\" (UID: \"606a9ffc-10e9-4114-ad7b-752ab0272a0d\") " pod="openshift-image-registry/image-registry-6b894bb7d5-8nmsf"
Jul 06 14:20:41.589781 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:41.589750    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/aecb0b51-89db-4843-9c3c-960ac76a1697-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-q7t4k\" (UID: \"aecb0b51-89db-4843-9c3c-960ac76a1697\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-q7t4k"
Jul 06 14:20:41.589893 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:41.589880    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/606a9ffc-10e9-4114-ad7b-752ab0272a0d-image-registry-private-configuration\") pod \"image-registry-6b894bb7d5-8nmsf\" (UID: \"606a9ffc-10e9-4114-ad7b-752ab0272a0d\") " pod="openshift-image-registry/image-registry-6b894bb7d5-8nmsf"
Jul 06 14:20:41.589990 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:41.589972    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/606a9ffc-10e9-4114-ad7b-752ab0272a0d-installation-pull-secrets\") pod \"image-registry-6b894bb7d5-8nmsf\" (UID: \"606a9ffc-10e9-4114-ad7b-752ab0272a0d\") " pod="openshift-image-registry/image-registry-6b894bb7d5-8nmsf"
Jul 06 14:20:41.592483 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:41.592460    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-g5w5m"]
Jul 06 14:20:41.594629 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:41.594613    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-68b757fb67-g5w5m"
Jul 06 14:20:41.598871 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:41.598844    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Jul 06 14:20:41.598871 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:41.598857    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Jul 06 14:20:41.599034 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:41.598924    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Jul 06 14:20:41.599034 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:41.598935    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Jul 06 14:20:41.599034 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:41.598954    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-rp8zx\""
Jul 06 14:20:41.605482 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:41.605457    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-2s94h"]
Jul 06 14:20:41.605630 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:41.605605    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Jul 06 14:20:41.607190 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:41.607172    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-w8t5x"]
Jul 06 14:20:41.607327 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:41.607312    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5c847dcfc4-2s94h"
Jul 06 14:20:41.608865 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:41.608847    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-g5w5m"]
Jul 06 14:20:41.608963 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:41.608925    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-w8t5x"
Jul 06 14:20:41.609923 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:41.609903    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Jul 06 14:20:41.611510 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:41.611487    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8l6xb\" (UniqueName: \"kubernetes.io/projected/aecb0b51-89db-4843-9c3c-960ac76a1697-kube-api-access-8l6xb\") pod \"kube-storage-version-migrator-operator-846bf668f7-q7t4k\" (UID: \"aecb0b51-89db-4843-9c3c-960ac76a1697\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-q7t4k"
Jul 06 14:20:41.612441 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:41.612424    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 06 14:20:41.612602 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:41.612585    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Jul 06 14:20:41.612678 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:41.612635    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 06 14:20:41.612824 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:41.612788    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-mkwr5\""
Jul 06 14:20:41.612988 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:41.612973    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-xttw2\""
Jul 06 14:20:41.613360 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:41.613340    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Jul 06 14:20:41.614150 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:41.614128    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Jul 06 14:20:41.614284 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:41.614193    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Jul 06 14:20:41.615507 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:41.615488    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/606a9ffc-10e9-4114-ad7b-752ab0272a0d-bound-sa-token\") pod \"image-registry-6b894bb7d5-8nmsf\" (UID: \"606a9ffc-10e9-4114-ad7b-752ab0272a0d\") " pod="openshift-image-registry/image-registry-6b894bb7d5-8nmsf"
Jul 06 14:20:41.616357 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:41.616337    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-js2fr\" (UniqueName: \"kubernetes.io/projected/606a9ffc-10e9-4114-ad7b-752ab0272a0d-kube-api-access-js2fr\") pod \"image-registry-6b894bb7d5-8nmsf\" (UID: \"606a9ffc-10e9-4114-ad7b-752ab0272a0d\") " pod="openshift-image-registry/image-registry-6b894bb7d5-8nmsf"
Jul 06 14:20:41.619153 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:41.618718    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Jul 06 14:20:41.619839 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:41.619769    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-w8t5x"]
Jul 06 14:20:41.624622 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:41.624601    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-2s94h"]
Jul 06 14:20:41.687092 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:41.687063    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e35b0247-7c4e-4bd1-bc28-c084dc40ffa1-serving-cert\") pod \"console-operator-68b757fb67-g5w5m\" (UID: \"e35b0247-7c4e-4bd1-bc28-c084dc40ffa1\") " pod="openshift-console-operator/console-operator-68b757fb67-g5w5m"
Jul 06 14:20:41.687265 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:41.687103    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/de6193cf-468b-4733-934b-71d909d33445-snapshots\") pod \"insights-operator-5c847dcfc4-2s94h\" (UID: \"de6193cf-468b-4733-934b-71d909d33445\") " pod="openshift-insights/insights-operator-5c847dcfc4-2s94h"
Jul 06 14:20:41.687265 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:41.687160    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/de6193cf-468b-4733-934b-71d909d33445-tmp\") pod \"insights-operator-5c847dcfc4-2s94h\" (UID: \"de6193cf-468b-4733-934b-71d909d33445\") " pod="openshift-insights/insights-operator-5c847dcfc4-2s94h"
Jul 06 14:20:41.687265 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:41.687188    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e35b0247-7c4e-4bd1-bc28-c084dc40ffa1-trusted-ca\") pod \"console-operator-68b757fb67-g5w5m\" (UID: \"e35b0247-7c4e-4bd1-bc28-c084dc40ffa1\") " pod="openshift-console-operator/console-operator-68b757fb67-g5w5m"
Jul 06 14:20:41.687265 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:41.687206    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/de6193cf-468b-4733-934b-71d909d33445-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-2s94h\" (UID: \"de6193cf-468b-4733-934b-71d909d33445\") " pod="openshift-insights/insights-operator-5c847dcfc4-2s94h"
Jul 06 14:20:41.687265 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:41.687254    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/de6193cf-468b-4733-934b-71d909d33445-serving-cert\") pod \"insights-operator-5c847dcfc4-2s94h\" (UID: \"de6193cf-468b-4733-934b-71d909d33445\") " pod="openshift-insights/insights-operator-5c847dcfc4-2s94h"
Jul 06 14:20:41.687456 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:41.687305    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e35b0247-7c4e-4bd1-bc28-c084dc40ffa1-config\") pod \"console-operator-68b757fb67-g5w5m\" (UID: \"e35b0247-7c4e-4bd1-bc28-c084dc40ffa1\") " pod="openshift-console-operator/console-operator-68b757fb67-g5w5m"
Jul 06 14:20:41.687456 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:41.687360    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/de6193cf-468b-4733-934b-71d909d33445-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-2s94h\" (UID: \"de6193cf-468b-4733-934b-71d909d33445\") " pod="openshift-insights/insights-operator-5c847dcfc4-2s94h"
Jul 06 14:20:41.687456 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:41.687387    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-828vn\" (UniqueName: \"kubernetes.io/projected/e35b0247-7c4e-4bd1-bc28-c084dc40ffa1-kube-api-access-828vn\") pod \"console-operator-68b757fb67-g5w5m\" (UID: \"e35b0247-7c4e-4bd1-bc28-c084dc40ffa1\") " pod="openshift-console-operator/console-operator-68b757fb67-g5w5m"
Jul 06 14:20:41.687456 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:41.687411    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nmfm6\" (UniqueName: \"kubernetes.io/projected/fdb626d0-4fcb-42aa-8069-c1d7c8a76975-kube-api-access-nmfm6\") pod \"network-check-source-989f8b8dc-5t8dp\" (UID: \"fdb626d0-4fcb-42aa-8069-c1d7c8a76975\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-5t8dp"
Jul 06 14:20:41.687456 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:41.687451    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-plldn\" (UniqueName: \"kubernetes.io/projected/de6193cf-468b-4733-934b-71d909d33445-kube-api-access-plldn\") pod \"insights-operator-5c847dcfc4-2s94h\" (UID: \"de6193cf-468b-4733-934b-71d909d33445\") " pod="openshift-insights/insights-operator-5c847dcfc4-2s94h"
Jul 06 14:20:41.687643 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:41.687543    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gt8c6\" (UniqueName: \"kubernetes.io/projected/a3b572f0-699a-4473-8804-ebac7a1e47d4-kube-api-access-gt8c6\") pod \"cluster-samples-operator-56fcf86bf-w8t5x\" (UID: \"a3b572f0-699a-4473-8804-ebac7a1e47d4\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-w8t5x"
Jul 06 14:20:41.687643 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:41.687603    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a3b572f0-699a-4473-8804-ebac7a1e47d4-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-w8t5x\" (UID: \"a3b572f0-699a-4473-8804-ebac7a1e47d4\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-w8t5x"
Jul 06 14:20:41.695697 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:41.695679    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 06 14:20:41.698444 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:41.698429    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-q7t4k"
Jul 06 14:20:41.706148 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:41.706131    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 06 14:20:41.717542 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:41.717519    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nmfm6\" (UniqueName: \"kubernetes.io/projected/fdb626d0-4fcb-42aa-8069-c1d7c8a76975-kube-api-access-nmfm6\") pod \"network-check-source-989f8b8dc-5t8dp\" (UID: \"fdb626d0-4fcb-42aa-8069-c1d7c8a76975\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-5t8dp"
Jul 06 14:20:41.788902 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:41.788873    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-plldn\" (UniqueName: \"kubernetes.io/projected/de6193cf-468b-4733-934b-71d909d33445-kube-api-access-plldn\") pod \"insights-operator-5c847dcfc4-2s94h\" (UID: \"de6193cf-468b-4733-934b-71d909d33445\") " pod="openshift-insights/insights-operator-5c847dcfc4-2s94h"
Jul 06 14:20:41.788902 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:41.788911    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gt8c6\" (UniqueName: \"kubernetes.io/projected/a3b572f0-699a-4473-8804-ebac7a1e47d4-kube-api-access-gt8c6\") pod \"cluster-samples-operator-56fcf86bf-w8t5x\" (UID: \"a3b572f0-699a-4473-8804-ebac7a1e47d4\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-w8t5x"
Jul 06 14:20:41.789162 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:41.788937    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a3b572f0-699a-4473-8804-ebac7a1e47d4-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-w8t5x\" (UID: \"a3b572f0-699a-4473-8804-ebac7a1e47d4\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-w8t5x"
Jul 06 14:20:41.789162 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:41.789042    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e35b0247-7c4e-4bd1-bc28-c084dc40ffa1-serving-cert\") pod \"console-operator-68b757fb67-g5w5m\" (UID: \"e35b0247-7c4e-4bd1-bc28-c084dc40ffa1\") " pod="openshift-console-operator/console-operator-68b757fb67-g5w5m"
Jul 06 14:20:41.789162 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:41.789098    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/de6193cf-468b-4733-934b-71d909d33445-snapshots\") pod \"insights-operator-5c847dcfc4-2s94h\" (UID: \"de6193cf-468b-4733-934b-71d909d33445\") " pod="openshift-insights/insights-operator-5c847dcfc4-2s94h"
Jul 06 14:20:41.789162 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:41.789145    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/de6193cf-468b-4733-934b-71d909d33445-tmp\") pod \"insights-operator-5c847dcfc4-2s94h\" (UID: \"de6193cf-468b-4733-934b-71d909d33445\") " pod="openshift-insights/insights-operator-5c847dcfc4-2s94h"
Jul 06 14:20:41.790352 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:41.789176    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e35b0247-7c4e-4bd1-bc28-c084dc40ffa1-trusted-ca\") pod \"console-operator-68b757fb67-g5w5m\" (UID: \"e35b0247-7c4e-4bd1-bc28-c084dc40ffa1\") " pod="openshift-console-operator/console-operator-68b757fb67-g5w5m"
Jul 06 14:20:41.790352 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:41.789199    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/de6193cf-468b-4733-934b-71d909d33445-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-2s94h\" (UID: \"de6193cf-468b-4733-934b-71d909d33445\") " pod="openshift-insights/insights-operator-5c847dcfc4-2s94h"
Jul 06 14:20:41.790352 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:41.789228    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/de6193cf-468b-4733-934b-71d909d33445-serving-cert\") pod \"insights-operator-5c847dcfc4-2s94h\" (UID: \"de6193cf-468b-4733-934b-71d909d33445\") " pod="openshift-insights/insights-operator-5c847dcfc4-2s94h"
Jul 06 14:20:41.790352 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:41.789273    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e35b0247-7c4e-4bd1-bc28-c084dc40ffa1-config\") pod \"console-operator-68b757fb67-g5w5m\" (UID: \"e35b0247-7c4e-4bd1-bc28-c084dc40ffa1\") " pod="openshift-console-operator/console-operator-68b757fb67-g5w5m"
Jul 06 14:20:41.790352 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:41.789334    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/de6193cf-468b-4733-934b-71d909d33445-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-2s94h\" (UID: \"de6193cf-468b-4733-934b-71d909d33445\") " pod="openshift-insights/insights-operator-5c847dcfc4-2s94h"
Jul 06 14:20:41.790352 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:41.789366    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-828vn\" (UniqueName: \"kubernetes.io/projected/e35b0247-7c4e-4bd1-bc28-c084dc40ffa1-kube-api-access-828vn\") pod \"console-operator-68b757fb67-g5w5m\" (UID: \"e35b0247-7c4e-4bd1-bc28-c084dc40ffa1\") " pod="openshift-console-operator/console-operator-68b757fb67-g5w5m"
Jul 06 14:20:41.790676 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:41.790559    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e35b0247-7c4e-4bd1-bc28-c084dc40ffa1-config\") pod \"console-operator-68b757fb67-g5w5m\" (UID: \"e35b0247-7c4e-4bd1-bc28-c084dc40ffa1\") " pod="openshift-console-operator/console-operator-68b757fb67-g5w5m"
Jul 06 14:20:41.790676 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:20:41.789046    2559 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 06 14:20:41.790779 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:20:41.790693    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3b572f0-699a-4473-8804-ebac7a1e47d4-samples-operator-tls podName:a3b572f0-699a-4473-8804-ebac7a1e47d4 nodeName:}" failed. No retries permitted until 2026-07-06 14:20:42.290669045 +0000 UTC m=+64.597961108 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/a3b572f0-699a-4473-8804-ebac7a1e47d4-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-w8t5x" (UID: "a3b572f0-699a-4473-8804-ebac7a1e47d4") : secret "samples-operator-tls" not found
Jul 06 14:20:41.790779 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:41.790713    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e35b0247-7c4e-4bd1-bc28-c084dc40ffa1-trusted-ca\") pod \"console-operator-68b757fb67-g5w5m\" (UID: \"e35b0247-7c4e-4bd1-bc28-c084dc40ffa1\") " pod="openshift-console-operator/console-operator-68b757fb67-g5w5m"
Jul 06 14:20:41.791273 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:41.791250    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/de6193cf-468b-4733-934b-71d909d33445-snapshots\") pod \"insights-operator-5c847dcfc4-2s94h\" (UID: \"de6193cf-468b-4733-934b-71d909d33445\") " pod="openshift-insights/insights-operator-5c847dcfc4-2s94h"
Jul 06 14:20:41.791365 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:41.791344    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/de6193cf-468b-4733-934b-71d909d33445-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-2s94h\" (UID: \"de6193cf-468b-4733-934b-71d909d33445\") " pod="openshift-insights/insights-operator-5c847dcfc4-2s94h"
Jul 06 14:20:41.791486 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:41.791465    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/de6193cf-468b-4733-934b-71d909d33445-tmp\") pod \"insights-operator-5c847dcfc4-2s94h\" (UID: \"de6193cf-468b-4733-934b-71d909d33445\") " pod="openshift-insights/insights-operator-5c847dcfc4-2s94h"
Jul 06 14:20:41.791718 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:41.791700    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e35b0247-7c4e-4bd1-bc28-c084dc40ffa1-serving-cert\") pod \"console-operator-68b757fb67-g5w5m\" (UID: \"e35b0247-7c4e-4bd1-bc28-c084dc40ffa1\") " pod="openshift-console-operator/console-operator-68b757fb67-g5w5m"
Jul 06 14:20:41.791859 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:41.791842    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/de6193cf-468b-4733-934b-71d909d33445-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-2s94h\" (UID: \"de6193cf-468b-4733-934b-71d909d33445\") " pod="openshift-insights/insights-operator-5c847dcfc4-2s94h"
Jul 06 14:20:41.793072 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:41.793049    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/de6193cf-468b-4733-934b-71d909d33445-serving-cert\") pod \"insights-operator-5c847dcfc4-2s94h\" (UID: \"de6193cf-468b-4733-934b-71d909d33445\") " pod="openshift-insights/insights-operator-5c847dcfc4-2s94h"
Jul 06 14:20:41.797925 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:41.797901    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-plldn\" (UniqueName: \"kubernetes.io/projected/de6193cf-468b-4733-934b-71d909d33445-kube-api-access-plldn\") pod \"insights-operator-5c847dcfc4-2s94h\" (UID: \"de6193cf-468b-4733-934b-71d909d33445\") " pod="openshift-insights/insights-operator-5c847dcfc4-2s94h"
Jul 06 14:20:41.798041 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:41.798001    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gt8c6\" (UniqueName: \"kubernetes.io/projected/a3b572f0-699a-4473-8804-ebac7a1e47d4-kube-api-access-gt8c6\") pod \"cluster-samples-operator-56fcf86bf-w8t5x\" (UID: \"a3b572f0-699a-4473-8804-ebac7a1e47d4\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-w8t5x"
Jul 06 14:20:41.799005 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:41.798984    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-828vn\" (UniqueName: \"kubernetes.io/projected/e35b0247-7c4e-4bd1-bc28-c084dc40ffa1-kube-api-access-828vn\") pod \"console-operator-68b757fb67-g5w5m\" (UID: \"e35b0247-7c4e-4bd1-bc28-c084dc40ffa1\") " pod="openshift-console-operator/console-operator-68b757fb67-g5w5m"
Jul 06 14:20:41.799087 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:41.799074    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-5t8dp"
Jul 06 14:20:41.820771 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:41.820742    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-q7t4k"]
Jul 06 14:20:41.824436 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:20:41.824402    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaecb0b51_89db_4843_9c3c_960ac76a1697.slice/crio-c639015f94b0d21dc4c085f103b612f02982185845ca32e5adfdd4d1b9025a09 WatchSource:0}: Error finding container c639015f94b0d21dc4c085f103b612f02982185845ca32e5adfdd4d1b9025a09: Status 404 returned error can't find the container with id c639015f94b0d21dc4c085f103b612f02982185845ca32e5adfdd4d1b9025a09
Jul 06 14:20:41.907845 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:41.907813    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-68b757fb67-g5w5m"
Jul 06 14:20:41.928915 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:41.928883    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-5t8dp"]
Jul 06 14:20:41.929900 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:41.929881    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5c847dcfc4-2s94h"
Jul 06 14:20:41.931678 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:20:41.931645    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfdb626d0_4fcb_42aa_8069_c1d7c8a76975.slice/crio-ec296d8f623579be9863e1d77190e593aac7cb908b7d18957372ee67d8c88dcc WatchSource:0}: Error finding container ec296d8f623579be9863e1d77190e593aac7cb908b7d18957372ee67d8c88dcc: Status 404 returned error can't find the container with id ec296d8f623579be9863e1d77190e593aac7cb908b7d18957372ee67d8c88dcc
Jul 06 14:20:42.068599 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:20:42.068559    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode35b0247_7c4e_4bd1_bc28_c084dc40ffa1.slice/crio-767767007e1c319c476d45179026fe2184b9ad8ba040821c3e2edb6b52fff576 WatchSource:0}: Error finding container 767767007e1c319c476d45179026fe2184b9ad8ba040821c3e2edb6b52fff576: Status 404 returned error can't find the container with id 767767007e1c319c476d45179026fe2184b9ad8ba040821c3e2edb6b52fff576
Jul 06 14:20:42.072288 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:42.072261    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-g5w5m"]
Jul 06 14:20:42.080869 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:42.080847    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-2s94h"]
Jul 06 14:20:42.083553 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:20:42.083528    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde6193cf_468b_4733_934b_71d909d33445.slice/crio-209edb2a6008084e1bb9b1396635beb7d0c311f9db32c4c6829a31f0edd8cb27 WatchSource:0}: Error finding container 209edb2a6008084e1bb9b1396635beb7d0c311f9db32c4c6829a31f0edd8cb27: Status 404 returned error can't find the container with id 209edb2a6008084e1bb9b1396635beb7d0c311f9db32c4c6829a31f0edd8cb27
Jul 06 14:20:42.096800 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:42.096781    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/606a9ffc-10e9-4114-ad7b-752ab0272a0d-registry-tls\") pod \"image-registry-6b894bb7d5-8nmsf\" (UID: \"606a9ffc-10e9-4114-ad7b-752ab0272a0d\") " pod="openshift-image-registry/image-registry-6b894bb7d5-8nmsf"
Jul 06 14:20:42.096939 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:20:42.096920    2559 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 06 14:20:42.096977 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:20:42.096941    2559 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6b894bb7d5-8nmsf: secret "image-registry-tls" not found
Jul 06 14:20:42.097010 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:20:42.096990    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/606a9ffc-10e9-4114-ad7b-752ab0272a0d-registry-tls podName:606a9ffc-10e9-4114-ad7b-752ab0272a0d nodeName:}" failed. No retries permitted until 2026-07-06 14:20:43.096973758 +0000 UTC m=+65.404265829 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/606a9ffc-10e9-4114-ad7b-752ab0272a0d-registry-tls") pod "image-registry-6b894bb7d5-8nmsf" (UID: "606a9ffc-10e9-4114-ad7b-752ab0272a0d") : secret "image-registry-tls" not found
Jul 06 14:20:42.299115 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:42.299022    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a3b572f0-699a-4473-8804-ebac7a1e47d4-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-w8t5x\" (UID: \"a3b572f0-699a-4473-8804-ebac7a1e47d4\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-w8t5x"
Jul 06 14:20:42.299279 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:20:42.299197    2559 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 06 14:20:42.299279 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:20:42.299276    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3b572f0-699a-4473-8804-ebac7a1e47d4-samples-operator-tls podName:a3b572f0-699a-4473-8804-ebac7a1e47d4 nodeName:}" failed. No retries permitted until 2026-07-06 14:20:43.299257015 +0000 UTC m=+65.606549097 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/a3b572f0-699a-4473-8804-ebac7a1e47d4-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-w8t5x" (UID: "a3b572f0-699a-4473-8804-ebac7a1e47d4") : secret "samples-operator-tls" not found
Jul 06 14:20:42.460724 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:42.460685    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-2s94h" event={"ID":"de6193cf-468b-4733-934b-71d909d33445","Type":"ContainerStarted","Data":"209edb2a6008084e1bb9b1396635beb7d0c311f9db32c4c6829a31f0edd8cb27"}
Jul 06 14:20:42.462147 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:42.462118    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-q7t4k" event={"ID":"aecb0b51-89db-4843-9c3c-960ac76a1697","Type":"ContainerStarted","Data":"c639015f94b0d21dc4c085f103b612f02982185845ca32e5adfdd4d1b9025a09"}
Jul 06 14:20:42.463314 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:42.463286    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-g5w5m" event={"ID":"e35b0247-7c4e-4bd1-bc28-c084dc40ffa1","Type":"ContainerStarted","Data":"767767007e1c319c476d45179026fe2184b9ad8ba040821c3e2edb6b52fff576"}
Jul 06 14:20:42.464447 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:42.464423    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-5t8dp" event={"ID":"fdb626d0-4fcb-42aa-8069-c1d7c8a76975","Type":"ContainerStarted","Data":"ec296d8f623579be9863e1d77190e593aac7cb908b7d18957372ee67d8c88dcc"}
Jul 06 14:20:43.106213 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:43.106175    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/606a9ffc-10e9-4114-ad7b-752ab0272a0d-registry-tls\") pod \"image-registry-6b894bb7d5-8nmsf\" (UID: \"606a9ffc-10e9-4114-ad7b-752ab0272a0d\") " pod="openshift-image-registry/image-registry-6b894bb7d5-8nmsf"
Jul 06 14:20:43.106428 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:20:43.106348    2559 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 06 14:20:43.106428 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:20:43.106365    2559 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6b894bb7d5-8nmsf: secret "image-registry-tls" not found
Jul 06 14:20:43.106428 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:20:43.106419    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/606a9ffc-10e9-4114-ad7b-752ab0272a0d-registry-tls podName:606a9ffc-10e9-4114-ad7b-752ab0272a0d nodeName:}" failed. No retries permitted until 2026-07-06 14:20:45.106406025 +0000 UTC m=+67.413698086 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/606a9ffc-10e9-4114-ad7b-752ab0272a0d-registry-tls") pod "image-registry-6b894bb7d5-8nmsf" (UID: "606a9ffc-10e9-4114-ad7b-752ab0272a0d") : secret "image-registry-tls" not found
Jul 06 14:20:43.309119 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:43.309077    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a3b572f0-699a-4473-8804-ebac7a1e47d4-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-w8t5x\" (UID: \"a3b572f0-699a-4473-8804-ebac7a1e47d4\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-w8t5x"
Jul 06 14:20:43.309323 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:20:43.309306    2559 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 06 14:20:43.309399 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:20:43.309376    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3b572f0-699a-4473-8804-ebac7a1e47d4-samples-operator-tls podName:a3b572f0-699a-4473-8804-ebac7a1e47d4 nodeName:}" failed. No retries permitted until 2026-07-06 14:20:45.309358556 +0000 UTC m=+67.616650630 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/a3b572f0-699a-4473-8804-ebac7a1e47d4-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-w8t5x" (UID: "a3b572f0-699a-4473-8804-ebac7a1e47d4") : secret "samples-operator-tls" not found
Jul 06 14:20:43.390551 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:43.390473    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-prjxn"
Jul 06 14:20:43.915857 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:43.915376    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1c05cc5f-44d6-46b6-947a-9d221f282365-metrics-certs\") pod \"network-metrics-daemon-v94gg\" (UID: \"1c05cc5f-44d6-46b6-947a-9d221f282365\") " pod="openshift-multus/network-metrics-daemon-v94gg"
Jul 06 14:20:43.915857 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:43.915433    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ab514f6f-ea23-486a-b065-aac1b24a5035-original-pull-secret\") pod \"global-pull-secret-syncer-qjsqm\" (UID: \"ab514f6f-ea23-486a-b065-aac1b24a5035\") " pod="kube-system/global-pull-secret-syncer-qjsqm"
Jul 06 14:20:43.918112 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:43.918082    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 06 14:20:43.918224 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:43.918167    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 06 14:20:43.926313 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:20:43.926179    2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 06 14:20:43.926313 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:20:43.926265    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1c05cc5f-44d6-46b6-947a-9d221f282365-metrics-certs podName:1c05cc5f-44d6-46b6-947a-9d221f282365 nodeName:}" failed. No retries permitted until 2026-07-06 14:21:47.926242885 +0000 UTC m=+130.233534948 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1c05cc5f-44d6-46b6-947a-9d221f282365-metrics-certs") pod "network-metrics-daemon-v94gg" (UID: "1c05cc5f-44d6-46b6-947a-9d221f282365") : secret "metrics-daemon-secret" not found
Jul 06 14:20:43.930469 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:43.930399    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ab514f6f-ea23-486a-b065-aac1b24a5035-original-pull-secret\") pod \"global-pull-secret-syncer-qjsqm\" (UID: \"ab514f6f-ea23-486a-b065-aac1b24a5035\") " pod="kube-system/global-pull-secret-syncer-qjsqm"
Jul 06 14:20:43.942221 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:43.942191    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qjsqm"
Jul 06 14:20:44.116958 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:44.116917    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2r8z4\" (UniqueName: \"kubernetes.io/projected/98382dda-aacf-4546-8a91-e6bd00632d71-kube-api-access-2r8z4\") pod \"network-check-target-l87kx\" (UID: \"98382dda-aacf-4546-8a91-e6bd00632d71\") " pod="openshift-network-diagnostics/network-check-target-l87kx"
Jul 06 14:20:44.120034 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:44.120010    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2r8z4\" (UniqueName: \"kubernetes.io/projected/98382dda-aacf-4546-8a91-e6bd00632d71-kube-api-access-2r8z4\") pod \"network-check-target-l87kx\" (UID: \"98382dda-aacf-4546-8a91-e6bd00632d71\") " pod="openshift-network-diagnostics/network-check-target-l87kx"
Jul 06 14:20:44.239583 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:44.239529    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-74wbt\""
Jul 06 14:20:44.247884 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:44.247851    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l87kx"
Jul 06 14:20:45.125889 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:45.125856    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/606a9ffc-10e9-4114-ad7b-752ab0272a0d-registry-tls\") pod \"image-registry-6b894bb7d5-8nmsf\" (UID: \"606a9ffc-10e9-4114-ad7b-752ab0272a0d\") " pod="openshift-image-registry/image-registry-6b894bb7d5-8nmsf"
Jul 06 14:20:45.126300 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:20:45.126021    2559 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 06 14:20:45.126300 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:20:45.126040    2559 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6b894bb7d5-8nmsf: secret "image-registry-tls" not found
Jul 06 14:20:45.126300 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:20:45.126093    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/606a9ffc-10e9-4114-ad7b-752ab0272a0d-registry-tls podName:606a9ffc-10e9-4114-ad7b-752ab0272a0d nodeName:}" failed. No retries permitted until 2026-07-06 14:20:49.126078409 +0000 UTC m=+71.433370474 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/606a9ffc-10e9-4114-ad7b-752ab0272a0d-registry-tls") pod "image-registry-6b894bb7d5-8nmsf" (UID: "606a9ffc-10e9-4114-ad7b-752ab0272a0d") : secret "image-registry-tls" not found
Jul 06 14:20:45.328050 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:45.328009    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a3b572f0-699a-4473-8804-ebac7a1e47d4-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-w8t5x\" (UID: \"a3b572f0-699a-4473-8804-ebac7a1e47d4\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-w8t5x"
Jul 06 14:20:45.328245 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:20:45.328157    2559 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 06 14:20:45.328245 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:20:45.328234    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3b572f0-699a-4473-8804-ebac7a1e47d4-samples-operator-tls podName:a3b572f0-699a-4473-8804-ebac7a1e47d4 nodeName:}" failed. No retries permitted until 2026-07-06 14:20:49.328216968 +0000 UTC m=+71.635509049 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/a3b572f0-699a-4473-8804-ebac7a1e47d4-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-w8t5x" (UID: "a3b572f0-699a-4473-8804-ebac7a1e47d4") : secret "samples-operator-tls" not found
Jul 06 14:20:46.617068 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:46.616953    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-qjsqm"]
Jul 06 14:20:46.645895 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:46.643617    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-l87kx"]
Jul 06 14:20:47.480045 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:47.480014    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-g5w5m_e35b0247-7c4e-4bd1-bc28-c084dc40ffa1/console-operator/0.log"
Jul 06 14:20:47.480233 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:47.480062    2559 generic.go:356] "Generic (PLEG): container finished" podID="e35b0247-7c4e-4bd1-bc28-c084dc40ffa1" containerID="e07aded9b7fe91786edc8a0c976d31b5fc64107d22c5c9dd3b284f218d10824d" exitCode=255
Jul 06 14:20:47.480233 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:47.480144    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-g5w5m" event={"ID":"e35b0247-7c4e-4bd1-bc28-c084dc40ffa1","Type":"ContainerDied","Data":"e07aded9b7fe91786edc8a0c976d31b5fc64107d22c5c9dd3b284f218d10824d"}
Jul 06 14:20:47.480597 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:47.480393    2559 scope.go:117] "RemoveContainer" containerID="e07aded9b7fe91786edc8a0c976d31b5fc64107d22c5c9dd3b284f218d10824d"
Jul 06 14:20:47.482748 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:47.482718    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-5t8dp" event={"ID":"fdb626d0-4fcb-42aa-8069-c1d7c8a76975","Type":"ContainerStarted","Data":"4a2d559b0ecc9976eb8e6ca5a4c73f1720bd9ff120f5f52997b88d9c1a0f4a2a"}
Jul 06 14:20:47.484453 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:47.484421    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-l87kx" event={"ID":"98382dda-aacf-4546-8a91-e6bd00632d71","Type":"ContainerStarted","Data":"bf173c41199a748373c0e388ae5680de30f1d639b8be316b7ea336c5d366b40b"}
Jul 06 14:20:47.484546 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:47.484467    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-l87kx" event={"ID":"98382dda-aacf-4546-8a91-e6bd00632d71","Type":"ContainerStarted","Data":"dc2353e9d9045c77fb3f460bc5249441469c5c6a04c98a34dfb85badbbe274f4"}
Jul 06 14:20:47.484619 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:47.484604    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-l87kx"
Jul 06 14:20:47.486532 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:47.486404    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-2s94h" event={"ID":"de6193cf-468b-4733-934b-71d909d33445","Type":"ContainerStarted","Data":"56590fead34e02130d3a89cf05524b3035be7f2f89e5e72607b2beec3eac7cdd"}
Jul 06 14:20:47.488248 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:47.488145    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-q7t4k" event={"ID":"aecb0b51-89db-4843-9c3c-960ac76a1697","Type":"ContainerStarted","Data":"1a1725296a64bb1ddb7861e702867255fb2c9e6c120bb16fe2af6259b73fd11d"}
Jul 06 14:20:47.490521 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:47.490498    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-qjsqm" event={"ID":"ab514f6f-ea23-486a-b065-aac1b24a5035","Type":"ContainerStarted","Data":"a372e7706a4f9d26a2ea90758014ebd30668193becc31066c905dada7e4b4f7f"}
Jul 06 14:20:47.524158 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:47.524104    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-l87kx" podStartSLOduration=68.524083286 podStartE2EDuration="1m8.524083286s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:19:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:20:47.523328806 +0000 UTC m=+69.830620899" watchObservedRunningTime="2026-07-06 14:20:47.524083286 +0000 UTC m=+69.831375368"
Jul 06 14:20:47.578901 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:47.578827    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5c847dcfc4-2s94h" podStartSLOduration=2.201956103 podStartE2EDuration="6.578802933s" totalImagesPullingTime="4.37684683s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:20:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:20:47.576530849 +0000 UTC m=+69.883822970" watchObservedRunningTime="2026-07-06 14:20:47.578802933 +0000 UTC m=+69.886095014"
Jul 06 14:20:47.579434 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:47.579182    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-q7t4k" podStartSLOduration=1.941854686 podStartE2EDuration="6.579168958s" totalImagesPullingTime="4.637314272s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:20:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:20:47.549687124 +0000 UTC m=+69.856979206" watchObservedRunningTime="2026-07-06 14:20:47.579168958 +0000 UTC m=+69.886461037"
Jul 06 14:20:47.891019 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:47.890897    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-5t8dp" podStartSLOduration=2.34947297 podStartE2EDuration="6.890862748s" totalImagesPullingTime="4.541389778s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:20:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:20:47.604834573 +0000 UTC m=+69.912126656" watchObservedRunningTime="2026-07-06 14:20:47.890862748 +0000 UTC m=+70.198154831"
Jul 06 14:20:47.891474 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:47.891429    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-pfrz4"]
Jul 06 14:20:47.894706 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:47.894675    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-pfrz4"
Jul 06 14:20:47.897031 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:47.897005    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-vfj8b\""
Jul 06 14:20:47.897218 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:47.897187    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Jul 06 14:20:47.897642 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:47.897623    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Jul 06 14:20:47.910785 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:47.910729    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-pfrz4"]
Jul 06 14:20:47.952215 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:47.952184    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f24xg\" (UniqueName: \"kubernetes.io/projected/8a987d28-7a8b-4a1c-a9a8-551bf382b523-kube-api-access-f24xg\") pod \"migrator-d57d58cc9-pfrz4\" (UID: \"8a987d28-7a8b-4a1c-a9a8-551bf382b523\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-pfrz4"
Jul 06 14:20:48.053486 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:48.053434    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f24xg\" (UniqueName: \"kubernetes.io/projected/8a987d28-7a8b-4a1c-a9a8-551bf382b523-kube-api-access-f24xg\") pod \"migrator-d57d58cc9-pfrz4\" (UID: \"8a987d28-7a8b-4a1c-a9a8-551bf382b523\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-pfrz4"
Jul 06 14:20:48.063387 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:48.063324    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f24xg\" (UniqueName: \"kubernetes.io/projected/8a987d28-7a8b-4a1c-a9a8-551bf382b523-kube-api-access-f24xg\") pod \"migrator-d57d58cc9-pfrz4\" (UID: \"8a987d28-7a8b-4a1c-a9a8-551bf382b523\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-pfrz4"
Jul 06 14:20:48.205633 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:48.205591    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-pfrz4"
Jul 06 14:20:48.366778 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:48.366745    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-pfrz4"]
Jul 06 14:20:48.369900 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:20:48.369867    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a987d28_7a8b_4a1c_a9a8_551bf382b523.slice/crio-79ddbab2a3cd3d951322e1bdca350b80455b882fbbf8d1c49262f497d09feaaf WatchSource:0}: Error finding container 79ddbab2a3cd3d951322e1bdca350b80455b882fbbf8d1c49262f497d09feaaf: Status 404 returned error can't find the container with id 79ddbab2a3cd3d951322e1bdca350b80455b882fbbf8d1c49262f497d09feaaf
Jul 06 14:20:48.495530 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:48.495442    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-g5w5m_e35b0247-7c4e-4bd1-bc28-c084dc40ffa1/console-operator/1.log"
Jul 06 14:20:48.495867 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:48.495849    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-g5w5m_e35b0247-7c4e-4bd1-bc28-c084dc40ffa1/console-operator/0.log"
Jul 06 14:20:48.495922 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:48.495888    2559 generic.go:356] "Generic (PLEG): container finished" podID="e35b0247-7c4e-4bd1-bc28-c084dc40ffa1" containerID="6b5c9e0091935c5919b1a1b334c0e4f598226d892286fbcaeae314ebb4fa8ecb" exitCode=255
Jul 06 14:20:48.495971 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:48.495956    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-g5w5m" event={"ID":"e35b0247-7c4e-4bd1-bc28-c084dc40ffa1","Type":"ContainerDied","Data":"6b5c9e0091935c5919b1a1b334c0e4f598226d892286fbcaeae314ebb4fa8ecb"}
Jul 06 14:20:48.496039 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:48.496020    2559 scope.go:117] "RemoveContainer" containerID="e07aded9b7fe91786edc8a0c976d31b5fc64107d22c5c9dd3b284f218d10824d"
Jul 06 14:20:48.496254 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:48.496239    2559 scope.go:117] "RemoveContainer" containerID="6b5c9e0091935c5919b1a1b334c0e4f598226d892286fbcaeae314ebb4fa8ecb"
Jul 06 14:20:48.496451 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:20:48.496427    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-g5w5m_openshift-console-operator(e35b0247-7c4e-4bd1-bc28-c084dc40ffa1)\"" pod="openshift-console-operator/console-operator-68b757fb67-g5w5m" podUID="e35b0247-7c4e-4bd1-bc28-c084dc40ffa1"
Jul 06 14:20:48.497520 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:48.497251    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-pfrz4" event={"ID":"8a987d28-7a8b-4a1c-a9a8-551bf382b523","Type":"ContainerStarted","Data":"79ddbab2a3cd3d951322e1bdca350b80455b882fbbf8d1c49262f497d09feaaf"}
Jul 06 14:20:48.960353 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:48.960310    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e1c99840-57b6-4e0b-8e6d-80bca474168d-metrics-tls\") pod \"dns-default-kfvv5\" (UID: \"e1c99840-57b6-4e0b-8e6d-80bca474168d\") " pod="openshift-dns/dns-default-kfvv5"
Jul 06 14:20:48.960825 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:20:48.960438    2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 06 14:20:48.960825 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:20:48.960507    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e1c99840-57b6-4e0b-8e6d-80bca474168d-metrics-tls podName:e1c99840-57b6-4e0b-8e6d-80bca474168d nodeName:}" failed. No retries permitted until 2026-07-06 14:21:20.960489344 +0000 UTC m=+103.267781423 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e1c99840-57b6-4e0b-8e6d-80bca474168d-metrics-tls") pod "dns-default-kfvv5" (UID: "e1c99840-57b6-4e0b-8e6d-80bca474168d") : secret "dns-default-metrics-tls" not found
Jul 06 14:20:49.162304 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:49.162270    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/606a9ffc-10e9-4114-ad7b-752ab0272a0d-registry-tls\") pod \"image-registry-6b894bb7d5-8nmsf\" (UID: \"606a9ffc-10e9-4114-ad7b-752ab0272a0d\") " pod="openshift-image-registry/image-registry-6b894bb7d5-8nmsf"
Jul 06 14:20:49.162470 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:20:49.162442    2559 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 06 14:20:49.162470 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:20:49.162465    2559 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6b894bb7d5-8nmsf: secret "image-registry-tls" not found
Jul 06 14:20:49.162605 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:20:49.162536    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/606a9ffc-10e9-4114-ad7b-752ab0272a0d-registry-tls podName:606a9ffc-10e9-4114-ad7b-752ab0272a0d nodeName:}" failed. No retries permitted until 2026-07-06 14:20:57.16251678 +0000 UTC m=+79.469808843 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/606a9ffc-10e9-4114-ad7b-752ab0272a0d-registry-tls") pod "image-registry-6b894bb7d5-8nmsf" (UID: "606a9ffc-10e9-4114-ad7b-752ab0272a0d") : secret "image-registry-tls" not found
Jul 06 14:20:49.363308 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:49.363223    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a3b572f0-699a-4473-8804-ebac7a1e47d4-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-w8t5x\" (UID: \"a3b572f0-699a-4473-8804-ebac7a1e47d4\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-w8t5x"
Jul 06 14:20:49.363449 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:20:49.363380    2559 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 06 14:20:49.363494 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:20:49.363450    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3b572f0-699a-4473-8804-ebac7a1e47d4-samples-operator-tls podName:a3b572f0-699a-4473-8804-ebac7a1e47d4 nodeName:}" failed. No retries permitted until 2026-07-06 14:20:57.363432798 +0000 UTC m=+79.670724858 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/a3b572f0-699a-4473-8804-ebac7a1e47d4-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-w8t5x" (UID: "a3b572f0-699a-4473-8804-ebac7a1e47d4") : secret "samples-operator-tls" not found
Jul 06 14:20:49.502387 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:49.502359    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-g5w5m_e35b0247-7c4e-4bd1-bc28-c084dc40ffa1/console-operator/1.log"
Jul 06 14:20:49.502761 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:49.502743    2559 scope.go:117] "RemoveContainer" containerID="6b5c9e0091935c5919b1a1b334c0e4f598226d892286fbcaeae314ebb4fa8ecb"
Jul 06 14:20:49.502961 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:20:49.502940    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-g5w5m_openshift-console-operator(e35b0247-7c4e-4bd1-bc28-c084dc40ffa1)\"" pod="openshift-console-operator/console-operator-68b757fb67-g5w5m" podUID="e35b0247-7c4e-4bd1-bc28-c084dc40ffa1"
Jul 06 14:20:50.508212 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:50.508166    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-pfrz4" event={"ID":"8a987d28-7a8b-4a1c-a9a8-551bf382b523","Type":"ContainerStarted","Data":"e39b978e9c3063a9fbd65bfbdb0b9ec7ec20767357cdfefb35452cce598cb718"}
Jul 06 14:20:51.208612 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:51.208559    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-9tndp_f07d461c-9de3-43c8-b501-f50c0cd463eb/dns-node-resolver/0.log"
Jul 06 14:20:51.908390 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:51.908174    2559 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-68b757fb67-g5w5m"
Jul 06 14:20:51.908828 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:51.908604    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-g5w5m"
Jul 06 14:20:51.908828 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:51.908759    2559 scope.go:117] "RemoveContainer" containerID="6b5c9e0091935c5919b1a1b334c0e4f598226d892286fbcaeae314ebb4fa8ecb"
Jul 06 14:20:51.908936 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:20:51.908917    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-g5w5m_openshift-console-operator(e35b0247-7c4e-4bd1-bc28-c084dc40ffa1)\"" pod="openshift-console-operator/console-operator-68b757fb67-g5w5m" podUID="e35b0247-7c4e-4bd1-bc28-c084dc40ffa1"
Jul 06 14:20:52.206634 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:52.206604    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-pr7f9_ef2ced35-fb83-46cb-9e31-2c0814bd5db2/node-ca/0.log"
Jul 06 14:20:52.515798 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:52.515710    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-qjsqm" event={"ID":"ab514f6f-ea23-486a-b065-aac1b24a5035","Type":"ContainerStarted","Data":"f0ad6aa5cbd07f8d71c6b3ae866dda1094e42c44380a4ff6401d4fc6e80869e8"}
Jul 06 14:20:52.517308 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:52.517286    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-pfrz4" event={"ID":"8a987d28-7a8b-4a1c-a9a8-551bf382b523","Type":"ContainerStarted","Data":"a9d97e57043be9c9ba003aec32f06ceec5ff038d35fe5051c4c4f23ffb850fe7"}
Jul 06 14:20:52.517542 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:52.517527    2559 scope.go:117] "RemoveContainer" containerID="6b5c9e0091935c5919b1a1b334c0e4f598226d892286fbcaeae314ebb4fa8ecb"
Jul 06 14:20:52.517710 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:20:52.517692    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-g5w5m_openshift-console-operator(e35b0247-7c4e-4bd1-bc28-c084dc40ffa1)\"" pod="openshift-console-operator/console-operator-68b757fb67-g5w5m" podUID="e35b0247-7c4e-4bd1-bc28-c084dc40ffa1"
Jul 06 14:20:52.536109 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:52.536053    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-qjsqm" podStartSLOduration=68.35007296 podStartE2EDuration="1m13.536041321s" totalImagesPullingTime="5.185968361s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 14:19:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:20:52.534301526 +0000 UTC m=+74.841593609" watchObservedRunningTime="2026-07-06 14:20:52.536041321 +0000 UTC m=+74.843333402"
Jul 06 14:20:52.560972 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:52.560928    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-pfrz4" podStartSLOduration=3.685809862 podStartE2EDuration="5.560914445s" totalImagesPullingTime="1.875104583s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:20:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:20:52.560102403 +0000 UTC m=+74.867394485" watchObservedRunningTime="2026-07-06 14:20:52.560914445 +0000 UTC m=+74.868206526"
Jul 06 14:20:53.208618 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:53.208587    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-7r8pk_e19cda1c-b504-43fa-8dfb-8f53092dd8d8/serve-healthcheck-canary/0.log"
Jul 06 14:20:53.409788 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:53.409760    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-pfrz4_8a987d28-7a8b-4a1c-a9a8-551bf382b523/migrator/0.log"
Jul 06 14:20:53.606620 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:53.606525    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-pfrz4_8a987d28-7a8b-4a1c-a9a8-551bf382b523/graceful-termination/0.log"
Jul 06 14:20:53.808677 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:53.808643    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-846bf668f7-q7t4k_aecb0b51-89db-4843-9c3c-960ac76a1697/kube-storage-version-migrator-operator/0.log"
Jul 06 14:20:57.229628 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:57.229566    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/606a9ffc-10e9-4114-ad7b-752ab0272a0d-registry-tls\") pod \"image-registry-6b894bb7d5-8nmsf\" (UID: \"606a9ffc-10e9-4114-ad7b-752ab0272a0d\") " pod="openshift-image-registry/image-registry-6b894bb7d5-8nmsf"
Jul 06 14:20:57.231821 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:57.231795    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/606a9ffc-10e9-4114-ad7b-752ab0272a0d-registry-tls\") pod \"image-registry-6b894bb7d5-8nmsf\" (UID: \"606a9ffc-10e9-4114-ad7b-752ab0272a0d\") " pod="openshift-image-registry/image-registry-6b894bb7d5-8nmsf"
Jul 06 14:20:57.303302 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:57.303259    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6b894bb7d5-8nmsf"
Jul 06 14:20:57.430558 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:57.430501    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a3b572f0-699a-4473-8804-ebac7a1e47d4-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-w8t5x\" (UID: \"a3b572f0-699a-4473-8804-ebac7a1e47d4\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-w8t5x"
Jul 06 14:20:57.432946 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:57.432914    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a3b572f0-699a-4473-8804-ebac7a1e47d4-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-w8t5x\" (UID: \"a3b572f0-699a-4473-8804-ebac7a1e47d4\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-w8t5x"
Jul 06 14:20:57.436700 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:57.436674    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6b894bb7d5-8nmsf"]
Jul 06 14:20:57.439749 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:20:57.439722    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod606a9ffc_10e9_4114_ad7b_752ab0272a0d.slice/crio-1ec27bd440006feb2a01cdb79bd4d9c5e75fb53263c404444088727ed9e57d4d WatchSource:0}: Error finding container 1ec27bd440006feb2a01cdb79bd4d9c5e75fb53263c404444088727ed9e57d4d: Status 404 returned error can't find the container with id 1ec27bd440006feb2a01cdb79bd4d9c5e75fb53263c404444088727ed9e57d4d
Jul 06 14:20:57.533773 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:57.533734    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6b894bb7d5-8nmsf" event={"ID":"606a9ffc-10e9-4114-ad7b-752ab0272a0d","Type":"ContainerStarted","Data":"8dfba37aeea2c0ba20876ac55afeedadd9d517da5830bda4289653bf92a3603e"}
Jul 06 14:20:57.533939 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:57.533782    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6b894bb7d5-8nmsf" event={"ID":"606a9ffc-10e9-4114-ad7b-752ab0272a0d","Type":"ContainerStarted","Data":"1ec27bd440006feb2a01cdb79bd4d9c5e75fb53263c404444088727ed9e57d4d"}
Jul 06 14:20:57.533939 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:57.533867    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6b894bb7d5-8nmsf"
Jul 06 14:20:57.534043 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:57.534029    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-w8t5x"
Jul 06 14:20:57.557606 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:57.557525    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-6b894bb7d5-8nmsf" podStartSLOduration=16.557504311 podStartE2EDuration="16.557504311s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:20:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:20:57.555996593 +0000 UTC m=+79.863288676" watchObservedRunningTime="2026-07-06 14:20:57.557504311 +0000 UTC m=+79.864796397"
Jul 06 14:20:57.665170 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:57.665143    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-w8t5x"]
Jul 06 14:20:58.542298 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:20:58.542252    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-w8t5x" event={"ID":"a3b572f0-699a-4473-8804-ebac7a1e47d4","Type":"ContainerStarted","Data":"d11c5d8ea0ce6b011ed254d229bf3c3825a2b345a63786f4dde15b1bc790252b"}
Jul 06 14:21:00.550319 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:00.550225    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-w8t5x" event={"ID":"a3b572f0-699a-4473-8804-ebac7a1e47d4","Type":"ContainerStarted","Data":"a288161134370c53dc529efaa3706c7ea4976d7f87ababf7c09d40efc8c09b16"}
Jul 06 14:21:00.550319 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:00.550278    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-w8t5x" event={"ID":"a3b572f0-699a-4473-8804-ebac7a1e47d4","Type":"ContainerStarted","Data":"c4516f5df69a1a76231cf56ff1164045713f0b826e5d63f99be889149dfe9e52"}
Jul 06 14:21:00.579310 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:00.579256    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-w8t5x" podStartSLOduration=17.495700178 podStartE2EDuration="19.579241496s" totalImagesPullingTime="2.083541318s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:20:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:21:00.57900024 +0000 UTC m=+82.886292323" watchObservedRunningTime="2026-07-06 14:21:00.579241496 +0000 UTC m=+82.886533577"
Jul 06 14:21:04.221167 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:04.221133    2559 scope.go:117] "RemoveContainer" containerID="6b5c9e0091935c5919b1a1b334c0e4f598226d892286fbcaeae314ebb4fa8ecb"
Jul 06 14:21:04.565226 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:04.565151    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-g5w5m_e35b0247-7c4e-4bd1-bc28-c084dc40ffa1/console-operator/1.log"
Jul 06 14:21:04.565226 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:04.565202    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-g5w5m" event={"ID":"e35b0247-7c4e-4bd1-bc28-c084dc40ffa1","Type":"ContainerStarted","Data":"9b74ab4d640d0eed28422fc8a16577132c8895c1df5d9faaf2f5bacf6d8f214f"}
Jul 06 14:21:04.565488 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:04.565466    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-g5w5m"
Jul 06 14:21:04.587587 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:04.587527    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-68b757fb67-g5w5m" podStartSLOduration=19.194924182 podStartE2EDuration="23.587511167s" totalImagesPullingTime="4.392586985s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:20:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:21:04.586686205 +0000 UTC m=+86.893978322" watchObservedRunningTime="2026-07-06 14:21:04.587511167 +0000 UTC m=+86.894803249"
Jul 06 14:21:05.010432 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:05.010398    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-68b757fb67-g5w5m"
Jul 06 14:21:11.948622 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:11.948588    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-lghzh"]
Jul 06 14:21:11.953019 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:11.953004    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-lghzh"
Jul 06 14:21:11.956186 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:11.956166    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-nzwtp\""
Jul 06 14:21:11.956305 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:11.956166    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 06 14:21:11.956305 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:11.956166    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 06 14:21:11.966306 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:11.966281    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-lghzh"]
Jul 06 14:21:12.042192 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:12.042156    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/38ad328a-62dd-4a21-aaf9-3141c532fafb-crio-socket\") pod \"insights-runtime-extractor-lghzh\" (UID: \"38ad328a-62dd-4a21-aaf9-3141c532fafb\") " pod="openshift-insights/insights-runtime-extractor-lghzh"
Jul 06 14:21:12.042192 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:12.042187    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/38ad328a-62dd-4a21-aaf9-3141c532fafb-data-volume\") pod \"insights-runtime-extractor-lghzh\" (UID: \"38ad328a-62dd-4a21-aaf9-3141c532fafb\") " pod="openshift-insights/insights-runtime-extractor-lghzh"
Jul 06 14:21:12.042420 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:12.042223    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/38ad328a-62dd-4a21-aaf9-3141c532fafb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lghzh\" (UID: \"38ad328a-62dd-4a21-aaf9-3141c532fafb\") " pod="openshift-insights/insights-runtime-extractor-lghzh"
Jul 06 14:21:12.042420 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:12.042247    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/38ad328a-62dd-4a21-aaf9-3141c532fafb-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lghzh\" (UID: \"38ad328a-62dd-4a21-aaf9-3141c532fafb\") " pod="openshift-insights/insights-runtime-extractor-lghzh"
Jul 06 14:21:12.042420 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:12.042352    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lds2l\" (UniqueName: \"kubernetes.io/projected/38ad328a-62dd-4a21-aaf9-3141c532fafb-kube-api-access-lds2l\") pod \"insights-runtime-extractor-lghzh\" (UID: \"38ad328a-62dd-4a21-aaf9-3141c532fafb\") " pod="openshift-insights/insights-runtime-extractor-lghzh"
Jul 06 14:21:12.078842 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:12.078803    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-k5fj6"]
Jul 06 14:21:12.082104 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:12.082086    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-855494d94-jnhl5"]
Jul 06 14:21:12.082231 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:12.082215    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-k5fj6"
Jul 06 14:21:12.085012 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:12.084994    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-855494d94-jnhl5"
Jul 06 14:21:12.085112 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:12.085074    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Jul 06 14:21:12.088842 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:12.088815    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Jul 06 14:21:12.088957 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:12.088846    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-zfcv6\""
Jul 06 14:21:12.089110 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:12.088978    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Jul 06 14:21:12.089157 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:12.089115    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 06 14:21:12.089392 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:12.089376    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Jul 06 14:21:12.089528 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:12.089505    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Jul 06 14:21:12.089528 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:12.089382    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Jul 06 14:21:12.089528 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:12.089513    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 06 14:21:12.089775 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:12.089673    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 06 14:21:12.104892 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:12.104867    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6b894bb7d5-8nmsf"]
Jul 06 14:21:12.105897 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:12.105875    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-k5fj6"]
Jul 06 14:21:12.109313 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:12.109289    2559 patch_prober.go:28] interesting pod/image-registry-6b894bb7d5-8nmsf container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 06 14:21:12.109418 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:12.109334    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-6b894bb7d5-8nmsf" podUID="606a9ffc-10e9-4114-ad7b-752ab0272a0d" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 06 14:21:12.131496 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:12.131462    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-855494d94-jnhl5"]
Jul 06 14:21:12.143703 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:12.143673    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lds2l\" (UniqueName: \"kubernetes.io/projected/38ad328a-62dd-4a21-aaf9-3141c532fafb-kube-api-access-lds2l\") pod \"insights-runtime-extractor-lghzh\" (UID: \"38ad328a-62dd-4a21-aaf9-3141c532fafb\") " pod="openshift-insights/insights-runtime-extractor-lghzh"
Jul 06 14:21:12.143824 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:12.143719    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/38ad328a-62dd-4a21-aaf9-3141c532fafb-crio-socket\") pod \"insights-runtime-extractor-lghzh\" (UID: \"38ad328a-62dd-4a21-aaf9-3141c532fafb\") " pod="openshift-insights/insights-runtime-extractor-lghzh"
Jul 06 14:21:12.143824 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:12.143738    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/38ad328a-62dd-4a21-aaf9-3141c532fafb-data-volume\") pod \"insights-runtime-extractor-lghzh\" (UID: \"38ad328a-62dd-4a21-aaf9-3141c532fafb\") " pod="openshift-insights/insights-runtime-extractor-lghzh"
Jul 06 14:21:12.143824 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:12.143768    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/38ad328a-62dd-4a21-aaf9-3141c532fafb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lghzh\" (UID: \"38ad328a-62dd-4a21-aaf9-3141c532fafb\") " pod="openshift-insights/insights-runtime-extractor-lghzh"
Jul 06 14:21:12.143824 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:12.143794    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/38ad328a-62dd-4a21-aaf9-3141c532fafb-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lghzh\" (UID: \"38ad328a-62dd-4a21-aaf9-3141c532fafb\") " pod="openshift-insights/insights-runtime-extractor-lghzh"
Jul 06 14:21:12.143955 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:12.143816    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/38ad328a-62dd-4a21-aaf9-3141c532fafb-crio-socket\") pod \"insights-runtime-extractor-lghzh\" (UID: \"38ad328a-62dd-4a21-aaf9-3141c532fafb\") " pod="openshift-insights/insights-runtime-extractor-lghzh"
Jul 06 14:21:12.144169 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:12.144149    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/38ad328a-62dd-4a21-aaf9-3141c532fafb-data-volume\") pod \"insights-runtime-extractor-lghzh\" (UID: \"38ad328a-62dd-4a21-aaf9-3141c532fafb\") " pod="openshift-insights/insights-runtime-extractor-lghzh"
Jul 06 14:21:12.144320 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:12.144300    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/38ad328a-62dd-4a21-aaf9-3141c532fafb-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lghzh\" (UID: \"38ad328a-62dd-4a21-aaf9-3141c532fafb\") " pod="openshift-insights/insights-runtime-extractor-lghzh"
Jul 06 14:21:12.146007 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:12.145990    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/38ad328a-62dd-4a21-aaf9-3141c532fafb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lghzh\" (UID: \"38ad328a-62dd-4a21-aaf9-3141c532fafb\") " pod="openshift-insights/insights-runtime-extractor-lghzh"
Jul 06 14:21:12.170292 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:12.170259    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lds2l\" (UniqueName: \"kubernetes.io/projected/38ad328a-62dd-4a21-aaf9-3141c532fafb-kube-api-access-lds2l\") pod \"insights-runtime-extractor-lghzh\" (UID: \"38ad328a-62dd-4a21-aaf9-3141c532fafb\") " pod="openshift-insights/insights-runtime-extractor-lghzh"
Jul 06 14:21:12.186230 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:12.186204    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-685bfb5f64-psvjq"]
Jul 06 14:21:12.189437 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:12.189419    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-685bfb5f64-psvjq"
Jul 06 14:21:12.193396 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:12.193368    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 06 14:21:12.194911 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:12.194884    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-z4962"]
Jul 06 14:21:12.196085 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:12.196063    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 06 14:21:12.196204 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:12.196112    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-kdmbg\""
Jul 06 14:21:12.197938 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:12.197916    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-564974887b-25vkr"]
Jul 06 14:21:12.198046 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:12.197977    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-z4962"
Jul 06 14:21:12.200899 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:12.200848    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-564974887b-25vkr"
Jul 06 14:21:12.202093 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:12.202077    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Jul 06 14:21:12.204070 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:12.203993    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-8fvv7\""
Jul 06 14:21:12.210824 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:12.210802    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-685bfb5f64-psvjq"]
Jul 06 14:21:12.217888 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:12.216511    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-z4962"]
Jul 06 14:21:12.225824 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:12.225803    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-564974887b-25vkr"]
Jul 06 14:21:12.245018 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:12.244986    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b0c48750-e081-4711-94d7-04ebce9ded67-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-855494d94-jnhl5\" (UID: \"b0c48750-e081-4711-94d7-04ebce9ded67\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-855494d94-jnhl5"
Jul 06 14:21:12.245153 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:12.245027    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/b0c48750-e081-4711-94d7-04ebce9ded67-additional-service-ca\") pod \"cluster-proxy-proxy-agent-855494d94-jnhl5\" (UID: \"b0c48750-e081-4711-94d7-04ebce9ded67\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-855494d94-jnhl5"
Jul 06 14:21:12.245153 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:12.245069    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/73b247d2-a95f-47c5-adc7-e511cbf91a3d-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-k5fj6\" (UID: \"73b247d2-a95f-47c5-adc7-e511cbf91a3d\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-k5fj6"
Jul 06 14:21:12.245153 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:12.245095    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/b0c48750-e081-4711-94d7-04ebce9ded67-hub\") pod \"cluster-proxy-proxy-agent-855494d94-jnhl5\" (UID: \"b0c48750-e081-4711-94d7-04ebce9ded67\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-855494d94-jnhl5"
Jul 06 14:21:12.245153 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:12.245121    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/73b247d2-a95f-47c5-adc7-e511cbf91a3d-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-k5fj6\" (UID: \"73b247d2-a95f-47c5-adc7-e511cbf91a3d\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-k5fj6"
Jul 06 14:21:12.245288 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:12.245226    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/b0c48750-e081-4711-94d7-04ebce9ded67-ca\") pod \"cluster-proxy-proxy-agent-855494d94-jnhl5\" (UID: \"b0c48750-e081-4711-94d7-04ebce9ded67\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-855494d94-jnhl5"
Jul 06 14:21:12.245288 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:12.245263    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bvmqf\" (UniqueName: \"kubernetes.io/projected/b0c48750-e081-4711-94d7-04ebce9ded67-kube-api-access-bvmqf\") pod \"cluster-proxy-proxy-agent-855494d94-jnhl5\" (UID: \"b0c48750-e081-4711-94d7-04ebce9ded67\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-855494d94-jnhl5"
Jul 06 14:21:12.245348 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:12.245324    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/b0c48750-e081-4711-94d7-04ebce9ded67-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-855494d94-jnhl5\" (UID: \"b0c48750-e081-4711-94d7-04ebce9ded67\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-855494d94-jnhl5"
Jul 06 14:21:12.261855 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:12.261831    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-lghzh"
Jul 06 14:21:12.346042 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:12.346012    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1ef4105d-8624-4b20-b82c-9130feb99981-image-registry-private-configuration\") pod \"image-registry-564974887b-25vkr\" (UID: \"1ef4105d-8624-4b20-b82c-9130feb99981\") " pod="openshift-image-registry/image-registry-564974887b-25vkr"
Jul 06 14:21:12.346185 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:12.346057    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/b0c48750-e081-4711-94d7-04ebce9ded67-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-855494d94-jnhl5\" (UID: \"b0c48750-e081-4711-94d7-04ebce9ded67\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-855494d94-jnhl5"
Jul 06 14:21:12.346185 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:12.346119    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1ef4105d-8624-4b20-b82c-9130feb99981-registry-tls\") pod \"image-registry-564974887b-25vkr\" (UID: \"1ef4105d-8624-4b20-b82c-9130feb99981\") " pod="openshift-image-registry/image-registry-564974887b-25vkr"
Jul 06 14:21:12.346185 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:12.346160    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m98vr\" (UniqueName: \"kubernetes.io/projected/1ef4105d-8624-4b20-b82c-9130feb99981-kube-api-access-m98vr\") pod \"image-registry-564974887b-25vkr\" (UID: \"1ef4105d-8624-4b20-b82c-9130feb99981\") " pod="openshift-image-registry/image-registry-564974887b-25vkr"
Jul 06 14:21:12.346324 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:12.346193    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b0c48750-e081-4711-94d7-04ebce9ded67-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-855494d94-jnhl5\" (UID: \"b0c48750-e081-4711-94d7-04ebce9ded67\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-855494d94-jnhl5"
Jul 06 14:21:12.346324 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:12.346232    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/b0c48750-e081-4711-94d7-04ebce9ded67-additional-service-ca\") pod \"cluster-proxy-proxy-agent-855494d94-jnhl5\" (UID: \"b0c48750-e081-4711-94d7-04ebce9ded67\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-855494d94-jnhl5"
Jul 06 14:21:12.346324 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:12.346297    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1ef4105d-8624-4b20-b82c-9130feb99981-registry-certificates\") pod \"image-registry-564974887b-25vkr\" (UID: \"1ef4105d-8624-4b20-b82c-9130feb99981\") " pod="openshift-image-registry/image-registry-564974887b-25vkr"
Jul 06 14:21:12.346461 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:12.346333    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1ef4105d-8624-4b20-b82c-9130feb99981-trusted-ca\") pod \"image-registry-564974887b-25vkr\" (UID: \"1ef4105d-8624-4b20-b82c-9130feb99981\") " pod="openshift-image-registry/image-registry-564974887b-25vkr"
Jul 06 14:21:12.346461 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:12.346372    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/73b247d2-a95f-47c5-adc7-e511cbf91a3d-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-k5fj6\" (UID: \"73b247d2-a95f-47c5-adc7-e511cbf91a3d\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-k5fj6"
Jul 06 14:21:12.346461 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:12.346417    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/b0c48750-e081-4711-94d7-04ebce9ded67-hub\") pod \"cluster-proxy-proxy-agent-855494d94-jnhl5\" (UID: \"b0c48750-e081-4711-94d7-04ebce9ded67\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-855494d94-jnhl5"
Jul 06 14:21:12.346634 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:12.346587    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1ef4105d-8624-4b20-b82c-9130feb99981-bound-sa-token\") pod \"image-registry-564974887b-25vkr\" (UID: \"1ef4105d-8624-4b20-b82c-9130feb99981\") " pod="openshift-image-registry/image-registry-564974887b-25vkr"
Jul 06 14:21:12.346691 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:12.346633    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/73b247d2-a95f-47c5-adc7-e511cbf91a3d-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-k5fj6\" (UID: \"73b247d2-a95f-47c5-adc7-e511cbf91a3d\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-k5fj6"
Jul 06 14:21:12.346691 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:12.346686    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/9019944c-f0a1-4309-af53-817d7f767b10-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-z4962\" (UID: \"9019944c-f0a1-4309-af53-817d7f767b10\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-z4962"
Jul 06 14:21:12.346806 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:12.346728    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c5lhr\" (UniqueName: \"kubernetes.io/projected/2855a4b2-bc67-4364-b3f0-40fa0c71c599-kube-api-access-c5lhr\") pod \"downloads-685bfb5f64-psvjq\" (UID: \"2855a4b2-bc67-4364-b3f0-40fa0c71c599\") " pod="openshift-console/downloads-685bfb5f64-psvjq"
Jul 06 14:21:12.346806 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:12.346772    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1ef4105d-8624-4b20-b82c-9130feb99981-installation-pull-secrets\") pod \"image-registry-564974887b-25vkr\" (UID: \"1ef4105d-8624-4b20-b82c-9130feb99981\") " pod="openshift-image-registry/image-registry-564974887b-25vkr"
Jul 06 14:21:12.346908 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:12.346816    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/b0c48750-e081-4711-94d7-04ebce9ded67-ca\") pod \"cluster-proxy-proxy-agent-855494d94-jnhl5\" (UID: \"b0c48750-e081-4711-94d7-04ebce9ded67\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-855494d94-jnhl5"
Jul 06 14:21:12.346908 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:12.346842    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1ef4105d-8624-4b20-b82c-9130feb99981-ca-trust-extracted\") pod \"image-registry-564974887b-25vkr\" (UID: \"1ef4105d-8624-4b20-b82c-9130feb99981\") " pod="openshift-image-registry/image-registry-564974887b-25vkr"
Jul 06 14:21:12.346908 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:12.346871    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bvmqf\" (UniqueName: \"kubernetes.io/projected/b0c48750-e081-4711-94d7-04ebce9ded67-kube-api-access-bvmqf\") pod \"cluster-proxy-proxy-agent-855494d94-jnhl5\" (UID: \"b0c48750-e081-4711-94d7-04ebce9ded67\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-855494d94-jnhl5"
Jul 06 14:21:12.347103 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:12.347072    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/b0c48750-e081-4711-94d7-04ebce9ded67-additional-service-ca\") pod \"cluster-proxy-proxy-agent-855494d94-jnhl5\" (UID: \"b0c48750-e081-4711-94d7-04ebce9ded67\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-855494d94-jnhl5"
Jul 06 14:21:12.347252 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:12.347192    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/73b247d2-a95f-47c5-adc7-e511cbf91a3d-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-k5fj6\" (UID: \"73b247d2-a95f-47c5-adc7-e511cbf91a3d\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-k5fj6"
Jul 06 14:21:12.349614 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:12.349566    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b0c48750-e081-4711-94d7-04ebce9ded67-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-855494d94-jnhl5\" (UID: \"b0c48750-e081-4711-94d7-04ebce9ded67\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-855494d94-jnhl5"
Jul 06 14:21:12.349802 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:12.349780    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/b0c48750-e081-4711-94d7-04ebce9ded67-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-855494d94-jnhl5\" (UID: \"b0c48750-e081-4711-94d7-04ebce9ded67\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-855494d94-jnhl5"
Jul 06 14:21:12.350015 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:12.349994    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/b0c48750-e081-4711-94d7-04ebce9ded67-hub\") pod \"cluster-proxy-proxy-agent-855494d94-jnhl5\" (UID: \"b0c48750-e081-4711-94d7-04ebce9ded67\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-855494d94-jnhl5"
Jul 06 14:21:12.350101 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:12.350077    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/b0c48750-e081-4711-94d7-04ebce9ded67-ca\") pod \"cluster-proxy-proxy-agent-855494d94-jnhl5\" (UID: \"b0c48750-e081-4711-94d7-04ebce9ded67\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-855494d94-jnhl5"
Jul 06 14:21:12.350185 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:12.350171    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/73b247d2-a95f-47c5-adc7-e511cbf91a3d-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-k5fj6\" (UID: \"73b247d2-a95f-47c5-adc7-e511cbf91a3d\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-k5fj6"
Jul 06 14:21:12.363051 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:12.363021    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bvmqf\" (UniqueName: \"kubernetes.io/projected/b0c48750-e081-4711-94d7-04ebce9ded67-kube-api-access-bvmqf\") pod \"cluster-proxy-proxy-agent-855494d94-jnhl5\" (UID: \"b0c48750-e081-4711-94d7-04ebce9ded67\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-855494d94-jnhl5"
Jul 06 14:21:12.392747 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:12.392718    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-k5fj6"
Jul 06 14:21:12.397892 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:12.397865    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-lghzh"]
Jul 06 14:21:12.401662 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:21:12.401640    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38ad328a_62dd_4a21_aaf9_3141c532fafb.slice/crio-50c035f26c18905edaf4acb5dc711651ea51caa4d88144ca0aa0df30032322f0 WatchSource:0}: Error finding container 50c035f26c18905edaf4acb5dc711651ea51caa4d88144ca0aa0df30032322f0: Status 404 returned error can't find the container with id 50c035f26c18905edaf4acb5dc711651ea51caa4d88144ca0aa0df30032322f0
Jul 06 14:21:12.408153 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:12.408131    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-855494d94-jnhl5"
Jul 06 14:21:12.448018 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:12.447982    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1ef4105d-8624-4b20-b82c-9130feb99981-image-registry-private-configuration\") pod \"image-registry-564974887b-25vkr\" (UID: \"1ef4105d-8624-4b20-b82c-9130feb99981\") " pod="openshift-image-registry/image-registry-564974887b-25vkr"
Jul 06 14:21:12.448145 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:12.448041    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1ef4105d-8624-4b20-b82c-9130feb99981-registry-tls\") pod \"image-registry-564974887b-25vkr\" (UID: \"1ef4105d-8624-4b20-b82c-9130feb99981\") " pod="openshift-image-registry/image-registry-564974887b-25vkr"
Jul 06 14:21:12.448145 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:12.448071    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m98vr\" (UniqueName: \"kubernetes.io/projected/1ef4105d-8624-4b20-b82c-9130feb99981-kube-api-access-m98vr\") pod \"image-registry-564974887b-25vkr\" (UID: \"1ef4105d-8624-4b20-b82c-9130feb99981\") " pod="openshift-image-registry/image-registry-564974887b-25vkr"
Jul 06 14:21:12.448145 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:12.448099    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1ef4105d-8624-4b20-b82c-9130feb99981-registry-certificates\") pod \"image-registry-564974887b-25vkr\" (UID: \"1ef4105d-8624-4b20-b82c-9130feb99981\") " pod="openshift-image-registry/image-registry-564974887b-25vkr"
Jul 06 14:21:12.448145 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:12.448133    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1ef4105d-8624-4b20-b82c-9130feb99981-trusted-ca\") pod \"image-registry-564974887b-25vkr\" (UID: \"1ef4105d-8624-4b20-b82c-9130feb99981\") " pod="openshift-image-registry/image-registry-564974887b-25vkr"
Jul 06 14:21:12.448281 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:12.448159    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1ef4105d-8624-4b20-b82c-9130feb99981-bound-sa-token\") pod \"image-registry-564974887b-25vkr\" (UID: \"1ef4105d-8624-4b20-b82c-9130feb99981\") " pod="openshift-image-registry/image-registry-564974887b-25vkr"
Jul 06 14:21:12.448281 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:12.448201    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/9019944c-f0a1-4309-af53-817d7f767b10-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-z4962\" (UID: \"9019944c-f0a1-4309-af53-817d7f767b10\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-z4962"
Jul 06 14:21:12.448281 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:12.448235    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c5lhr\" (UniqueName: \"kubernetes.io/projected/2855a4b2-bc67-4364-b3f0-40fa0c71c599-kube-api-access-c5lhr\") pod \"downloads-685bfb5f64-psvjq\" (UID: \"2855a4b2-bc67-4364-b3f0-40fa0c71c599\") " pod="openshift-console/downloads-685bfb5f64-psvjq"
Jul 06 14:21:12.448281 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:12.448258    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1ef4105d-8624-4b20-b82c-9130feb99981-installation-pull-secrets\") pod \"image-registry-564974887b-25vkr\" (UID: \"1ef4105d-8624-4b20-b82c-9130feb99981\") " pod="openshift-image-registry/image-registry-564974887b-25vkr"
Jul 06 14:21:12.448454 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:12.448302    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1ef4105d-8624-4b20-b82c-9130feb99981-ca-trust-extracted\") pod \"image-registry-564974887b-25vkr\" (UID: \"1ef4105d-8624-4b20-b82c-9130feb99981\") " pod="openshift-image-registry/image-registry-564974887b-25vkr"
Jul 06 14:21:12.448838 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:12.448778    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1ef4105d-8624-4b20-b82c-9130feb99981-ca-trust-extracted\") pod \"image-registry-564974887b-25vkr\" (UID: \"1ef4105d-8624-4b20-b82c-9130feb99981\") " pod="openshift-image-registry/image-registry-564974887b-25vkr"
Jul 06 14:21:12.449725 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:12.449470    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1ef4105d-8624-4b20-b82c-9130feb99981-trusted-ca\") pod \"image-registry-564974887b-25vkr\" (UID: \"1ef4105d-8624-4b20-b82c-9130feb99981\") " pod="openshift-image-registry/image-registry-564974887b-25vkr"
Jul 06 14:21:12.450009 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:12.449989    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1ef4105d-8624-4b20-b82c-9130feb99981-registry-certificates\") pod \"image-registry-564974887b-25vkr\" (UID: \"1ef4105d-8624-4b20-b82c-9130feb99981\") " pod="openshift-image-registry/image-registry-564974887b-25vkr"
Jul 06 14:21:12.453167 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:12.452493    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1ef4105d-8624-4b20-b82c-9130feb99981-registry-tls\") pod \"image-registry-564974887b-25vkr\" (UID: \"1ef4105d-8624-4b20-b82c-9130feb99981\") " pod="openshift-image-registry/image-registry-564974887b-25vkr"
Jul 06 14:21:12.453167 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:12.452537    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1ef4105d-8624-4b20-b82c-9130feb99981-installation-pull-secrets\") pod \"image-registry-564974887b-25vkr\" (UID: \"1ef4105d-8624-4b20-b82c-9130feb99981\") " pod="openshift-image-registry/image-registry-564974887b-25vkr"
Jul 06 14:21:12.453167 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:12.453071    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/9019944c-f0a1-4309-af53-817d7f767b10-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-z4962\" (UID: \"9019944c-f0a1-4309-af53-817d7f767b10\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-z4962"
Jul 06 14:21:12.455118 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:12.455069    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1ef4105d-8624-4b20-b82c-9130feb99981-image-registry-private-configuration\") pod \"image-registry-564974887b-25vkr\" (UID: \"1ef4105d-8624-4b20-b82c-9130feb99981\") " pod="openshift-image-registry/image-registry-564974887b-25vkr"
Jul 06 14:21:12.464509 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:12.464423    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1ef4105d-8624-4b20-b82c-9130feb99981-bound-sa-token\") pod \"image-registry-564974887b-25vkr\" (UID: \"1ef4105d-8624-4b20-b82c-9130feb99981\") " pod="openshift-image-registry/image-registry-564974887b-25vkr"
Jul 06 14:21:12.467038 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:12.466981    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m98vr\" (UniqueName: \"kubernetes.io/projected/1ef4105d-8624-4b20-b82c-9130feb99981-kube-api-access-m98vr\") pod \"image-registry-564974887b-25vkr\" (UID: \"1ef4105d-8624-4b20-b82c-9130feb99981\") " pod="openshift-image-registry/image-registry-564974887b-25vkr"
Jul 06 14:21:12.467149 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:12.467069    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c5lhr\" (UniqueName: \"kubernetes.io/projected/2855a4b2-bc67-4364-b3f0-40fa0c71c599-kube-api-access-c5lhr\") pod \"downloads-685bfb5f64-psvjq\" (UID: \"2855a4b2-bc67-4364-b3f0-40fa0c71c599\") " pod="openshift-console/downloads-685bfb5f64-psvjq"
Jul 06 14:21:12.499481 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:12.499417    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-685bfb5f64-psvjq"
Jul 06 14:21:12.507503 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:12.507471    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-z4962"
Jul 06 14:21:12.512911 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:12.512874    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-564974887b-25vkr"
Jul 06 14:21:12.539619 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:12.539559    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-k5fj6"]
Jul 06 14:21:12.542383 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:21:12.542309    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73b247d2_a95f_47c5_adc7_e511cbf91a3d.slice/crio-71c112ab47ce28652475d08ca7fdfa775dfdb3bac5ff8053c468fbe0bde72478 WatchSource:0}: Error finding container 71c112ab47ce28652475d08ca7fdfa775dfdb3bac5ff8053c468fbe0bde72478: Status 404 returned error can't find the container with id 71c112ab47ce28652475d08ca7fdfa775dfdb3bac5ff8053c468fbe0bde72478
Jul 06 14:21:12.586329 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:12.579042    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-855494d94-jnhl5"]
Jul 06 14:21:12.612855 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:12.611395    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6bffdf9778-k5fj6" event={"ID":"73b247d2-a95f-47c5-adc7-e511cbf91a3d","Type":"ContainerStarted","Data":"71c112ab47ce28652475d08ca7fdfa775dfdb3bac5ff8053c468fbe0bde72478"}
Jul 06 14:21:12.613306 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:21:12.613258    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb0c48750_e081_4711_94d7_04ebce9ded67.slice/crio-06861a7fa92a8e26a536db05ade8fa24cdbd985e2c5e11215f1572742a72feb4 WatchSource:0}: Error finding container 06861a7fa92a8e26a536db05ade8fa24cdbd985e2c5e11215f1572742a72feb4: Status 404 returned error can't find the container with id 06861a7fa92a8e26a536db05ade8fa24cdbd985e2c5e11215f1572742a72feb4
Jul 06 14:21:12.619623 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:12.619541    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lghzh" event={"ID":"38ad328a-62dd-4a21-aaf9-3141c532fafb","Type":"ContainerStarted","Data":"50c035f26c18905edaf4acb5dc711651ea51caa4d88144ca0aa0df30032322f0"}
Jul 06 14:21:12.697058 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:12.697026    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-685bfb5f64-psvjq"]
Jul 06 14:21:12.700668 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:21:12.700639    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2855a4b2_bc67_4364_b3f0_40fa0c71c599.slice/crio-195ef6918ac7ab85693b133ac157ff3f0aeed45846b3c3b1a685118225219375 WatchSource:0}: Error finding container 195ef6918ac7ab85693b133ac157ff3f0aeed45846b3c3b1a685118225219375: Status 404 returned error can't find the container with id 195ef6918ac7ab85693b133ac157ff3f0aeed45846b3c3b1a685118225219375
Jul 06 14:21:12.939059 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:12.929081    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-z4962"]
Jul 06 14:21:12.939059 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:12.930546    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-564974887b-25vkr"]
Jul 06 14:21:12.939059 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:21:12.930919    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9019944c_f0a1_4309_af53_817d7f767b10.slice/crio-307bcc4d324b86941dccb14aa539394783787a7f2fd8d457723df5d5d01e1159 WatchSource:0}: Error finding container 307bcc4d324b86941dccb14aa539394783787a7f2fd8d457723df5d5d01e1159: Status 404 returned error can't find the container with id 307bcc4d324b86941dccb14aa539394783787a7f2fd8d457723df5d5d01e1159
Jul 06 14:21:12.939059 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:21:12.935947    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ef4105d_8624_4b20_b82c_9130feb99981.slice/crio-c2d497da7b992074f8377f433eff8d3b7c9a25ac82fae88ba76a235545c4cf50 WatchSource:0}: Error finding container c2d497da7b992074f8377f433eff8d3b7c9a25ac82fae88ba76a235545c4cf50: Status 404 returned error can't find the container with id c2d497da7b992074f8377f433eff8d3b7c9a25ac82fae88ba76a235545c4cf50
Jul 06 14:21:13.639198 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:13.638089    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-564974887b-25vkr" event={"ID":"1ef4105d-8624-4b20-b82c-9130feb99981","Type":"ContainerStarted","Data":"63b121eacf2c25e8b18bc86ac09ceff1db1a5103bcc566ff147dc8a43633e847"}
Jul 06 14:21:13.639198 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:13.638159    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-564974887b-25vkr" event={"ID":"1ef4105d-8624-4b20-b82c-9130feb99981","Type":"ContainerStarted","Data":"c2d497da7b992074f8377f433eff8d3b7c9a25ac82fae88ba76a235545c4cf50"}
Jul 06 14:21:13.639198 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:13.639125    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-564974887b-25vkr"
Jul 06 14:21:13.644314 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:13.644199    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-z4962" event={"ID":"9019944c-f0a1-4309-af53-817d7f767b10","Type":"ContainerStarted","Data":"307bcc4d324b86941dccb14aa539394783787a7f2fd8d457723df5d5d01e1159"}
Jul 06 14:21:13.651861 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:13.651766    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-685bfb5f64-psvjq" event={"ID":"2855a4b2-bc67-4364-b3f0-40fa0c71c599","Type":"ContainerStarted","Data":"195ef6918ac7ab85693b133ac157ff3f0aeed45846b3c3b1a685118225219375"}
Jul 06 14:21:13.654244 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:13.654217    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-855494d94-jnhl5" event={"ID":"b0c48750-e081-4711-94d7-04ebce9ded67","Type":"ContainerStarted","Data":"06861a7fa92a8e26a536db05ade8fa24cdbd985e2c5e11215f1572742a72feb4"}
Jul 06 14:21:13.673508 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:13.673160    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-564974887b-25vkr" podStartSLOduration=1.673137916 podStartE2EDuration="1.673137916s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:21:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:21:13.669762992 +0000 UTC m=+95.977055087" watchObservedRunningTime="2026-07-06 14:21:13.673137916 +0000 UTC m=+95.980429994"
Jul 06 14:21:18.501048 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:18.501015    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-l87kx"
Jul 06 14:21:20.692331 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:20.692289    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6bffdf9778-k5fj6" event={"ID":"73b247d2-a95f-47c5-adc7-e511cbf91a3d","Type":"ContainerStarted","Data":"2e690ae19de9c16b421ad82b78ca381b25f284cc59046ad62857daf4d18da388"}
Jul 06 14:21:20.694527 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:20.694492    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lghzh" event={"ID":"38ad328a-62dd-4a21-aaf9-3141c532fafb","Type":"ContainerStarted","Data":"90f9a217fb6ab4352156049b72bca5741d43ef454eb8c64871a9ecd8bf119ca0"}
Jul 06 14:21:20.696489 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:20.696294    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-z4962" event={"ID":"9019944c-f0a1-4309-af53-817d7f767b10","Type":"ContainerStarted","Data":"5ed3fdd13d8f6aaac9e45d4702d274ac438fe9297095c10d8cdb97b4dd4fda78"}
Jul 06 14:21:20.696489 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:20.696457    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-z4962"
Jul 06 14:21:20.699237 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:20.699214    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-855494d94-jnhl5" event={"ID":"b0c48750-e081-4711-94d7-04ebce9ded67","Type":"ContainerStarted","Data":"046d0f99390d1d9e5e4a8f3b3051e2f46b5270b6b3d7e8b61994354766dbd225"}
Jul 06 14:21:20.699339 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:20.699250    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-855494d94-jnhl5" event={"ID":"b0c48750-e081-4711-94d7-04ebce9ded67","Type":"ContainerStarted","Data":"52b1b8fbb91e71c7af49905198483df1e34c9d3cc48a337ada1fe4e217c72a38"}
Jul 06 14:21:20.699339 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:20.699269    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-855494d94-jnhl5" event={"ID":"b0c48750-e081-4711-94d7-04ebce9ded67","Type":"ContainerStarted","Data":"09f43c1e0cc54693baf0db73d1812234accb68e19de35bf0f522265bd10b69b8"}
Jul 06 14:21:20.702655 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:20.702636    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-z4962"
Jul 06 14:21:20.718692 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:20.718594    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6bffdf9778-k5fj6" podStartSLOduration=1.1220746639999999 podStartE2EDuration="8.71854592s" totalImagesPullingTime="7.596471256s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:21:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:21:20.71702142 +0000 UTC m=+103.024313515" watchObservedRunningTime="2026-07-06 14:21:20.71854592 +0000 UTC m=+103.025838002"
Jul 06 14:21:20.746077 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:20.745977    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-z4962" podStartSLOduration=1.537932247 podStartE2EDuration="8.74595471s" totalImagesPullingTime="7.208022463s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:21:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:21:20.745357987 +0000 UTC m=+103.052650075" watchObservedRunningTime="2026-07-06 14:21:20.74595471 +0000 UTC m=+103.053246793"
Jul 06 14:21:21.027300 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:21.027265    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e1c99840-57b6-4e0b-8e6d-80bca474168d-metrics-tls\") pod \"dns-default-kfvv5\" (UID: \"e1c99840-57b6-4e0b-8e6d-80bca474168d\") " pod="openshift-dns/dns-default-kfvv5"
Jul 06 14:21:21.029782 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:21.029758    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e1c99840-57b6-4e0b-8e6d-80bca474168d-metrics-tls\") pod \"dns-default-kfvv5\" (UID: \"e1c99840-57b6-4e0b-8e6d-80bca474168d\") " pod="openshift-dns/dns-default-kfvv5"
Jul 06 14:21:21.250466 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:21.250388    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-tng92\""
Jul 06 14:21:21.257819 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:21.257787    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-kfvv5"
Jul 06 14:21:21.521790 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:21.521714    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-kfvv5"]
Jul 06 14:21:21.691895 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:21:21.691857    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1c99840_57b6_4e0b_8e6d_80bca474168d.slice/crio-cbb3914dc85ec76d6ed8a51e063c767b6a4aa8c18056e15fd5663cd0d2d9836e WatchSource:0}: Error finding container cbb3914dc85ec76d6ed8a51e063c767b6a4aa8c18056e15fd5663cd0d2d9836e: Status 404 returned error can't find the container with id cbb3914dc85ec76d6ed8a51e063c767b6a4aa8c18056e15fd5663cd0d2d9836e
Jul 06 14:21:21.703733 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:21.703701    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-kfvv5" event={"ID":"e1c99840-57b6-4e0b-8e6d-80bca474168d","Type":"ContainerStarted","Data":"cbb3914dc85ec76d6ed8a51e063c767b6a4aa8c18056e15fd5663cd0d2d9836e"}
Jul 06 14:21:21.728861 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:21.728809    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-855494d94-jnhl5" podStartSLOduration=2.183883712 podStartE2EDuration="9.728793463s" totalImagesPullingTime="7.544909751s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:21:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:21:21.728236735 +0000 UTC m=+104.035528810" watchObservedRunningTime="2026-07-06 14:21:21.728793463 +0000 UTC m=+104.036085544"
Jul 06 14:21:22.110275 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:22.110189    2559 patch_prober.go:28] interesting pod/image-registry-6b894bb7d5-8nmsf container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 06 14:21:22.110275 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:22.110251    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-6b894bb7d5-8nmsf" podUID="606a9ffc-10e9-4114-ad7b-752ab0272a0d" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 06 14:21:22.712152 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:22.712066    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lghzh" event={"ID":"38ad328a-62dd-4a21-aaf9-3141c532fafb","Type":"ContainerStarted","Data":"d4944e834c192a46985e8bcbee84059d6e0a524488f02b370ece70fdd4c39c3d"}
Jul 06 14:21:26.763966 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:26.763930    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-tq6gr"]
Jul 06 14:21:26.767228 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:26.767204    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-tq6gr"
Jul 06 14:21:26.770338 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:26.770312    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 06 14:21:26.770674 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:26.770632    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 06 14:21:26.770785 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:26.770688    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 06 14:21:26.770938 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:26.770922    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-z62w8\""
Jul 06 14:21:26.771269 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:26.771250    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 06 14:21:26.771372 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:26.771329    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 06 14:21:26.773998 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:26.773980    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 06 14:21:26.776369 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:26.776346    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/0383f486-87cb-45e9-aa3d-99bf4066cd9d-node-exporter-accelerators-collector-config\") pod \"node-exporter-tq6gr\" (UID: \"0383f486-87cb-45e9-aa3d-99bf4066cd9d\") " pod="openshift-monitoring/node-exporter-tq6gr"
Jul 06 14:21:26.776463 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:26.776391    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0383f486-87cb-45e9-aa3d-99bf4066cd9d-sys\") pod \"node-exporter-tq6gr\" (UID: \"0383f486-87cb-45e9-aa3d-99bf4066cd9d\") " pod="openshift-monitoring/node-exporter-tq6gr"
Jul 06 14:21:26.776463 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:26.776418    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/0383f486-87cb-45e9-aa3d-99bf4066cd9d-node-exporter-tls\") pod \"node-exporter-tq6gr\" (UID: \"0383f486-87cb-45e9-aa3d-99bf4066cd9d\") " pod="openshift-monitoring/node-exporter-tq6gr"
Jul 06 14:21:26.776554 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:26.776467    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/0383f486-87cb-45e9-aa3d-99bf4066cd9d-root\") pod \"node-exporter-tq6gr\" (UID: \"0383f486-87cb-45e9-aa3d-99bf4066cd9d\") " pod="openshift-monitoring/node-exporter-tq6gr"
Jul 06 14:21:26.776617 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:26.776602    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0383f486-87cb-45e9-aa3d-99bf4066cd9d-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-tq6gr\" (UID: \"0383f486-87cb-45e9-aa3d-99bf4066cd9d\") " pod="openshift-monitoring/node-exporter-tq6gr"
Jul 06 14:21:26.776676 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:26.776638    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0383f486-87cb-45e9-aa3d-99bf4066cd9d-metrics-client-ca\") pod \"node-exporter-tq6gr\" (UID: \"0383f486-87cb-45e9-aa3d-99bf4066cd9d\") " pod="openshift-monitoring/node-exporter-tq6gr"
Jul 06 14:21:26.776676 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:26.776663    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dzd95\" (UniqueName: \"kubernetes.io/projected/0383f486-87cb-45e9-aa3d-99bf4066cd9d-kube-api-access-dzd95\") pod \"node-exporter-tq6gr\" (UID: \"0383f486-87cb-45e9-aa3d-99bf4066cd9d\") " pod="openshift-monitoring/node-exporter-tq6gr"
Jul 06 14:21:26.776779 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:26.776695    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/0383f486-87cb-45e9-aa3d-99bf4066cd9d-node-exporter-textfile\") pod \"node-exporter-tq6gr\" (UID: \"0383f486-87cb-45e9-aa3d-99bf4066cd9d\") " pod="openshift-monitoring/node-exporter-tq6gr"
Jul 06 14:21:26.776779 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:26.776713    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/0383f486-87cb-45e9-aa3d-99bf4066cd9d-node-exporter-wtmp\") pod \"node-exporter-tq6gr\" (UID: \"0383f486-87cb-45e9-aa3d-99bf4066cd9d\") " pod="openshift-monitoring/node-exporter-tq6gr"
Jul 06 14:21:26.877506 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:26.877465    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0383f486-87cb-45e9-aa3d-99bf4066cd9d-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-tq6gr\" (UID: \"0383f486-87cb-45e9-aa3d-99bf4066cd9d\") " pod="openshift-monitoring/node-exporter-tq6gr"
Jul 06 14:21:26.877718 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:26.877514    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0383f486-87cb-45e9-aa3d-99bf4066cd9d-metrics-client-ca\") pod \"node-exporter-tq6gr\" (UID: \"0383f486-87cb-45e9-aa3d-99bf4066cd9d\") " pod="openshift-monitoring/node-exporter-tq6gr"
Jul 06 14:21:26.877718 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:26.877537    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dzd95\" (UniqueName: \"kubernetes.io/projected/0383f486-87cb-45e9-aa3d-99bf4066cd9d-kube-api-access-dzd95\") pod \"node-exporter-tq6gr\" (UID: \"0383f486-87cb-45e9-aa3d-99bf4066cd9d\") " pod="openshift-monitoring/node-exporter-tq6gr"
Jul 06 14:21:26.877718 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:26.877593    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/0383f486-87cb-45e9-aa3d-99bf4066cd9d-node-exporter-textfile\") pod \"node-exporter-tq6gr\" (UID: \"0383f486-87cb-45e9-aa3d-99bf4066cd9d\") " pod="openshift-monitoring/node-exporter-tq6gr"
Jul 06 14:21:26.877718 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:26.877614    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/0383f486-87cb-45e9-aa3d-99bf4066cd9d-node-exporter-wtmp\") pod \"node-exporter-tq6gr\" (UID: \"0383f486-87cb-45e9-aa3d-99bf4066cd9d\") " pod="openshift-monitoring/node-exporter-tq6gr"
Jul 06 14:21:26.877718 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:26.877648    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/0383f486-87cb-45e9-aa3d-99bf4066cd9d-node-exporter-accelerators-collector-config\") pod \"node-exporter-tq6gr\" (UID: \"0383f486-87cb-45e9-aa3d-99bf4066cd9d\") " pod="openshift-monitoring/node-exporter-tq6gr"
Jul 06 14:21:26.877718 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:26.877683    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0383f486-87cb-45e9-aa3d-99bf4066cd9d-sys\") pod \"node-exporter-tq6gr\" (UID: \"0383f486-87cb-45e9-aa3d-99bf4066cd9d\") " pod="openshift-monitoring/node-exporter-tq6gr"
Jul 06 14:21:26.877718 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:26.877708    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/0383f486-87cb-45e9-aa3d-99bf4066cd9d-node-exporter-tls\") pod \"node-exporter-tq6gr\" (UID: \"0383f486-87cb-45e9-aa3d-99bf4066cd9d\") " pod="openshift-monitoring/node-exporter-tq6gr"
Jul 06 14:21:26.878041 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:26.877738    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/0383f486-87cb-45e9-aa3d-99bf4066cd9d-root\") pod \"node-exporter-tq6gr\" (UID: \"0383f486-87cb-45e9-aa3d-99bf4066cd9d\") " pod="openshift-monitoring/node-exporter-tq6gr"
Jul 06 14:21:26.878041 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:26.877877    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/0383f486-87cb-45e9-aa3d-99bf4066cd9d-root\") pod \"node-exporter-tq6gr\" (UID: \"0383f486-87cb-45e9-aa3d-99bf4066cd9d\") " pod="openshift-monitoring/node-exporter-tq6gr"
Jul 06 14:21:26.878041 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:26.877975    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0383f486-87cb-45e9-aa3d-99bf4066cd9d-sys\") pod \"node-exporter-tq6gr\" (UID: \"0383f486-87cb-45e9-aa3d-99bf4066cd9d\") " pod="openshift-monitoring/node-exporter-tq6gr"
Jul 06 14:21:26.878041 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:26.878013    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/0383f486-87cb-45e9-aa3d-99bf4066cd9d-node-exporter-wtmp\") pod \"node-exporter-tq6gr\" (UID: \"0383f486-87cb-45e9-aa3d-99bf4066cd9d\") " pod="openshift-monitoring/node-exporter-tq6gr"
Jul 06 14:21:26.878284 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:21:26.878077    2559 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 06 14:21:26.878284 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:21:26.878148    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0383f486-87cb-45e9-aa3d-99bf4066cd9d-node-exporter-tls podName:0383f486-87cb-45e9-aa3d-99bf4066cd9d nodeName:}" failed. No retries permitted until 2026-07-06 14:21:27.378130593 +0000 UTC m=+109.685422657 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/0383f486-87cb-45e9-aa3d-99bf4066cd9d-node-exporter-tls") pod "node-exporter-tq6gr" (UID: "0383f486-87cb-45e9-aa3d-99bf4066cd9d") : secret "node-exporter-tls" not found
Jul 06 14:21:26.878284 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:26.878189    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0383f486-87cb-45e9-aa3d-99bf4066cd9d-metrics-client-ca\") pod \"node-exporter-tq6gr\" (UID: \"0383f486-87cb-45e9-aa3d-99bf4066cd9d\") " pod="openshift-monitoring/node-exporter-tq6gr"
Jul 06 14:21:26.878284 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:26.878273    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/0383f486-87cb-45e9-aa3d-99bf4066cd9d-node-exporter-textfile\") pod \"node-exporter-tq6gr\" (UID: \"0383f486-87cb-45e9-aa3d-99bf4066cd9d\") " pod="openshift-monitoring/node-exporter-tq6gr"
Jul 06 14:21:26.878482 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:26.878462    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/0383f486-87cb-45e9-aa3d-99bf4066cd9d-node-exporter-accelerators-collector-config\") pod \"node-exporter-tq6gr\" (UID: \"0383f486-87cb-45e9-aa3d-99bf4066cd9d\") " pod="openshift-monitoring/node-exporter-tq6gr"
Jul 06 14:21:26.880445 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:26.880425    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0383f486-87cb-45e9-aa3d-99bf4066cd9d-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-tq6gr\" (UID: \"0383f486-87cb-45e9-aa3d-99bf4066cd9d\") " pod="openshift-monitoring/node-exporter-tq6gr"
Jul 06 14:21:26.888500 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:26.888476    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dzd95\" (UniqueName: \"kubernetes.io/projected/0383f486-87cb-45e9-aa3d-99bf4066cd9d-kube-api-access-dzd95\") pod \"node-exporter-tq6gr\" (UID: \"0383f486-87cb-45e9-aa3d-99bf4066cd9d\") " pod="openshift-monitoring/node-exporter-tq6gr"
Jul 06 14:21:27.381240 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:27.381194    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/0383f486-87cb-45e9-aa3d-99bf4066cd9d-node-exporter-tls\") pod \"node-exporter-tq6gr\" (UID: \"0383f486-87cb-45e9-aa3d-99bf4066cd9d\") " pod="openshift-monitoring/node-exporter-tq6gr"
Jul 06 14:21:27.383767 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:27.383739    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/0383f486-87cb-45e9-aa3d-99bf4066cd9d-node-exporter-tls\") pod \"node-exporter-tq6gr\" (UID: \"0383f486-87cb-45e9-aa3d-99bf4066cd9d\") " pod="openshift-monitoring/node-exporter-tq6gr"
Jul 06 14:21:27.678714 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:27.678679    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-tq6gr"
Jul 06 14:21:32.109500 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:32.109463    2559 patch_prober.go:28] interesting pod/image-registry-6b894bb7d5-8nmsf container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 06 14:21:32.109884 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:32.109516    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-6b894bb7d5-8nmsf" podUID="606a9ffc-10e9-4114-ad7b-752ab0272a0d" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 06 14:21:32.441009 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:21:32.440966    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0383f486_87cb_45e9_aa3d_99bf4066cd9d.slice/crio-ab21b7c912f859008f70261f97660d06827752c625fcdc3ab6380b94cd9c8a24 WatchSource:0}: Error finding container ab21b7c912f859008f70261f97660d06827752c625fcdc3ab6380b94cd9c8a24: Status 404 returned error can't find the container with id ab21b7c912f859008f70261f97660d06827752c625fcdc3ab6380b94cd9c8a24
Jul 06 14:21:32.519138 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:32.518970    2559 patch_prober.go:28] interesting pod/image-registry-564974887b-25vkr container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 06 14:21:32.519138 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:32.519095    2559 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-564974887b-25vkr" podUID="1ef4105d-8624-4b20-b82c-9130feb99981" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 06 14:21:32.754626 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:32.754492    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-tq6gr" event={"ID":"0383f486-87cb-45e9-aa3d-99bf4066cd9d","Type":"ContainerStarted","Data":"ab21b7c912f859008f70261f97660d06827752c625fcdc3ab6380b94cd9c8a24"}
Jul 06 14:21:32.756901 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:32.756869    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lghzh" event={"ID":"38ad328a-62dd-4a21-aaf9-3141c532fafb","Type":"ContainerStarted","Data":"e8754dab91dfe6f26b6a93d2b5d324be19e0971f01a0ee93eb6a647e3a53643c"}
Jul 06 14:21:32.758593 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:32.758543    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-685bfb5f64-psvjq" event={"ID":"2855a4b2-bc67-4364-b3f0-40fa0c71c599","Type":"ContainerStarted","Data":"b2678df47547c76e4dc49a0e738a779cc5cc1477722983ec4a337cd61a8a8b8e"}
Jul 06 14:21:32.758716 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:32.758611    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-685bfb5f64-psvjq"
Jul 06 14:21:32.760927 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:32.760899    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-kfvv5" event={"ID":"e1c99840-57b6-4e0b-8e6d-80bca474168d","Type":"ContainerStarted","Data":"70b1180b26775713506efe9cc135571d50ce6762b639ce323e19342395d83740"}
Jul 06 14:21:32.761041 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:32.760947    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-kfvv5" event={"ID":"e1c99840-57b6-4e0b-8e6d-80bca474168d","Type":"ContainerStarted","Data":"cef4c0f2cf9cd353956853624bbc9471d3f378b1e2922a098df786f3d145e18b"}
Jul 06 14:21:32.761041 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:32.760994    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-kfvv5"
Jul 06 14:21:32.767323 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:32.767300    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-685bfb5f64-psvjq"
Jul 06 14:21:32.780253 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:32.780200    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-lghzh" podStartSLOduration=1.989418079 podStartE2EDuration="21.780186223s" totalImagesPullingTime="19.790768144s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 14:21:11 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:21:32.77844066 +0000 UTC m=+115.085732745" watchObservedRunningTime="2026-07-06 14:21:32.780186223 +0000 UTC m=+115.087478304"
Jul 06 14:21:32.800491 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:32.800430    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-kfvv5" podStartSLOduration=65.082450285 podStartE2EDuration="1m15.800409268s" totalImagesPullingTime="10.717958983s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:20:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:21:32.798851543 +0000 UTC m=+115.106143637" watchObservedRunningTime="2026-07-06 14:21:32.800409268 +0000 UTC m=+115.107701351"
Jul 06 14:21:32.821609 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:32.821517    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-685bfb5f64-psvjq" podStartSLOduration=1.026871219 podStartE2EDuration="20.821495722s" totalImagesPullingTime="19.794624503s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:21:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:21:32.821318505 +0000 UTC m=+115.128610589" watchObservedRunningTime="2026-07-06 14:21:32.821495722 +0000 UTC m=+115.128787805"
Jul 06 14:21:33.767712 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:33.767555    2559 generic.go:356] "Generic (PLEG): container finished" podID="0383f486-87cb-45e9-aa3d-99bf4066cd9d" containerID="34a89ed28b98887450efd605b5adc07a7db06e4e3e9637b962ac0eb55f96c4b7" exitCode=0
Jul 06 14:21:33.767712 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:33.767617    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-tq6gr" event={"ID":"0383f486-87cb-45e9-aa3d-99bf4066cd9d","Type":"ContainerDied","Data":"34a89ed28b98887450efd605b5adc07a7db06e4e3e9637b962ac0eb55f96c4b7"}
Jul 06 14:21:34.774789 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:34.774745    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-tq6gr" event={"ID":"0383f486-87cb-45e9-aa3d-99bf4066cd9d","Type":"ContainerStarted","Data":"83506e7062275995111b344ca9a6ed2838c9f265116cf31bcaff9df3274c1e62"}
Jul 06 14:21:34.775303 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:34.774816    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-tq6gr" event={"ID":"0383f486-87cb-45e9-aa3d-99bf4066cd9d","Type":"ContainerStarted","Data":"3714e795f017a2374cedcf7113dfc8a60726f27b540ee182e4015ca050afc715"}
Jul 06 14:21:34.813588 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:34.813499    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-tq6gr" podStartSLOduration=7.6650518640000005 podStartE2EDuration="8.813477996s" totalImagesPullingTime="1.059636497s" totalInitContainerRuntime="88.789635ms" isStatefulPod=true podCreationTimestamp="2026-07-06 14:21:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:21:34.813258017 +0000 UTC m=+117.120550124" watchObservedRunningTime="2026-07-06 14:21:34.813477996 +0000 UTC m=+117.120770079"
Jul 06 14:21:35.670684 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:35.670650    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-564974887b-25vkr"
Jul 06 14:21:37.125662 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:37.125611    2559 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-6b894bb7d5-8nmsf" podUID="606a9ffc-10e9-4114-ad7b-752ab0272a0d" containerName="registry" containerID="cri-o://8dfba37aeea2c0ba20876ac55afeedadd9d517da5830bda4289653bf92a3603e" gracePeriod=30
Jul 06 14:21:37.414853 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:37.414820    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6b894bb7d5-8nmsf"
Jul 06 14:21:37.475714 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:37.475671    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/606a9ffc-10e9-4114-ad7b-752ab0272a0d-image-registry-private-configuration\") pod \"606a9ffc-10e9-4114-ad7b-752ab0272a0d\" (UID: \"606a9ffc-10e9-4114-ad7b-752ab0272a0d\") "
Jul 06 14:21:37.475900 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:37.475721    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/606a9ffc-10e9-4114-ad7b-752ab0272a0d-registry-tls\") pod \"606a9ffc-10e9-4114-ad7b-752ab0272a0d\" (UID: \"606a9ffc-10e9-4114-ad7b-752ab0272a0d\") "
Jul 06 14:21:37.475900 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:37.475749    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/606a9ffc-10e9-4114-ad7b-752ab0272a0d-bound-sa-token\") pod \"606a9ffc-10e9-4114-ad7b-752ab0272a0d\" (UID: \"606a9ffc-10e9-4114-ad7b-752ab0272a0d\") "
Jul 06 14:21:37.475900 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:37.475783    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-js2fr\" (UniqueName: \"kubernetes.io/projected/606a9ffc-10e9-4114-ad7b-752ab0272a0d-kube-api-access-js2fr\") pod \"606a9ffc-10e9-4114-ad7b-752ab0272a0d\" (UID: \"606a9ffc-10e9-4114-ad7b-752ab0272a0d\") "
Jul 06 14:21:37.475900 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:37.475821    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/606a9ffc-10e9-4114-ad7b-752ab0272a0d-registry-certificates\") pod \"606a9ffc-10e9-4114-ad7b-752ab0272a0d\" (UID: \"606a9ffc-10e9-4114-ad7b-752ab0272a0d\") "
Jul 06 14:21:37.475900 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:37.475848    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/606a9ffc-10e9-4114-ad7b-752ab0272a0d-ca-trust-extracted\") pod \"606a9ffc-10e9-4114-ad7b-752ab0272a0d\" (UID: \"606a9ffc-10e9-4114-ad7b-752ab0272a0d\") "
Jul 06 14:21:37.475900 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:37.475880    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/606a9ffc-10e9-4114-ad7b-752ab0272a0d-trusted-ca\") pod \"606a9ffc-10e9-4114-ad7b-752ab0272a0d\" (UID: \"606a9ffc-10e9-4114-ad7b-752ab0272a0d\") "
Jul 06 14:21:37.476219 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:37.475972    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/606a9ffc-10e9-4114-ad7b-752ab0272a0d-installation-pull-secrets\") pod \"606a9ffc-10e9-4114-ad7b-752ab0272a0d\" (UID: \"606a9ffc-10e9-4114-ad7b-752ab0272a0d\") "
Jul 06 14:21:37.476473 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:37.476437    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/606a9ffc-10e9-4114-ad7b-752ab0272a0d-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "606a9ffc-10e9-4114-ad7b-752ab0272a0d" (UID: "606a9ffc-10e9-4114-ad7b-752ab0272a0d"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 14:21:37.477027 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:37.476973    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/606a9ffc-10e9-4114-ad7b-752ab0272a0d-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "606a9ffc-10e9-4114-ad7b-752ab0272a0d" (UID: "606a9ffc-10e9-4114-ad7b-752ab0272a0d"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 14:21:37.479232 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:37.479180    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/606a9ffc-10e9-4114-ad7b-752ab0272a0d-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "606a9ffc-10e9-4114-ad7b-752ab0272a0d" (UID: "606a9ffc-10e9-4114-ad7b-752ab0272a0d"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 14:21:37.479795 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:37.479408    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/606a9ffc-10e9-4114-ad7b-752ab0272a0d-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "606a9ffc-10e9-4114-ad7b-752ab0272a0d" (UID: "606a9ffc-10e9-4114-ad7b-752ab0272a0d"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 14:21:37.479795 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:37.479601    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/606a9ffc-10e9-4114-ad7b-752ab0272a0d-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "606a9ffc-10e9-4114-ad7b-752ab0272a0d" (UID: "606a9ffc-10e9-4114-ad7b-752ab0272a0d"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 14:21:37.479795 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:37.479658    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/606a9ffc-10e9-4114-ad7b-752ab0272a0d-kube-api-access-js2fr" (OuterVolumeSpecName: "kube-api-access-js2fr") pod "606a9ffc-10e9-4114-ad7b-752ab0272a0d" (UID: "606a9ffc-10e9-4114-ad7b-752ab0272a0d"). InnerVolumeSpecName "kube-api-access-js2fr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 14:21:37.480919 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:37.480896    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/606a9ffc-10e9-4114-ad7b-752ab0272a0d-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "606a9ffc-10e9-4114-ad7b-752ab0272a0d" (UID: "606a9ffc-10e9-4114-ad7b-752ab0272a0d"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 14:21:37.488697 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:37.488666    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/606a9ffc-10e9-4114-ad7b-752ab0272a0d-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "606a9ffc-10e9-4114-ad7b-752ab0272a0d" (UID: "606a9ffc-10e9-4114-ad7b-752ab0272a0d"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 14:21:37.576899 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:37.576850    2559 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/606a9ffc-10e9-4114-ad7b-752ab0272a0d-installation-pull-secrets\") on node \"ip-10-0-133-112.ec2.internal\" DevicePath \"\""
Jul 06 14:21:37.576899 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:37.576894    2559 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/606a9ffc-10e9-4114-ad7b-752ab0272a0d-image-registry-private-configuration\") on node \"ip-10-0-133-112.ec2.internal\" DevicePath \"\""
Jul 06 14:21:37.576899 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:37.576906    2559 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/606a9ffc-10e9-4114-ad7b-752ab0272a0d-registry-tls\") on node \"ip-10-0-133-112.ec2.internal\" DevicePath \"\""
Jul 06 14:21:37.577184 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:37.576918    2559 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/606a9ffc-10e9-4114-ad7b-752ab0272a0d-bound-sa-token\") on node \"ip-10-0-133-112.ec2.internal\" DevicePath \"\""
Jul 06 14:21:37.577184 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:37.576931    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-js2fr\" (UniqueName: \"kubernetes.io/projected/606a9ffc-10e9-4114-ad7b-752ab0272a0d-kube-api-access-js2fr\") on node \"ip-10-0-133-112.ec2.internal\" DevicePath \"\""
Jul 06 14:21:37.577184 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:37.576946    2559 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/606a9ffc-10e9-4114-ad7b-752ab0272a0d-registry-certificates\") on node \"ip-10-0-133-112.ec2.internal\" DevicePath \"\""
Jul 06 14:21:37.577184 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:37.576958    2559 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/606a9ffc-10e9-4114-ad7b-752ab0272a0d-ca-trust-extracted\") on node \"ip-10-0-133-112.ec2.internal\" DevicePath \"\""
Jul 06 14:21:37.577184 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:37.576971    2559 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/606a9ffc-10e9-4114-ad7b-752ab0272a0d-trusted-ca\") on node \"ip-10-0-133-112.ec2.internal\" DevicePath \"\""
Jul 06 14:21:37.788964 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:37.788921    2559 generic.go:356] "Generic (PLEG): container finished" podID="606a9ffc-10e9-4114-ad7b-752ab0272a0d" containerID="8dfba37aeea2c0ba20876ac55afeedadd9d517da5830bda4289653bf92a3603e" exitCode=0
Jul 06 14:21:37.789152 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:37.788976    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6b894bb7d5-8nmsf" event={"ID":"606a9ffc-10e9-4114-ad7b-752ab0272a0d","Type":"ContainerDied","Data":"8dfba37aeea2c0ba20876ac55afeedadd9d517da5830bda4289653bf92a3603e"}
Jul 06 14:21:37.789152 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:37.788980    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6b894bb7d5-8nmsf"
Jul 06 14:21:37.789152 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:37.789058    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6b894bb7d5-8nmsf" event={"ID":"606a9ffc-10e9-4114-ad7b-752ab0272a0d","Type":"ContainerDied","Data":"1ec27bd440006feb2a01cdb79bd4d9c5e75fb53263c404444088727ed9e57d4d"}
Jul 06 14:21:37.789152 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:37.789082    2559 scope.go:117] "RemoveContainer" containerID="8dfba37aeea2c0ba20876ac55afeedadd9d517da5830bda4289653bf92a3603e"
Jul 06 14:21:37.798893 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:37.798866    2559 scope.go:117] "RemoveContainer" containerID="8dfba37aeea2c0ba20876ac55afeedadd9d517da5830bda4289653bf92a3603e"
Jul 06 14:21:37.799249 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:21:37.799224    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8dfba37aeea2c0ba20876ac55afeedadd9d517da5830bda4289653bf92a3603e\": container with ID starting with 8dfba37aeea2c0ba20876ac55afeedadd9d517da5830bda4289653bf92a3603e not found: ID does not exist" containerID="8dfba37aeea2c0ba20876ac55afeedadd9d517da5830bda4289653bf92a3603e"
Jul 06 14:21:37.799342 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:37.799260    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8dfba37aeea2c0ba20876ac55afeedadd9d517da5830bda4289653bf92a3603e"} err="failed to get container status \"8dfba37aeea2c0ba20876ac55afeedadd9d517da5830bda4289653bf92a3603e\": rpc error: code = NotFound desc = could not find container \"8dfba37aeea2c0ba20876ac55afeedadd9d517da5830bda4289653bf92a3603e\": container with ID starting with 8dfba37aeea2c0ba20876ac55afeedadd9d517da5830bda4289653bf92a3603e not found: ID does not exist"
Jul 06 14:21:37.813604 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:37.813551    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6b894bb7d5-8nmsf"]
Jul 06 14:21:37.829197 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:37.829167    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-6b894bb7d5-8nmsf"]
Jul 06 14:21:38.228050 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:38.228010    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="606a9ffc-10e9-4114-ad7b-752ab0272a0d" path="/var/lib/kubelet/pods/606a9ffc-10e9-4114-ad7b-752ab0272a0d/volumes"
Jul 06 14:21:42.770353 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:42.770321    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-kfvv5"
Jul 06 14:21:47.968788 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:47.968753    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1c05cc5f-44d6-46b6-947a-9d221f282365-metrics-certs\") pod \"network-metrics-daemon-v94gg\" (UID: \"1c05cc5f-44d6-46b6-947a-9d221f282365\") " pod="openshift-multus/network-metrics-daemon-v94gg"
Jul 06 14:21:47.970982 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:47.970959    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1c05cc5f-44d6-46b6-947a-9d221f282365-metrics-certs\") pod \"network-metrics-daemon-v94gg\" (UID: \"1c05cc5f-44d6-46b6-947a-9d221f282365\") " pod="openshift-multus/network-metrics-daemon-v94gg"
Jul 06 14:21:48.144033 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:48.143993    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-nfstm\""
Jul 06 14:21:48.152043 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:48.152013    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v94gg"
Jul 06 14:21:48.425371 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:48.425315    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-v94gg"]
Jul 06 14:21:48.427516 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:21:48.427489    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c05cc5f_44d6_46b6_947a_9d221f282365.slice/crio-5ebfce0f79416c37cc3f894981f4cccd8761c172d453bb1318246b3f8edf0e7e WatchSource:0}: Error finding container 5ebfce0f79416c37cc3f894981f4cccd8761c172d453bb1318246b3f8edf0e7e: Status 404 returned error can't find the container with id 5ebfce0f79416c37cc3f894981f4cccd8761c172d453bb1318246b3f8edf0e7e
Jul 06 14:21:48.831101 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:48.831065    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-v94gg" event={"ID":"1c05cc5f-44d6-46b6-947a-9d221f282365","Type":"ContainerStarted","Data":"5ebfce0f79416c37cc3f894981f4cccd8761c172d453bb1318246b3f8edf0e7e"}
Jul 06 14:21:50.839557 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:50.839522    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-v94gg" event={"ID":"1c05cc5f-44d6-46b6-947a-9d221f282365","Type":"ContainerStarted","Data":"919b0c673637ec6e4768f7f39496a4983821fe65ccb21651434e2d67ea834e76"}
Jul 06 14:21:50.839955 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:50.839597    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-v94gg" event={"ID":"1c05cc5f-44d6-46b6-947a-9d221f282365","Type":"ContainerStarted","Data":"3080d6483c98e95bfd7336c09ba67ad07bbd2199a7baa26ce7e716995f057e46"}
Jul 06 14:21:50.859825 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:21:50.859769    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-v94gg" podStartSLOduration=130.293489417 podStartE2EDuration="2m11.859755165s" totalImagesPullingTime="1.566265748s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:19:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:21:50.858824928 +0000 UTC m=+133.166117014" watchObservedRunningTime="2026-07-06 14:21:50.859755165 +0000 UTC m=+133.167047247"
Jul 06 14:22:07.901976 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:22:07.901937    2559 generic.go:356] "Generic (PLEG): container finished" podID="aecb0b51-89db-4843-9c3c-960ac76a1697" containerID="1a1725296a64bb1ddb7861e702867255fb2c9e6c120bb16fe2af6259b73fd11d" exitCode=0
Jul 06 14:22:07.902501 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:22:07.902008    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-q7t4k" event={"ID":"aecb0b51-89db-4843-9c3c-960ac76a1697","Type":"ContainerDied","Data":"1a1725296a64bb1ddb7861e702867255fb2c9e6c120bb16fe2af6259b73fd11d"}
Jul 06 14:22:07.902501 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:22:07.902328    2559 scope.go:117] "RemoveContainer" containerID="1a1725296a64bb1ddb7861e702867255fb2c9e6c120bb16fe2af6259b73fd11d"
Jul 06 14:22:08.906934 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:22:08.906901    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-q7t4k" event={"ID":"aecb0b51-89db-4843-9c3c-960ac76a1697","Type":"ContainerStarted","Data":"855ce279d03612df7caa4919fc78fc45b3b364d3d0ef4dc50a5635a6a35637a8"}
Jul 06 14:22:17.938399 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:22:17.938364    2559 generic.go:356] "Generic (PLEG): container finished" podID="de6193cf-468b-4733-934b-71d909d33445" containerID="56590fead34e02130d3a89cf05524b3035be7f2f89e5e72607b2beec3eac7cdd" exitCode=0
Jul 06 14:22:17.938843 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:22:17.938440    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-2s94h" event={"ID":"de6193cf-468b-4733-934b-71d909d33445","Type":"ContainerDied","Data":"56590fead34e02130d3a89cf05524b3035be7f2f89e5e72607b2beec3eac7cdd"}
Jul 06 14:22:17.938895 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:22:17.938865    2559 scope.go:117] "RemoveContainer" containerID="56590fead34e02130d3a89cf05524b3035be7f2f89e5e72607b2beec3eac7cdd"
Jul 06 14:22:18.943820 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:22:18.943786    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-2s94h" event={"ID":"de6193cf-468b-4733-934b-71d909d33445","Type":"ContainerStarted","Data":"be757d63ed3307e76d8e2aa565590748463225c3e058e79081352f56fe713ef4"}
Jul 06 14:24:38.098950 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:24:38.098915    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-g5w5m_e35b0247-7c4e-4bd1-bc28-c084dc40ffa1/console-operator/1.log"
Jul 06 14:24:38.103543 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:24:38.103518    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-g5w5m_e35b0247-7c4e-4bd1-bc28-c084dc40ffa1/console-operator/1.log"
Jul 06 14:24:38.109499 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:24:38.109472    2559 kubelet.go:1648] "Image garbage collection succeeded"
Jul 06 14:24:55.647293 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:24:55.647259    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6ccf4f7c96-998d9"]
Jul 06 14:24:55.648658 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:24:55.647527    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="606a9ffc-10e9-4114-ad7b-752ab0272a0d" containerName="registry"
Jul 06 14:24:55.648658 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:24:55.647538    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="606a9ffc-10e9-4114-ad7b-752ab0272a0d" containerName="registry"
Jul 06 14:24:55.648658 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:24:55.647609    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="606a9ffc-10e9-4114-ad7b-752ab0272a0d" containerName="registry"
Jul 06 14:24:55.649299 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:24:55.649283    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6ccf4f7c96-998d9"
Jul 06 14:24:55.652011 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:24:55.651992    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 06 14:24:55.652412 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:24:55.652391    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-gdtdc\""
Jul 06 14:24:55.652494 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:24:55.652440    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 06 14:24:55.652545 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:24:55.652507    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 06 14:24:55.652606 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:24:55.652541    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 06 14:24:55.672079 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:24:55.672049    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6ccf4f7c96-998d9"]
Jul 06 14:24:55.802246 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:24:55.802196    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0143a9db-b269-4df3-9bd5-d489eb70716c-webhook-cert\") pod \"opendatahub-operator-controller-manager-6ccf4f7c96-998d9\" (UID: \"0143a9db-b269-4df3-9bd5-d489eb70716c\") " pod="opendatahub/opendatahub-operator-controller-manager-6ccf4f7c96-998d9"
Jul 06 14:24:55.802435 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:24:55.802277    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fwh26\" (UniqueName: \"kubernetes.io/projected/0143a9db-b269-4df3-9bd5-d489eb70716c-kube-api-access-fwh26\") pod \"opendatahub-operator-controller-manager-6ccf4f7c96-998d9\" (UID: \"0143a9db-b269-4df3-9bd5-d489eb70716c\") " pod="opendatahub/opendatahub-operator-controller-manager-6ccf4f7c96-998d9"
Jul 06 14:24:55.802435 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:24:55.802311    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/0143a9db-b269-4df3-9bd5-d489eb70716c-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6ccf4f7c96-998d9\" (UID: \"0143a9db-b269-4df3-9bd5-d489eb70716c\") " pod="opendatahub/opendatahub-operator-controller-manager-6ccf4f7c96-998d9"
Jul 06 14:24:55.802435 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:24:55.802366    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/0143a9db-b269-4df3-9bd5-d489eb70716c-manifests\") pod \"opendatahub-operator-controller-manager-6ccf4f7c96-998d9\" (UID: \"0143a9db-b269-4df3-9bd5-d489eb70716c\") " pod="opendatahub/opendatahub-operator-controller-manager-6ccf4f7c96-998d9"
Jul 06 14:24:55.802435 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:24:55.802383    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0143a9db-b269-4df3-9bd5-d489eb70716c-tmp\") pod \"opendatahub-operator-controller-manager-6ccf4f7c96-998d9\" (UID: \"0143a9db-b269-4df3-9bd5-d489eb70716c\") " pod="opendatahub/opendatahub-operator-controller-manager-6ccf4f7c96-998d9"
Jul 06 14:24:55.903350 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:24:55.903252    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/0143a9db-b269-4df3-9bd5-d489eb70716c-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6ccf4f7c96-998d9\" (UID: \"0143a9db-b269-4df3-9bd5-d489eb70716c\") " pod="opendatahub/opendatahub-operator-controller-manager-6ccf4f7c96-998d9"
Jul 06 14:24:55.903350 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:24:55.903321    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/0143a9db-b269-4df3-9bd5-d489eb70716c-manifests\") pod \"opendatahub-operator-controller-manager-6ccf4f7c96-998d9\" (UID: \"0143a9db-b269-4df3-9bd5-d489eb70716c\") " pod="opendatahub/opendatahub-operator-controller-manager-6ccf4f7c96-998d9"
Jul 06 14:24:55.903350 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:24:55.903341    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0143a9db-b269-4df3-9bd5-d489eb70716c-tmp\") pod \"opendatahub-operator-controller-manager-6ccf4f7c96-998d9\" (UID: \"0143a9db-b269-4df3-9bd5-d489eb70716c\") " pod="opendatahub/opendatahub-operator-controller-manager-6ccf4f7c96-998d9"
Jul 06 14:24:55.903567 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:24:55.903372    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0143a9db-b269-4df3-9bd5-d489eb70716c-webhook-cert\") pod \"opendatahub-operator-controller-manager-6ccf4f7c96-998d9\" (UID: \"0143a9db-b269-4df3-9bd5-d489eb70716c\") " pod="opendatahub/opendatahub-operator-controller-manager-6ccf4f7c96-998d9"
Jul 06 14:24:55.903567 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:24:55.903398    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fwh26\" (UniqueName: \"kubernetes.io/projected/0143a9db-b269-4df3-9bd5-d489eb70716c-kube-api-access-fwh26\") pod \"opendatahub-operator-controller-manager-6ccf4f7c96-998d9\" (UID: \"0143a9db-b269-4df3-9bd5-d489eb70716c\") " pod="opendatahub/opendatahub-operator-controller-manager-6ccf4f7c96-998d9"
Jul 06 14:24:55.903806 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:24:55.903771    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/0143a9db-b269-4df3-9bd5-d489eb70716c-manifests\") pod \"opendatahub-operator-controller-manager-6ccf4f7c96-998d9\" (UID: \"0143a9db-b269-4df3-9bd5-d489eb70716c\") " pod="opendatahub/opendatahub-operator-controller-manager-6ccf4f7c96-998d9"
Jul 06 14:24:55.905760 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:24:55.905732    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0143a9db-b269-4df3-9bd5-d489eb70716c-tmp\") pod \"opendatahub-operator-controller-manager-6ccf4f7c96-998d9\" (UID: \"0143a9db-b269-4df3-9bd5-d489eb70716c\") " pod="opendatahub/opendatahub-operator-controller-manager-6ccf4f7c96-998d9"
Jul 06 14:24:55.905919 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:24:55.905904    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0143a9db-b269-4df3-9bd5-d489eb70716c-webhook-cert\") pod \"opendatahub-operator-controller-manager-6ccf4f7c96-998d9\" (UID: \"0143a9db-b269-4df3-9bd5-d489eb70716c\") " pod="opendatahub/opendatahub-operator-controller-manager-6ccf4f7c96-998d9"
Jul 06 14:24:55.906061 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:24:55.906041    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/0143a9db-b269-4df3-9bd5-d489eb70716c-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6ccf4f7c96-998d9\" (UID: \"0143a9db-b269-4df3-9bd5-d489eb70716c\") " pod="opendatahub/opendatahub-operator-controller-manager-6ccf4f7c96-998d9"
Jul 06 14:24:55.915538 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:24:55.915510    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fwh26\" (UniqueName: \"kubernetes.io/projected/0143a9db-b269-4df3-9bd5-d489eb70716c-kube-api-access-fwh26\") pod \"opendatahub-operator-controller-manager-6ccf4f7c96-998d9\" (UID: \"0143a9db-b269-4df3-9bd5-d489eb70716c\") " pod="opendatahub/opendatahub-operator-controller-manager-6ccf4f7c96-998d9"
Jul 06 14:24:55.959876 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:24:55.959833    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6ccf4f7c96-998d9"
Jul 06 14:24:56.109462 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:24:56.109438    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6ccf4f7c96-998d9"]
Jul 06 14:24:56.111526 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:24:56.111482    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0143a9db_b269_4df3_9bd5_d489eb70716c.slice/crio-4436f0ce3d31771c4ff35f6282271a7de379be8a814864608c24e3d0dfb2237d WatchSource:0}: Error finding container 4436f0ce3d31771c4ff35f6282271a7de379be8a814864608c24e3d0dfb2237d: Status 404 returned error can't find the container with id 4436f0ce3d31771c4ff35f6282271a7de379be8a814864608c24e3d0dfb2237d
Jul 06 14:24:56.113185 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:24:56.113169    2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 06 14:24:56.523512 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:24:56.523473    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6ccf4f7c96-998d9" event={"ID":"0143a9db-b269-4df3-9bd5-d489eb70716c","Type":"ContainerStarted","Data":"4436f0ce3d31771c4ff35f6282271a7de379be8a814864608c24e3d0dfb2237d"}
Jul 06 14:24:59.536433 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:24:59.536394    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6ccf4f7c96-998d9" event={"ID":"0143a9db-b269-4df3-9bd5-d489eb70716c","Type":"ContainerStarted","Data":"85cef446f647f3c300d81fc315cad53dd7960af0d105c227c240d3b0dcf95db9"}
Jul 06 14:25:00.541864 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:25:00.541830    2559 generic.go:356] "Generic (PLEG): container finished" podID="0143a9db-b269-4df3-9bd5-d489eb70716c" containerID="85cef446f647f3c300d81fc315cad53dd7960af0d105c227c240d3b0dcf95db9" exitCode=0
Jul 06 14:25:00.542239 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:25:00.541912    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6ccf4f7c96-998d9" event={"ID":"0143a9db-b269-4df3-9bd5-d489eb70716c","Type":"ContainerDied","Data":"85cef446f647f3c300d81fc315cad53dd7960af0d105c227c240d3b0dcf95db9"}
Jul 06 14:25:01.547617 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:25:01.547561    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6ccf4f7c96-998d9" event={"ID":"0143a9db-b269-4df3-9bd5-d489eb70716c","Type":"ContainerStarted","Data":"6700623ff3d398ed8455969e10d593f1e71c9cd2478b4f7739e7b58d2d3a894e"}
Jul 06 14:25:01.548104 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:25:01.547685    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-6ccf4f7c96-998d9"
Jul 06 14:25:01.571342 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:25:01.571250    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-6ccf4f7c96-998d9" podStartSLOduration=2.211908028 podStartE2EDuration="6.571229428s" totalImagesPullingTime="3.296934862s" totalInitContainerRuntime="1.062386538s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:24:55 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:25:01.570076243 +0000 UTC m=+323.877368326" watchObservedRunningTime="2026-07-06 14:25:01.571229428 +0000 UTC m=+323.878521508"
Jul 06 14:25:12.553369 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:25:12.553336    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-6ccf4f7c96-998d9"
Jul 06 14:25:15.249494 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:25:15.249457    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-hdzl7"]
Jul 06 14:25:15.258084 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:25:15.258059    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-hdzl7"
Jul 06 14:25:15.260234 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:25:15.260202    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Jul 06 14:25:15.260361 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:25:15.260248    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-wp2t4\""
Jul 06 14:25:15.268017 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:25:15.267991    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-hdzl7"]
Jul 06 14:25:15.357734 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:25:15.357695    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4729c197-d9fa-4278-a2b0-79812874376a-cert\") pod \"odh-model-controller-7998c67dc4-hdzl7\" (UID: \"4729c197-d9fa-4278-a2b0-79812874376a\") " pod="opendatahub/odh-model-controller-7998c67dc4-hdzl7"
Jul 06 14:25:15.357897 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:25:15.357844    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nlgr8\" (UniqueName: \"kubernetes.io/projected/4729c197-d9fa-4278-a2b0-79812874376a-kube-api-access-nlgr8\") pod \"odh-model-controller-7998c67dc4-hdzl7\" (UID: \"4729c197-d9fa-4278-a2b0-79812874376a\") " pod="opendatahub/odh-model-controller-7998c67dc4-hdzl7"
Jul 06 14:25:15.458946 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:25:15.458916    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nlgr8\" (UniqueName: \"kubernetes.io/projected/4729c197-d9fa-4278-a2b0-79812874376a-kube-api-access-nlgr8\") pod \"odh-model-controller-7998c67dc4-hdzl7\" (UID: \"4729c197-d9fa-4278-a2b0-79812874376a\") " pod="opendatahub/odh-model-controller-7998c67dc4-hdzl7"
Jul 06 14:25:15.459129 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:25:15.458967    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4729c197-d9fa-4278-a2b0-79812874376a-cert\") pod \"odh-model-controller-7998c67dc4-hdzl7\" (UID: \"4729c197-d9fa-4278-a2b0-79812874376a\") " pod="opendatahub/odh-model-controller-7998c67dc4-hdzl7"
Jul 06 14:25:15.459129 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:25:15.459069    2559 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Jul 06 14:25:15.459129 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:25:15.459124    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4729c197-d9fa-4278-a2b0-79812874376a-cert podName:4729c197-d9fa-4278-a2b0-79812874376a nodeName:}" failed. No retries permitted until 2026-07-06 14:25:15.959107698 +0000 UTC m=+338.266399757 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/4729c197-d9fa-4278-a2b0-79812874376a-cert") pod "odh-model-controller-7998c67dc4-hdzl7" (UID: "4729c197-d9fa-4278-a2b0-79812874376a") : secret "odh-model-controller-webhook-cert" not found
Jul 06 14:25:15.469197 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:25:15.469167    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nlgr8\" (UniqueName: \"kubernetes.io/projected/4729c197-d9fa-4278-a2b0-79812874376a-kube-api-access-nlgr8\") pod \"odh-model-controller-7998c67dc4-hdzl7\" (UID: \"4729c197-d9fa-4278-a2b0-79812874376a\") " pod="opendatahub/odh-model-controller-7998c67dc4-hdzl7"
Jul 06 14:25:15.964695 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:25:15.964664    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4729c197-d9fa-4278-a2b0-79812874376a-cert\") pod \"odh-model-controller-7998c67dc4-hdzl7\" (UID: \"4729c197-d9fa-4278-a2b0-79812874376a\") " pod="opendatahub/odh-model-controller-7998c67dc4-hdzl7"
Jul 06 14:25:15.967145 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:25:15.967120    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4729c197-d9fa-4278-a2b0-79812874376a-cert\") pod \"odh-model-controller-7998c67dc4-hdzl7\" (UID: \"4729c197-d9fa-4278-a2b0-79812874376a\") " pod="opendatahub/odh-model-controller-7998c67dc4-hdzl7"
Jul 06 14:25:16.046951 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:25:16.046913    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-662tf"]
Jul 06 14:25:16.049131 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:25:16.049115    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-662tf"
Jul 06 14:25:16.051658 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:25:16.051636    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Jul 06 14:25:16.064296 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:25:16.064274    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-662tf"]
Jul 06 14:25:16.167146 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:25:16.167106    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p5bdv\" (UniqueName: \"kubernetes.io/projected/3697cccd-51bf-47c0-8db1-7237afd847a1-kube-api-access-p5bdv\") pod \"model-serving-api-54d89bd5fc-662tf\" (UID: \"3697cccd-51bf-47c0-8db1-7237afd847a1\") " pod="opendatahub/model-serving-api-54d89bd5fc-662tf"
Jul 06 14:25:16.167146 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:25:16.167146    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3697cccd-51bf-47c0-8db1-7237afd847a1-tls-certs\") pod \"model-serving-api-54d89bd5fc-662tf\" (UID: \"3697cccd-51bf-47c0-8db1-7237afd847a1\") " pod="opendatahub/model-serving-api-54d89bd5fc-662tf"
Jul 06 14:25:16.170052 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:25:16.170034    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-hdzl7"
Jul 06 14:25:16.268089 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:25:16.268044    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p5bdv\" (UniqueName: \"kubernetes.io/projected/3697cccd-51bf-47c0-8db1-7237afd847a1-kube-api-access-p5bdv\") pod \"model-serving-api-54d89bd5fc-662tf\" (UID: \"3697cccd-51bf-47c0-8db1-7237afd847a1\") " pod="opendatahub/model-serving-api-54d89bd5fc-662tf"
Jul 06 14:25:16.268518 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:25:16.268099    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3697cccd-51bf-47c0-8db1-7237afd847a1-tls-certs\") pod \"model-serving-api-54d89bd5fc-662tf\" (UID: \"3697cccd-51bf-47c0-8db1-7237afd847a1\") " pod="opendatahub/model-serving-api-54d89bd5fc-662tf"
Jul 06 14:25:16.270604 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:25:16.270560    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3697cccd-51bf-47c0-8db1-7237afd847a1-tls-certs\") pod \"model-serving-api-54d89bd5fc-662tf\" (UID: \"3697cccd-51bf-47c0-8db1-7237afd847a1\") " pod="opendatahub/model-serving-api-54d89bd5fc-662tf"
Jul 06 14:25:16.285146 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:25:16.285119    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p5bdv\" (UniqueName: \"kubernetes.io/projected/3697cccd-51bf-47c0-8db1-7237afd847a1-kube-api-access-p5bdv\") pod \"model-serving-api-54d89bd5fc-662tf\" (UID: \"3697cccd-51bf-47c0-8db1-7237afd847a1\") " pod="opendatahub/model-serving-api-54d89bd5fc-662tf"
Jul 06 14:25:16.308792 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:25:16.308763    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-hdzl7"]
Jul 06 14:25:16.311185 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:25:16.311155    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4729c197_d9fa_4278_a2b0_79812874376a.slice/crio-bf8f3319cf37a54c050f7cd193d8eec3ac61816e1e9af277f001d31cbe7d7db2 WatchSource:0}: Error finding container bf8f3319cf37a54c050f7cd193d8eec3ac61816e1e9af277f001d31cbe7d7db2: Status 404 returned error can't find the container with id bf8f3319cf37a54c050f7cd193d8eec3ac61816e1e9af277f001d31cbe7d7db2
Jul 06 14:25:16.359650 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:25:16.359612    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-662tf"
Jul 06 14:25:16.497447 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:25:16.497362    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-662tf"]
Jul 06 14:25:16.500695 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:25:16.500665    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3697cccd_51bf_47c0_8db1_7237afd847a1.slice/crio-2c2447f2ce85de0124092363e3c1aab85f342706a97df8476cd79b81916339a5 WatchSource:0}: Error finding container 2c2447f2ce85de0124092363e3c1aab85f342706a97df8476cd79b81916339a5: Status 404 returned error can't find the container with id 2c2447f2ce85de0124092363e3c1aab85f342706a97df8476cd79b81916339a5
Jul 06 14:25:16.603879 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:25:16.603842    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-662tf" event={"ID":"3697cccd-51bf-47c0-8db1-7237afd847a1","Type":"ContainerStarted","Data":"2c2447f2ce85de0124092363e3c1aab85f342706a97df8476cd79b81916339a5"}
Jul 06 14:25:16.604931 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:25:16.604892    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-hdzl7" event={"ID":"4729c197-d9fa-4278-a2b0-79812874376a","Type":"ContainerStarted","Data":"bf8f3319cf37a54c050f7cd193d8eec3ac61816e1e9af277f001d31cbe7d7db2"}
Jul 06 14:25:19.622755 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:25:19.622712    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-662tf" event={"ID":"3697cccd-51bf-47c0-8db1-7237afd847a1","Type":"ContainerStarted","Data":"367498f0897296a2392749969363ee1908fee3d2590d78b3defb334cd548e36c"}
Jul 06 14:25:19.623188 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:25:19.622828    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-662tf"
Jul 06 14:25:19.644059 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:25:19.644006    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-662tf" podStartSLOduration=0.601292355 podStartE2EDuration="3.643986866s" totalImagesPullingTime="3.042694511s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:25:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:25:19.643032733 +0000 UTC m=+341.950324814" watchObservedRunningTime="2026-07-06 14:25:19.643986866 +0000 UTC m=+341.951278950"
Jul 06 14:25:20.628281 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:25:20.628238    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-hdzl7" event={"ID":"4729c197-d9fa-4278-a2b0-79812874376a","Type":"ContainerStarted","Data":"941f162e1de181f2dc31ceaaa637651e19c9008dbd8972f8cde840d039fe7c42"}
Jul 06 14:25:20.628685 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:25:20.628333    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-hdzl7"
Jul 06 14:25:20.647867 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:25:20.647812    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-hdzl7" podStartSLOduration=2.361313977 podStartE2EDuration="5.647796813s" totalImagesPullingTime="3.286482836s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:25:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:25:20.646449404 +0000 UTC m=+342.953741498" watchObservedRunningTime="2026-07-06 14:25:20.647796813 +0000 UTC m=+342.955088941"
Jul 06 14:25:25.400834 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:25:25.400797    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-64c7864974-tk5t2"]
Jul 06 14:25:25.404079 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:25:25.404061    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-64c7864974-tk5t2"
Jul 06 14:25:25.408776 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:25:25.408753    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 06 14:25:25.408895 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:25:25.408844    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 06 14:25:25.409598 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:25:25.409566    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 06 14:25:25.409698 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:25:25.409597    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 06 14:25:25.409698 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:25:25.409588    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-s2gjs\""
Jul 06 14:25:25.420712 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:25:25.420690    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-64c7864974-tk5t2"]
Jul 06 14:25:25.437506 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:25:25.437479    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5t92v\" (UniqueName: \"kubernetes.io/projected/53e0f0e5-2817-4b79-b14e-967b32b18de6-kube-api-access-5t92v\") pod \"kube-auth-proxy-64c7864974-tk5t2\" (UID: \"53e0f0e5-2817-4b79-b14e-967b32b18de6\") " pod="openshift-ingress/kube-auth-proxy-64c7864974-tk5t2"
Jul 06 14:25:25.437653 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:25:25.437531    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/53e0f0e5-2817-4b79-b14e-967b32b18de6-tls-certs\") pod \"kube-auth-proxy-64c7864974-tk5t2\" (UID: \"53e0f0e5-2817-4b79-b14e-967b32b18de6\") " pod="openshift-ingress/kube-auth-proxy-64c7864974-tk5t2"
Jul 06 14:25:25.437653 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:25:25.437623    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/53e0f0e5-2817-4b79-b14e-967b32b18de6-tmp\") pod \"kube-auth-proxy-64c7864974-tk5t2\" (UID: \"53e0f0e5-2817-4b79-b14e-967b32b18de6\") " pod="openshift-ingress/kube-auth-proxy-64c7864974-tk5t2"
Jul 06 14:25:25.538968 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:25:25.538929    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/53e0f0e5-2817-4b79-b14e-967b32b18de6-tls-certs\") pod \"kube-auth-proxy-64c7864974-tk5t2\" (UID: \"53e0f0e5-2817-4b79-b14e-967b32b18de6\") " pod="openshift-ingress/kube-auth-proxy-64c7864974-tk5t2"
Jul 06 14:25:25.539158 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:25:25.538987    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/53e0f0e5-2817-4b79-b14e-967b32b18de6-tmp\") pod \"kube-auth-proxy-64c7864974-tk5t2\" (UID: \"53e0f0e5-2817-4b79-b14e-967b32b18de6\") " pod="openshift-ingress/kube-auth-proxy-64c7864974-tk5t2"
Jul 06 14:25:25.539158 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:25:25.539016    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5t92v\" (UniqueName: \"kubernetes.io/projected/53e0f0e5-2817-4b79-b14e-967b32b18de6-kube-api-access-5t92v\") pod \"kube-auth-proxy-64c7864974-tk5t2\" (UID: \"53e0f0e5-2817-4b79-b14e-967b32b18de6\") " pod="openshift-ingress/kube-auth-proxy-64c7864974-tk5t2"
Jul 06 14:25:25.539158 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:25:25.539073    2559 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found
Jul 06 14:25:25.539158 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:25:25.539136    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/53e0f0e5-2817-4b79-b14e-967b32b18de6-tls-certs podName:53e0f0e5-2817-4b79-b14e-967b32b18de6 nodeName:}" failed. No retries permitted until 2026-07-06 14:25:26.039120077 +0000 UTC m=+348.346412136 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/53e0f0e5-2817-4b79-b14e-967b32b18de6-tls-certs") pod "kube-auth-proxy-64c7864974-tk5t2" (UID: "53e0f0e5-2817-4b79-b14e-967b32b18de6") : secret "kube-auth-proxy-tls" not found
Jul 06 14:25:25.541243 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:25:25.541218    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/53e0f0e5-2817-4b79-b14e-967b32b18de6-tmp\") pod \"kube-auth-proxy-64c7864974-tk5t2\" (UID: \"53e0f0e5-2817-4b79-b14e-967b32b18de6\") " pod="openshift-ingress/kube-auth-proxy-64c7864974-tk5t2"
Jul 06 14:25:25.557186 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:25:25.557153    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5t92v\" (UniqueName: \"kubernetes.io/projected/53e0f0e5-2817-4b79-b14e-967b32b18de6-kube-api-access-5t92v\") pod \"kube-auth-proxy-64c7864974-tk5t2\" (UID: \"53e0f0e5-2817-4b79-b14e-967b32b18de6\") " pod="openshift-ingress/kube-auth-proxy-64c7864974-tk5t2"
Jul 06 14:25:26.042224 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:25:26.042191    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/53e0f0e5-2817-4b79-b14e-967b32b18de6-tls-certs\") pod \"kube-auth-proxy-64c7864974-tk5t2\" (UID: \"53e0f0e5-2817-4b79-b14e-967b32b18de6\") " pod="openshift-ingress/kube-auth-proxy-64c7864974-tk5t2"
Jul 06 14:25:26.044696 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:25:26.044670    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/53e0f0e5-2817-4b79-b14e-967b32b18de6-tls-certs\") pod \"kube-auth-proxy-64c7864974-tk5t2\" (UID: \"53e0f0e5-2817-4b79-b14e-967b32b18de6\") " pod="openshift-ingress/kube-auth-proxy-64c7864974-tk5t2"
Jul 06 14:25:26.313620 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:25:26.313520    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-64c7864974-tk5t2"
Jul 06 14:25:26.456137 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:25:26.456100    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-64c7864974-tk5t2"]
Jul 06 14:25:26.459302 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:25:26.459264    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod53e0f0e5_2817_4b79_b14e_967b32b18de6.slice/crio-c892cf0153e30ac56144165de3db445b95caf62532f0ba69a0c3328f3dd923fb WatchSource:0}: Error finding container c892cf0153e30ac56144165de3db445b95caf62532f0ba69a0c3328f3dd923fb: Status 404 returned error can't find the container with id c892cf0153e30ac56144165de3db445b95caf62532f0ba69a0c3328f3dd923fb
Jul 06 14:25:26.548008 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:25:26.547972    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-cdhfk"]
Jul 06 14:25:26.552339 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:25:26.552314    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-cdhfk"
Jul 06 14:25:26.556087 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:25:26.556066    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Jul 06 14:25:26.556206 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:25:26.556121    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-8zsv8\""
Jul 06 14:25:26.575221 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:25:26.575154    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-rbh57"]
Jul 06 14:25:26.578786 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:25:26.578755    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-cdhfk"]
Jul 06 14:25:26.578907 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:25:26.578864    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rbh57"
Jul 06 14:25:26.598196 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:25:26.598167    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Jul 06 14:25:26.598332 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:25:26.598239    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-zt4nw\""
Jul 06 14:25:26.630309 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:25:26.630272    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-rbh57"]
Jul 06 14:25:26.646379 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:25:26.646346    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2dtkg\" (UniqueName: \"kubernetes.io/projected/4c04aa96-f5dc-4734-9d06-e53704e040f1-kube-api-access-2dtkg\") pod \"llmisvc-controller-manager-777c6bd7d5-rbh57\" (UID: \"4c04aa96-f5dc-4734-9d06-e53704e040f1\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rbh57"
Jul 06 14:25:26.646546 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:25:26.646437    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4c04aa96-f5dc-4734-9d06-e53704e040f1-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-rbh57\" (UID: \"4c04aa96-f5dc-4734-9d06-e53704e040f1\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rbh57"
Jul 06 14:25:26.653542 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:25:26.653514    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-64c7864974-tk5t2" event={"ID":"53e0f0e5-2817-4b79-b14e-967b32b18de6","Type":"ContainerStarted","Data":"c892cf0153e30ac56144165de3db445b95caf62532f0ba69a0c3328f3dd923fb"}
Jul 06 14:25:26.747759 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:25:26.747724    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gn487\" (UniqueName: \"kubernetes.io/projected/58a6adba-bfd3-41bd-b3f1-ea4b6c13daea-kube-api-access-gn487\") pod \"kserve-controller-manager-d6f4b65b8-cdhfk\" (UID: \"58a6adba-bfd3-41bd-b3f1-ea4b6c13daea\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-cdhfk"
Jul 06 14:25:26.747759 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:25:26.747771    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4c04aa96-f5dc-4734-9d06-e53704e040f1-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-rbh57\" (UID: \"4c04aa96-f5dc-4734-9d06-e53704e040f1\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rbh57"
Jul 06 14:25:26.747979 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:25:26.747798    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/58a6adba-bfd3-41bd-b3f1-ea4b6c13daea-cert\") pod \"kserve-controller-manager-d6f4b65b8-cdhfk\" (UID: \"58a6adba-bfd3-41bd-b3f1-ea4b6c13daea\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-cdhfk"
Jul 06 14:25:26.747979 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:25:26.747824    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2dtkg\" (UniqueName: \"kubernetes.io/projected/4c04aa96-f5dc-4734-9d06-e53704e040f1-kube-api-access-2dtkg\") pod \"llmisvc-controller-manager-777c6bd7d5-rbh57\" (UID: \"4c04aa96-f5dc-4734-9d06-e53704e040f1\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rbh57"
Jul 06 14:25:26.750126 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:25:26.750106    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4c04aa96-f5dc-4734-9d06-e53704e040f1-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-rbh57\" (UID: \"4c04aa96-f5dc-4734-9d06-e53704e040f1\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rbh57"
Jul 06 14:25:26.769078 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:25:26.769050    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2dtkg\" (UniqueName: \"kubernetes.io/projected/4c04aa96-f5dc-4734-9d06-e53704e040f1-kube-api-access-2dtkg\") pod \"llmisvc-controller-manager-777c6bd7d5-rbh57\" (UID: \"4c04aa96-f5dc-4734-9d06-e53704e040f1\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rbh57"
Jul 06 14:25:26.848756 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:25:26.848670    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gn487\" (UniqueName: \"kubernetes.io/projected/58a6adba-bfd3-41bd-b3f1-ea4b6c13daea-kube-api-access-gn487\") pod \"kserve-controller-manager-d6f4b65b8-cdhfk\" (UID: \"58a6adba-bfd3-41bd-b3f1-ea4b6c13daea\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-cdhfk"
Jul 06 14:25:26.848756 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:25:26.848721    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/58a6adba-bfd3-41bd-b3f1-ea4b6c13daea-cert\") pod \"kserve-controller-manager-d6f4b65b8-cdhfk\" (UID: \"58a6adba-bfd3-41bd-b3f1-ea4b6c13daea\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-cdhfk"
Jul 06 14:25:26.851097 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:25:26.851074    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/58a6adba-bfd3-41bd-b3f1-ea4b6c13daea-cert\") pod \"kserve-controller-manager-d6f4b65b8-cdhfk\" (UID: \"58a6adba-bfd3-41bd-b3f1-ea4b6c13daea\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-cdhfk"
Jul 06 14:25:26.865027 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:25:26.864996    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gn487\" (UniqueName: \"kubernetes.io/projected/58a6adba-bfd3-41bd-b3f1-ea4b6c13daea-kube-api-access-gn487\") pod \"kserve-controller-manager-d6f4b65b8-cdhfk\" (UID: \"58a6adba-bfd3-41bd-b3f1-ea4b6c13daea\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-cdhfk"
Jul 06 14:25:26.887879 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:25:26.887840    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rbh57"
Jul 06 14:25:27.043512 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:25:27.043474    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-rbh57"]
Jul 06 14:25:27.047504 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:25:27.047472    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod4c04aa96_f5dc_4734_9d06_e53704e040f1.slice/crio-97c8d95091e8da48aa90d91990443e556c0d2f2f75052d1b9eb4f1441f8cd406 WatchSource:0}: Error finding container 97c8d95091e8da48aa90d91990443e556c0d2f2f75052d1b9eb4f1441f8cd406: Status 404 returned error can't find the container with id 97c8d95091e8da48aa90d91990443e556c0d2f2f75052d1b9eb4f1441f8cd406
Jul 06 14:25:27.162505 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:25:27.162415    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-cdhfk"
Jul 06 14:25:27.312069 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:25:27.312042    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-cdhfk"]
Jul 06 14:25:27.314777 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:25:27.314751    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58a6adba_bfd3_41bd_b3f1_ea4b6c13daea.slice/crio-96ac20b2d16ebae31a44dba85fd924216e143f72e6e8eefa143b95481f16e9cb WatchSource:0}: Error finding container 96ac20b2d16ebae31a44dba85fd924216e143f72e6e8eefa143b95481f16e9cb: Status 404 returned error can't find the container with id 96ac20b2d16ebae31a44dba85fd924216e143f72e6e8eefa143b95481f16e9cb
Jul 06 14:25:27.659052 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:25:27.659010    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-cdhfk" event={"ID":"58a6adba-bfd3-41bd-b3f1-ea4b6c13daea","Type":"ContainerStarted","Data":"96ac20b2d16ebae31a44dba85fd924216e143f72e6e8eefa143b95481f16e9cb"}
Jul 06 14:25:27.660247 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:25:27.660222    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rbh57" event={"ID":"4c04aa96-f5dc-4734-9d06-e53704e040f1","Type":"ContainerStarted","Data":"97c8d95091e8da48aa90d91990443e556c0d2f2f75052d1b9eb4f1441f8cd406"}
Jul 06 14:25:28.778784 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:25:28.778753    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-w5jmh"]
Jul 06 14:25:28.782375 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:25:28.782350    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-w5jmh"
Jul 06 14:25:28.788639 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:25:28.788617    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 06 14:25:28.789857 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:25:28.789837    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 06 14:25:28.790229 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:25:28.790050    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 06 14:25:28.790229 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:25:28.790192    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-hlg2l\""
Jul 06 14:25:28.790562 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:25:28.790542    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 06 14:25:28.799500 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:25:28.799471    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-w5jmh"]
Jul 06 14:25:28.863644 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:25:28.863340    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x6rzp\" (UniqueName: \"kubernetes.io/projected/a0a508b6-5385-43de-8245-76384185d86c-kube-api-access-x6rzp\") pod \"istiod-openshift-gateway-94bb8fbfd-w5jmh\" (UID: \"a0a508b6-5385-43de-8245-76384185d86c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-w5jmh"
Jul 06 14:25:28.863644 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:25:28.863408    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a0a508b6-5385-43de-8245-76384185d86c-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-w5jmh\" (UID: \"a0a508b6-5385-43de-8245-76384185d86c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-w5jmh"
Jul 06 14:25:28.863644 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:25:28.863432    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/a0a508b6-5385-43de-8245-76384185d86c-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-w5jmh\" (UID: \"a0a508b6-5385-43de-8245-76384185d86c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-w5jmh"
Jul 06 14:25:28.863644 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:25:28.863470    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a0a508b6-5385-43de-8245-76384185d86c-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-w5jmh\" (UID: \"a0a508b6-5385-43de-8245-76384185d86c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-w5jmh"
Jul 06 14:25:28.863644 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:25:28.863510    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/a0a508b6-5385-43de-8245-76384185d86c-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-w5jmh\" (UID: \"a0a508b6-5385-43de-8245-76384185d86c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-w5jmh"
Jul 06 14:25:28.863644 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:25:28.863551    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/a0a508b6-5385-43de-8245-76384185d86c-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-w5jmh\" (UID: \"a0a508b6-5385-43de-8245-76384185d86c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-w5jmh"
Jul 06 14:25:28.863644 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:25:28.863587    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/a0a508b6-5385-43de-8245-76384185d86c-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-w5jmh\" (UID: \"a0a508b6-5385-43de-8245-76384185d86c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-w5jmh"
Jul 06 14:25:28.964810 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:25:28.964770    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/a0a508b6-5385-43de-8245-76384185d86c-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-w5jmh\" (UID: \"a0a508b6-5385-43de-8245-76384185d86c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-w5jmh"
Jul 06 14:25:28.965001 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:25:28.964834    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/a0a508b6-5385-43de-8245-76384185d86c-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-w5jmh\" (UID: \"a0a508b6-5385-43de-8245-76384185d86c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-w5jmh"
Jul 06 14:25:28.965001 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:25:28.964870    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/a0a508b6-5385-43de-8245-76384185d86c-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-w5jmh\" (UID: \"a0a508b6-5385-43de-8245-76384185d86c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-w5jmh"
Jul 06 14:25:28.965001 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:25:28.964922    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x6rzp\" (UniqueName: \"kubernetes.io/projected/a0a508b6-5385-43de-8245-76384185d86c-kube-api-access-x6rzp\") pod \"istiod-openshift-gateway-94bb8fbfd-w5jmh\" (UID: \"a0a508b6-5385-43de-8245-76384185d86c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-w5jmh"
Jul 06 14:25:28.965001 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:25:28.964966    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a0a508b6-5385-43de-8245-76384185d86c-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-w5jmh\" (UID: \"a0a508b6-5385-43de-8245-76384185d86c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-w5jmh"
Jul 06 14:25:28.965001 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:25:28.964991    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/a0a508b6-5385-43de-8245-76384185d86c-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-w5jmh\" (UID: \"a0a508b6-5385-43de-8245-76384185d86c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-w5jmh"
Jul 06 14:25:28.965248 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:25:28.965031    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a0a508b6-5385-43de-8245-76384185d86c-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-w5jmh\" (UID: \"a0a508b6-5385-43de-8245-76384185d86c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-w5jmh"
Jul 06 14:25:28.965897 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:25:28.965793    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/a0a508b6-5385-43de-8245-76384185d86c-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-w5jmh\" (UID: \"a0a508b6-5385-43de-8245-76384185d86c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-w5jmh"
Jul 06 14:25:28.967732 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:25:28.967703    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/a0a508b6-5385-43de-8245-76384185d86c-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-w5jmh\" (UID: \"a0a508b6-5385-43de-8245-76384185d86c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-w5jmh"
Jul 06 14:25:28.968200 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:25:28.968169    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/a0a508b6-5385-43de-8245-76384185d86c-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-w5jmh\" (UID: \"a0a508b6-5385-43de-8245-76384185d86c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-w5jmh"
Jul 06 14:25:28.968459 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:25:28.968413    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a0a508b6-5385-43de-8245-76384185d86c-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-w5jmh\" (UID: \"a0a508b6-5385-43de-8245-76384185d86c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-w5jmh"
Jul 06 14:25:28.968839 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:25:28.968819    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/a0a508b6-5385-43de-8245-76384185d86c-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-w5jmh\" (UID: \"a0a508b6-5385-43de-8245-76384185d86c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-w5jmh"
Jul 06 14:25:28.979682 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:25:28.979655    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a0a508b6-5385-43de-8245-76384185d86c-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-w5jmh\" (UID: \"a0a508b6-5385-43de-8245-76384185d86c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-w5jmh"
Jul 06 14:25:28.982219 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:25:28.982187    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x6rzp\" (UniqueName: \"kubernetes.io/projected/a0a508b6-5385-43de-8245-76384185d86c-kube-api-access-x6rzp\") pod \"istiod-openshift-gateway-94bb8fbfd-w5jmh\" (UID: \"a0a508b6-5385-43de-8245-76384185d86c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-w5jmh"
Jul 06 14:25:29.094608 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:25:29.094499    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-w5jmh"
Jul 06 14:25:29.281360 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:25:29.281326    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-w5jmh"]
Jul 06 14:25:29.901327 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:25:29.901295    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0a508b6_5385_43de_8245_76384185d86c.slice/crio-6409a279021033727605fda548a617fa04ed3e0d0e84a523b112f80c178c2119 WatchSource:0}: Error finding container 6409a279021033727605fda548a617fa04ed3e0d0e84a523b112f80c178c2119: Status 404 returned error can't find the container with id 6409a279021033727605fda548a617fa04ed3e0d0e84a523b112f80c178c2119
Jul 06 14:25:30.634101 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:25:30.634061    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-662tf"
Jul 06 14:25:30.677585 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:25:30.677531    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-w5jmh" event={"ID":"a0a508b6-5385-43de-8245-76384185d86c","Type":"ContainerStarted","Data":"6409a279021033727605fda548a617fa04ed3e0d0e84a523b112f80c178c2119"}
Jul 06 14:25:31.679114 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:25:31.679083    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-hdzl7"
Jul 06 14:25:31.684760 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:25:31.684724    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-cdhfk" event={"ID":"58a6adba-bfd3-41bd-b3f1-ea4b6c13daea","Type":"ContainerStarted","Data":"244688aff06f43c44e0ef64c83536b3abe5935b370280dd5fba8a3702e2ddc57"}
Jul 06 14:25:31.684923 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:25:31.684913    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-cdhfk"
Jul 06 14:25:31.686461 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:25:31.686432    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rbh57" event={"ID":"4c04aa96-f5dc-4734-9d06-e53704e040f1","Type":"ContainerStarted","Data":"29c52814de33b89c3f098f09ba3ff897bb6a56ea36186dfdd3fb543cfa7a175c"}
Jul 06 14:25:31.686609 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:25:31.686494    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rbh57"
Jul 06 14:25:31.688083 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:25:31.688058    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-64c7864974-tk5t2" event={"ID":"53e0f0e5-2817-4b79-b14e-967b32b18de6","Type":"ContainerStarted","Data":"1c7f74a4aa6f8b71a2999ad9ba554829406e0e7a7783c188d615712a43fad931"}
Jul 06 14:25:31.730354 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:25:31.730287    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-64c7864974-tk5t2" podStartSLOduration=2.163401984 podStartE2EDuration="6.730267042s" totalImagesPullingTime="4.566865058s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:25:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:25:31.728601087 +0000 UTC m=+354.035893170" watchObservedRunningTime="2026-07-06 14:25:31.730267042 +0000 UTC m=+354.037559125"
Jul 06 14:25:31.759688 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:25:31.759621    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-cdhfk" podStartSLOduration=2.046448848 podStartE2EDuration="5.759598249s" totalImagesPullingTime="3.713149401s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:25:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:25:31.757351284 +0000 UTC m=+354.064643366" watchObservedRunningTime="2026-07-06 14:25:31.759598249 +0000 UTC m=+354.066890329"
Jul 06 14:25:31.781642 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:25:31.781581    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rbh57" podStartSLOduration=1.797206866 podStartE2EDuration="5.781548614s" totalImagesPullingTime="3.984341748s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:25:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:25:31.780803361 +0000 UTC m=+354.088095439" watchObservedRunningTime="2026-07-06 14:25:31.781548614 +0000 UTC m=+354.088840695"
Jul 06 14:25:32.915170 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:25:32.915122    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 06 14:25:32.915451 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:25:32.915199    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 06 14:25:33.703153 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:25:33.703101    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-w5jmh" event={"ID":"a0a508b6-5385-43de-8245-76384185d86c","Type":"ContainerStarted","Data":"c225a4b5c8e156f20cfe01e26257226f094af0da4fb11230d4272d053409ed74"}
Jul 06 14:25:33.703419 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:25:33.703390    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-w5jmh"
Jul 06 14:25:33.704608 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:25:33.704583    2559 patch_prober.go:28] interesting pod/istiod-openshift-gateway-94bb8fbfd-w5jmh container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 06 14:25:33.704711 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:25:33.704633    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-w5jmh" podUID="a0a508b6-5385-43de-8245-76384185d86c" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 06 14:25:33.763197 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:25:33.763140    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-w5jmh" podStartSLOduration=2.7515219269999998 podStartE2EDuration="5.763123443s" totalImagesPullingTime="3.011601516s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:25:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:25:33.760622862 +0000 UTC m=+356.067914942" watchObservedRunningTime="2026-07-06 14:25:33.763123443 +0000 UTC m=+356.070415523"
Jul 06 14:25:34.708613 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:25:34.708558    2559 patch_prober.go:28] interesting pod/istiod-openshift-gateway-94bb8fbfd-w5jmh container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 06 14:25:34.708980 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:25:34.708655    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-w5jmh" podUID="a0a508b6-5385-43de-8245-76384185d86c" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 06 14:25:37.709467 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:25:37.709434    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-w5jmh"
Jul 06 14:26:02.698237 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:26:02.698199    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rbh57"
Jul 06 14:26:02.701364 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:26:02.701331    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-cdhfk"
Jul 06 14:26:53.904880 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:26:53.904844    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-q8k2f"]
Jul 06 14:26:53.907235 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:26:53.907212    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-q8k2f"
Jul 06 14:26:53.909657 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:26:53.909638    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 06 14:26:53.909775 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:26:53.909678    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 06 14:26:53.915197 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:26:53.909842    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-psqvr\""
Jul 06 14:26:53.920555 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:26:53.920533    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-q8k2f"]
Jul 06 14:26:53.970074 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:26:53.970040    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/048ed666-061a-4f9c-b30b-2ebbdb7a864b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-q8k2f\" (UID: \"048ed666-061a-4f9c-b30b-2ebbdb7a864b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-q8k2f"
Jul 06 14:26:53.970226 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:26:53.970090    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-99zqn\" (UniqueName: \"kubernetes.io/projected/048ed666-061a-4f9c-b30b-2ebbdb7a864b-kube-api-access-99zqn\") pod \"kuadrant-operator-controller-manager-8684c9ff49-q8k2f\" (UID: \"048ed666-061a-4f9c-b30b-2ebbdb7a864b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-q8k2f"
Jul 06 14:26:54.070861 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:26:54.070822    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/048ed666-061a-4f9c-b30b-2ebbdb7a864b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-q8k2f\" (UID: \"048ed666-061a-4f9c-b30b-2ebbdb7a864b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-q8k2f"
Jul 06 14:26:54.071062 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:26:54.070870    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-99zqn\" (UniqueName: \"kubernetes.io/projected/048ed666-061a-4f9c-b30b-2ebbdb7a864b-kube-api-access-99zqn\") pod \"kuadrant-operator-controller-manager-8684c9ff49-q8k2f\" (UID: \"048ed666-061a-4f9c-b30b-2ebbdb7a864b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-q8k2f"
Jul 06 14:26:54.071220 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:26:54.071196    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/048ed666-061a-4f9c-b30b-2ebbdb7a864b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-q8k2f\" (UID: \"048ed666-061a-4f9c-b30b-2ebbdb7a864b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-q8k2f"
Jul 06 14:26:54.079775 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:26:54.079746    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-99zqn\" (UniqueName: \"kubernetes.io/projected/048ed666-061a-4f9c-b30b-2ebbdb7a864b-kube-api-access-99zqn\") pod \"kuadrant-operator-controller-manager-8684c9ff49-q8k2f\" (UID: \"048ed666-061a-4f9c-b30b-2ebbdb7a864b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-q8k2f"
Jul 06 14:26:54.222958 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:26:54.222925    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-q8k2f"
Jul 06 14:26:54.358093 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:26:54.358064    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-q8k2f"]
Jul 06 14:26:54.363343 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:26:54.363309    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod048ed666_061a_4f9c_b30b_2ebbdb7a864b.slice/crio-a351aa7cb2db967c3b60cf1913c0d52cccdcc06a0de312a251cd8e0803ded4de WatchSource:0}: Error finding container a351aa7cb2db967c3b60cf1913c0d52cccdcc06a0de312a251cd8e0803ded4de: Status 404 returned error can't find the container with id a351aa7cb2db967c3b60cf1913c0d52cccdcc06a0de312a251cd8e0803ded4de
Jul 06 14:26:55.048553 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:26:55.048517    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-q8k2f" event={"ID":"048ed666-061a-4f9c-b30b-2ebbdb7a864b","Type":"ContainerStarted","Data":"a351aa7cb2db967c3b60cf1913c0d52cccdcc06a0de312a251cd8e0803ded4de"}
Jul 06 14:27:01.075185 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:27:01.075145    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-q8k2f" event={"ID":"048ed666-061a-4f9c-b30b-2ebbdb7a864b","Type":"ContainerStarted","Data":"c96440a98c7e6bc7d3a3e45b3e407b68e1460102ceaec1cc3c63960c1b191042"}
Jul 06 14:27:01.075622 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:27:01.075265    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-q8k2f"
Jul 06 14:27:01.098686 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:27:01.098637    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-q8k2f" podStartSLOduration=1.892434049 podStartE2EDuration="8.098620079s" totalImagesPullingTime="6.20618603s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:26:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:27:01.096535527 +0000 UTC m=+443.403827607" watchObservedRunningTime="2026-07-06 14:27:01.098620079 +0000 UTC m=+443.405912244"
Jul 06 14:27:12.082030 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:27:12.082000    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-q8k2f"
Jul 06 14:27:14.043785 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:27:14.043745    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-q8k2f"]
Jul 06 14:27:14.044260 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:27:14.043986    2559 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-q8k2f" podUID="048ed666-061a-4f9c-b30b-2ebbdb7a864b" containerName="manager" containerID="cri-o://c96440a98c7e6bc7d3a3e45b3e407b68e1460102ceaec1cc3c63960c1b191042" gracePeriod=2
Jul 06 14:27:14.045631 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:27:14.045608    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-q8k2f"]
Jul 06 14:27:14.073551 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:27:14.073507    2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-q8k2f\" is forbidden: User \"system:node:ip-10-0-133-112.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-112.ec2.internal' and this object" podUID="048ed666-061a-4f9c-b30b-2ebbdb7a864b" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-q8k2f"
Jul 06 14:27:14.306158 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:27:14.306080    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cxcqd"]
Jul 06 14:27:14.306406 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:27:14.306395    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="048ed666-061a-4f9c-b30b-2ebbdb7a864b" containerName="manager"
Jul 06 14:27:14.306456 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:27:14.306407    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="048ed666-061a-4f9c-b30b-2ebbdb7a864b" containerName="manager"
Jul 06 14:27:14.306499 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:27:14.306490    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="048ed666-061a-4f9c-b30b-2ebbdb7a864b" containerName="manager"
Jul 06 14:27:14.309511 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:27:14.309488    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cxcqd"
Jul 06 14:27:14.312941 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:27:14.312911    2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-cxcqd\" is forbidden: User \"system:node:ip-10-0-133-112.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-112.ec2.internal' and this object" podUID="d9d5c33b-665b-42e6-8f3c-d962fe3e31e5" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cxcqd"
Jul 06 14:27:14.328002 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:27:14.327980    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-q8k2f"
Jul 06 14:27:14.380669 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:27:14.380638    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cxcqd"]
Jul 06 14:27:14.455372 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:27:14.455342    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/048ed666-061a-4f9c-b30b-2ebbdb7a864b-extensions-socket-volume\") pod \"048ed666-061a-4f9c-b30b-2ebbdb7a864b\" (UID: \"048ed666-061a-4f9c-b30b-2ebbdb7a864b\") "
Jul 06 14:27:14.455584 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:27:14.455414    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-99zqn\" (UniqueName: \"kubernetes.io/projected/048ed666-061a-4f9c-b30b-2ebbdb7a864b-kube-api-access-99zqn\") pod \"048ed666-061a-4f9c-b30b-2ebbdb7a864b\" (UID: \"048ed666-061a-4f9c-b30b-2ebbdb7a864b\") "
Jul 06 14:27:14.455584 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:27:14.455485    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4wnkd\" (UniqueName: \"kubernetes.io/projected/d9d5c33b-665b-42e6-8f3c-d962fe3e31e5-kube-api-access-4wnkd\") pod \"kuadrant-operator-controller-manager-8684c9ff49-cxcqd\" (UID: \"d9d5c33b-665b-42e6-8f3c-d962fe3e31e5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cxcqd"
Jul 06 14:27:14.455584 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:27:14.455533    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d9d5c33b-665b-42e6-8f3c-d962fe3e31e5-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-cxcqd\" (UID: \"d9d5c33b-665b-42e6-8f3c-d962fe3e31e5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cxcqd"
Jul 06 14:27:14.455870 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:27:14.455847    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/048ed666-061a-4f9c-b30b-2ebbdb7a864b-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "048ed666-061a-4f9c-b30b-2ebbdb7a864b" (UID: "048ed666-061a-4f9c-b30b-2ebbdb7a864b"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 14:27:14.457566 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:27:14.457544    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/048ed666-061a-4f9c-b30b-2ebbdb7a864b-kube-api-access-99zqn" (OuterVolumeSpecName: "kube-api-access-99zqn") pod "048ed666-061a-4f9c-b30b-2ebbdb7a864b" (UID: "048ed666-061a-4f9c-b30b-2ebbdb7a864b"). InnerVolumeSpecName "kube-api-access-99zqn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 14:27:14.556088 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:27:14.556052    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4wnkd\" (UniqueName: \"kubernetes.io/projected/d9d5c33b-665b-42e6-8f3c-d962fe3e31e5-kube-api-access-4wnkd\") pod \"kuadrant-operator-controller-manager-8684c9ff49-cxcqd\" (UID: \"d9d5c33b-665b-42e6-8f3c-d962fe3e31e5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cxcqd"
Jul 06 14:27:14.556310 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:27:14.556118    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d9d5c33b-665b-42e6-8f3c-d962fe3e31e5-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-cxcqd\" (UID: \"d9d5c33b-665b-42e6-8f3c-d962fe3e31e5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cxcqd"
Jul 06 14:27:14.556310 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:27:14.556217    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-99zqn\" (UniqueName: \"kubernetes.io/projected/048ed666-061a-4f9c-b30b-2ebbdb7a864b-kube-api-access-99zqn\") on node \"ip-10-0-133-112.ec2.internal\" DevicePath \"\""
Jul 06 14:27:14.556310 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:27:14.556239    2559 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/048ed666-061a-4f9c-b30b-2ebbdb7a864b-extensions-socket-volume\") on node \"ip-10-0-133-112.ec2.internal\" DevicePath \"\""
Jul 06 14:27:14.556505 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:27:14.556482    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d9d5c33b-665b-42e6-8f3c-d962fe3e31e5-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-cxcqd\" (UID: \"d9d5c33b-665b-42e6-8f3c-d962fe3e31e5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cxcqd"
Jul 06 14:27:14.574900 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:27:14.574862    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4wnkd\" (UniqueName: \"kubernetes.io/projected/d9d5c33b-665b-42e6-8f3c-d962fe3e31e5-kube-api-access-4wnkd\") pod \"kuadrant-operator-controller-manager-8684c9ff49-cxcqd\" (UID: \"d9d5c33b-665b-42e6-8f3c-d962fe3e31e5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cxcqd"
Jul 06 14:27:14.633587 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:27:14.633521    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cxcqd"
Jul 06 14:27:14.775303 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:27:14.775275    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cxcqd"]
Jul 06 14:27:14.777354 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:27:14.777329    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9d5c33b_665b_42e6_8f3c_d962fe3e31e5.slice/crio-95a228b798cb0b422b702090bebaabd1135dd719a57b9bcbe3ab3f3fb226abfb WatchSource:0}: Error finding container 95a228b798cb0b422b702090bebaabd1135dd719a57b9bcbe3ab3f3fb226abfb: Status 404 returned error can't find the container with id 95a228b798cb0b422b702090bebaabd1135dd719a57b9bcbe3ab3f3fb226abfb
Jul 06 14:27:15.134533 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:27:15.134436    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cxcqd" event={"ID":"d9d5c33b-665b-42e6-8f3c-d962fe3e31e5","Type":"ContainerStarted","Data":"d3ff9ef2d8beab14c9e4f5a7f3d45c7af6989d9e96d8df573b35c6ab4e3f7b7d"}
Jul 06 14:27:15.134533 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:27:15.134505    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cxcqd"
Jul 06 14:27:15.134533 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:27:15.134528    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cxcqd" event={"ID":"d9d5c33b-665b-42e6-8f3c-d962fe3e31e5","Type":"ContainerStarted","Data":"95a228b798cb0b422b702090bebaabd1135dd719a57b9bcbe3ab3f3fb226abfb"}
Jul 06 14:27:15.135649 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:27:15.135623    2559 generic.go:356] "Generic (PLEG): container finished" podID="048ed666-061a-4f9c-b30b-2ebbdb7a864b" containerID="c96440a98c7e6bc7d3a3e45b3e407b68e1460102ceaec1cc3c63960c1b191042" exitCode=0
Jul 06 14:27:15.135751 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:27:15.135662    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-q8k2f"
Jul 06 14:27:15.135751 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:27:15.135693    2559 scope.go:117] "RemoveContainer" containerID="c96440a98c7e6bc7d3a3e45b3e407b68e1460102ceaec1cc3c63960c1b191042"
Jul 06 14:27:15.143933 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:27:15.143914    2559 scope.go:117] "RemoveContainer" containerID="c96440a98c7e6bc7d3a3e45b3e407b68e1460102ceaec1cc3c63960c1b191042"
Jul 06 14:27:15.144186 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:27:15.144167    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c96440a98c7e6bc7d3a3e45b3e407b68e1460102ceaec1cc3c63960c1b191042\": container with ID starting with c96440a98c7e6bc7d3a3e45b3e407b68e1460102ceaec1cc3c63960c1b191042 not found: ID does not exist" containerID="c96440a98c7e6bc7d3a3e45b3e407b68e1460102ceaec1cc3c63960c1b191042"
Jul 06 14:27:15.144234 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:27:15.144195    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c96440a98c7e6bc7d3a3e45b3e407b68e1460102ceaec1cc3c63960c1b191042"} err="failed to get container status \"c96440a98c7e6bc7d3a3e45b3e407b68e1460102ceaec1cc3c63960c1b191042\": rpc error: code = NotFound desc = could not find container \"c96440a98c7e6bc7d3a3e45b3e407b68e1460102ceaec1cc3c63960c1b191042\": container with ID starting with c96440a98c7e6bc7d3a3e45b3e407b68e1460102ceaec1cc3c63960c1b191042 not found: ID does not exist"
Jul 06 14:27:15.158280 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:27:15.158225    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cxcqd" podStartSLOduration=1.158213033 podStartE2EDuration="1.158213033s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:27:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:27:15.157040606 +0000 UTC m=+457.464332687" watchObservedRunningTime="2026-07-06 14:27:15.158213033 +0000 UTC m=+457.465505113"
Jul 06 14:27:16.226660 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:27:16.226630    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="048ed666-061a-4f9c-b30b-2ebbdb7a864b" path="/var/lib/kubelet/pods/048ed666-061a-4f9c-b30b-2ebbdb7a864b/volumes"
Jul 06 14:27:26.143195 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:27:26.143166    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cxcqd"
Jul 06 14:27:31.509865 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:27:31.509831    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cxcqd"]
Jul 06 14:27:31.510310 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:27:31.510008    2559 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cxcqd" podUID="d9d5c33b-665b-42e6-8f3c-d962fe3e31e5" containerName="manager" containerID="cri-o://d3ff9ef2d8beab14c9e4f5a7f3d45c7af6989d9e96d8df573b35c6ab4e3f7b7d" gracePeriod=10
Jul 06 14:27:31.773038 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:27:31.773014    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cxcqd"
Jul 06 14:27:31.900911 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:27:31.900870    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d9d5c33b-665b-42e6-8f3c-d962fe3e31e5-extensions-socket-volume\") pod \"d9d5c33b-665b-42e6-8f3c-d962fe3e31e5\" (UID: \"d9d5c33b-665b-42e6-8f3c-d962fe3e31e5\") "
Jul 06 14:27:31.901098 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:27:31.900938    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4wnkd\" (UniqueName: \"kubernetes.io/projected/d9d5c33b-665b-42e6-8f3c-d962fe3e31e5-kube-api-access-4wnkd\") pod \"d9d5c33b-665b-42e6-8f3c-d962fe3e31e5\" (UID: \"d9d5c33b-665b-42e6-8f3c-d962fe3e31e5\") "
Jul 06 14:27:31.901273 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:27:31.901250    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d9d5c33b-665b-42e6-8f3c-d962fe3e31e5-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "d9d5c33b-665b-42e6-8f3c-d962fe3e31e5" (UID: "d9d5c33b-665b-42e6-8f3c-d962fe3e31e5"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 14:27:31.902937 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:27:31.902904    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d9d5c33b-665b-42e6-8f3c-d962fe3e31e5-kube-api-access-4wnkd" (OuterVolumeSpecName: "kube-api-access-4wnkd") pod "d9d5c33b-665b-42e6-8f3c-d962fe3e31e5" (UID: "d9d5c33b-665b-42e6-8f3c-d962fe3e31e5"). InnerVolumeSpecName "kube-api-access-4wnkd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 14:27:32.002114 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:27:32.002076    2559 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d9d5c33b-665b-42e6-8f3c-d962fe3e31e5-extensions-socket-volume\") on node \"ip-10-0-133-112.ec2.internal\" DevicePath \"\""
Jul 06 14:27:32.002114 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:27:32.002107    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4wnkd\" (UniqueName: \"kubernetes.io/projected/d9d5c33b-665b-42e6-8f3c-d962fe3e31e5-kube-api-access-4wnkd\") on node \"ip-10-0-133-112.ec2.internal\" DevicePath \"\""
Jul 06 14:27:32.216387 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:27:32.216353    2559 generic.go:356] "Generic (PLEG): container finished" podID="d9d5c33b-665b-42e6-8f3c-d962fe3e31e5" containerID="d3ff9ef2d8beab14c9e4f5a7f3d45c7af6989d9e96d8df573b35c6ab4e3f7b7d" exitCode=0
Jul 06 14:27:32.216615 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:27:32.216394    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cxcqd"
Jul 06 14:27:32.216615 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:27:32.216435    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cxcqd" event={"ID":"d9d5c33b-665b-42e6-8f3c-d962fe3e31e5","Type":"ContainerDied","Data":"d3ff9ef2d8beab14c9e4f5a7f3d45c7af6989d9e96d8df573b35c6ab4e3f7b7d"}
Jul 06 14:27:32.216615 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:27:32.216517    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cxcqd" event={"ID":"d9d5c33b-665b-42e6-8f3c-d962fe3e31e5","Type":"ContainerDied","Data":"95a228b798cb0b422b702090bebaabd1135dd719a57b9bcbe3ab3f3fb226abfb"}
Jul 06 14:27:32.216615 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:27:32.216546    2559 scope.go:117] "RemoveContainer" containerID="d3ff9ef2d8beab14c9e4f5a7f3d45c7af6989d9e96d8df573b35c6ab4e3f7b7d"
Jul 06 14:27:32.225412 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:27:32.225389    2559 scope.go:117] "RemoveContainer" containerID="d3ff9ef2d8beab14c9e4f5a7f3d45c7af6989d9e96d8df573b35c6ab4e3f7b7d"
Jul 06 14:27:32.225708 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:27:32.225685    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d3ff9ef2d8beab14c9e4f5a7f3d45c7af6989d9e96d8df573b35c6ab4e3f7b7d\": container with ID starting with d3ff9ef2d8beab14c9e4f5a7f3d45c7af6989d9e96d8df573b35c6ab4e3f7b7d not found: ID does not exist" containerID="d3ff9ef2d8beab14c9e4f5a7f3d45c7af6989d9e96d8df573b35c6ab4e3f7b7d"
Jul 06 14:27:32.225770 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:27:32.225719    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d3ff9ef2d8beab14c9e4f5a7f3d45c7af6989d9e96d8df573b35c6ab4e3f7b7d"} err="failed to get container status \"d3ff9ef2d8beab14c9e4f5a7f3d45c7af6989d9e96d8df573b35c6ab4e3f7b7d\": rpc error: code = NotFound desc = could not find container \"d3ff9ef2d8beab14c9e4f5a7f3d45c7af6989d9e96d8df573b35c6ab4e3f7b7d\": container with ID starting with d3ff9ef2d8beab14c9e4f5a7f3d45c7af6989d9e96d8df573b35c6ab4e3f7b7d not found: ID does not exist"
Jul 06 14:27:32.242963 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:27:32.242936    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cxcqd"]
Jul 06 14:27:32.269122 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:27:32.269097    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cxcqd"]
Jul 06 14:27:34.225881 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:27:34.225852    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d9d5c33b-665b-42e6-8f3c-d962fe3e31e5" path="/var/lib/kubelet/pods/d9d5c33b-665b-42e6-8f3c-d962fe3e31e5/volumes"
Jul 06 14:27:47.973283 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:27:47.973248    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vszcz"]
Jul 06 14:27:47.973700 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:27:47.973566    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d9d5c33b-665b-42e6-8f3c-d962fe3e31e5" containerName="manager"
Jul 06 14:27:47.973700 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:27:47.973590    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="d9d5c33b-665b-42e6-8f3c-d962fe3e31e5" containerName="manager"
Jul 06 14:27:47.973700 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:27:47.973651    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="d9d5c33b-665b-42e6-8f3c-d962fe3e31e5" containerName="manager"
Jul 06 14:27:47.978004 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:27:47.977988    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vszcz"
Jul 06 14:27:47.980359 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:27:47.980338    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-scd57\""
Jul 06 14:27:47.993955 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:27:47.993933    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vszcz"]
Jul 06 14:27:48.135903 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:27:48.135865    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/065f6787-1fa0-40e0-ae75-6f87a4df4d10-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-vszcz\" (UID: \"065f6787-1fa0-40e0-ae75-6f87a4df4d10\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vszcz"
Jul 06 14:27:48.135903 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:27:48.135900    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/065f6787-1fa0-40e0-ae75-6f87a4df4d10-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-vszcz\" (UID: \"065f6787-1fa0-40e0-ae75-6f87a4df4d10\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vszcz"
Jul 06 14:27:48.136121 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:27:48.135981    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/065f6787-1fa0-40e0-ae75-6f87a4df4d10-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-vszcz\" (UID: \"065f6787-1fa0-40e0-ae75-6f87a4df4d10\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vszcz"
Jul 06 14:27:48.136121 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:27:48.136011    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/065f6787-1fa0-40e0-ae75-6f87a4df4d10-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-vszcz\" (UID: \"065f6787-1fa0-40e0-ae75-6f87a4df4d10\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vszcz"
Jul 06 14:27:48.136121 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:27:48.136095    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/065f6787-1fa0-40e0-ae75-6f87a4df4d10-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-vszcz\" (UID: \"065f6787-1fa0-40e0-ae75-6f87a4df4d10\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vszcz"
Jul 06 14:27:48.136211 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:27:48.136124    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fkbkl\" (UniqueName: \"kubernetes.io/projected/065f6787-1fa0-40e0-ae75-6f87a4df4d10-kube-api-access-fkbkl\") pod \"maas-default-gateway-openshift-default-56654dcf7-vszcz\" (UID: \"065f6787-1fa0-40e0-ae75-6f87a4df4d10\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vszcz"
Jul 06 14:27:48.136211 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:27:48.136146    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/065f6787-1fa0-40e0-ae75-6f87a4df4d10-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-vszcz\" (UID: \"065f6787-1fa0-40e0-ae75-6f87a4df4d10\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vszcz"
Jul 06 14:27:48.136211 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:27:48.136168    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/065f6787-1fa0-40e0-ae75-6f87a4df4d10-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-vszcz\" (UID: \"065f6787-1fa0-40e0-ae75-6f87a4df4d10\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vszcz"
Jul 06 14:27:48.136211 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:27:48.136199    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/065f6787-1fa0-40e0-ae75-6f87a4df4d10-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-vszcz\" (UID: \"065f6787-1fa0-40e0-ae75-6f87a4df4d10\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vszcz"
Jul 06 14:27:48.237278 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:27:48.237194    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/065f6787-1fa0-40e0-ae75-6f87a4df4d10-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-vszcz\" (UID: \"065f6787-1fa0-40e0-ae75-6f87a4df4d10\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vszcz"
Jul 06 14:27:48.237278 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:27:48.237247    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/065f6787-1fa0-40e0-ae75-6f87a4df4d10-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-vszcz\" (UID: \"065f6787-1fa0-40e0-ae75-6f87a4df4d10\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vszcz"
Jul 06 14:27:48.237500 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:27:48.237283    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/065f6787-1fa0-40e0-ae75-6f87a4df4d10-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-vszcz\" (UID: \"065f6787-1fa0-40e0-ae75-6f87a4df4d10\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vszcz"
Jul 06 14:27:48.237500 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:27:48.237308    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/065f6787-1fa0-40e0-ae75-6f87a4df4d10-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-vszcz\" (UID: \"065f6787-1fa0-40e0-ae75-6f87a4df4d10\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vszcz"
Jul 06 14:27:48.237500 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:27:48.237350    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/065f6787-1fa0-40e0-ae75-6f87a4df4d10-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-vszcz\" (UID: \"065f6787-1fa0-40e0-ae75-6f87a4df4d10\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vszcz"
Jul 06 14:27:48.237500 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:27:48.237475    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/065f6787-1fa0-40e0-ae75-6f87a4df4d10-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-vszcz\" (UID: \"065f6787-1fa0-40e0-ae75-6f87a4df4d10\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vszcz"
Jul 06 14:27:48.237719 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:27:48.237559    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/065f6787-1fa0-40e0-ae75-6f87a4df4d10-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-vszcz\" (UID: \"065f6787-1fa0-40e0-ae75-6f87a4df4d10\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vszcz"
Jul 06 14:27:48.237719 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:27:48.237608    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fkbkl\" (UniqueName: \"kubernetes.io/projected/065f6787-1fa0-40e0-ae75-6f87a4df4d10-kube-api-access-fkbkl\") pod \"maas-default-gateway-openshift-default-56654dcf7-vszcz\" (UID: \"065f6787-1fa0-40e0-ae75-6f87a4df4d10\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vszcz"
Jul 06 14:27:48.237719 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:27:48.237640    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/065f6787-1fa0-40e0-ae75-6f87a4df4d10-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-vszcz\" (UID: \"065f6787-1fa0-40e0-ae75-6f87a4df4d10\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vszcz"
Jul 06 14:27:48.237719 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:27:48.237651    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/065f6787-1fa0-40e0-ae75-6f87a4df4d10-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-vszcz\" (UID: \"065f6787-1fa0-40e0-ae75-6f87a4df4d10\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vszcz"
Jul 06 14:27:48.237719 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:27:48.237653    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/065f6787-1fa0-40e0-ae75-6f87a4df4d10-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-vszcz\" (UID: \"065f6787-1fa0-40e0-ae75-6f87a4df4d10\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vszcz"
Jul 06 14:27:48.237958 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:27:48.237715    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/065f6787-1fa0-40e0-ae75-6f87a4df4d10-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-vszcz\" (UID: \"065f6787-1fa0-40e0-ae75-6f87a4df4d10\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vszcz"
Jul 06 14:27:48.237958 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:27:48.237867    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/065f6787-1fa0-40e0-ae75-6f87a4df4d10-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-vszcz\" (UID: \"065f6787-1fa0-40e0-ae75-6f87a4df4d10\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vszcz"
Jul 06 14:27:48.238030 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:27:48.238003    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/065f6787-1fa0-40e0-ae75-6f87a4df4d10-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-vszcz\" (UID: \"065f6787-1fa0-40e0-ae75-6f87a4df4d10\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vszcz"
Jul 06 14:27:48.239705 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:27:48.239684    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/065f6787-1fa0-40e0-ae75-6f87a4df4d10-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-vszcz\" (UID: \"065f6787-1fa0-40e0-ae75-6f87a4df4d10\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vszcz"
Jul 06 14:27:48.239938 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:27:48.239922    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/065f6787-1fa0-40e0-ae75-6f87a4df4d10-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-vszcz\" (UID: \"065f6787-1fa0-40e0-ae75-6f87a4df4d10\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vszcz"
Jul 06 14:27:48.250166 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:27:48.250140    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fkbkl\" (UniqueName: \"kubernetes.io/projected/065f6787-1fa0-40e0-ae75-6f87a4df4d10-kube-api-access-fkbkl\") pod \"maas-default-gateway-openshift-default-56654dcf7-vszcz\" (UID: \"065f6787-1fa0-40e0-ae75-6f87a4df4d10\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vszcz"
Jul 06 14:27:48.250269 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:27:48.250235    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/065f6787-1fa0-40e0-ae75-6f87a4df4d10-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-vszcz\" (UID: \"065f6787-1fa0-40e0-ae75-6f87a4df4d10\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vszcz"
Jul 06 14:27:48.286844 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:27:48.286815    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vszcz"
Jul 06 14:27:48.436617 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:27:48.436588    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vszcz"]
Jul 06 14:27:48.439216 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:27:48.439188    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod065f6787_1fa0_40e0_ae75_6f87a4df4d10.slice/crio-aaadf49115b4f2abd4f671eee5aae188b5d85153fba03189ad07960f623856d3 WatchSource:0}: Error finding container aaadf49115b4f2abd4f671eee5aae188b5d85153fba03189ad07960f623856d3: Status 404 returned error can't find the container with id aaadf49115b4f2abd4f671eee5aae188b5d85153fba03189ad07960f623856d3
Jul 06 14:27:49.288633 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:27:49.288595    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vszcz" event={"ID":"065f6787-1fa0-40e0-ae75-6f87a4df4d10","Type":"ContainerStarted","Data":"aaadf49115b4f2abd4f671eee5aae188b5d85153fba03189ad07960f623856d3"}
Jul 06 14:27:51.494438 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:27:51.494385    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 06 14:27:51.494814 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:27:51.494471    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 06 14:27:51.494814 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:27:51.494520    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 06 14:27:52.303566 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:27:52.303527    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vszcz" event={"ID":"065f6787-1fa0-40e0-ae75-6f87a4df4d10","Type":"ContainerStarted","Data":"2a3c184f557f77febb333066e328f8f0937a3bf7219fe2e0731e2c939ee65c22"}
Jul 06 14:27:52.328251 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:27:52.328203    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vszcz" podStartSLOduration=2.275078107 podStartE2EDuration="5.328188183s" totalImagesPullingTime="3.053110076s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:27:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:27:52.326451669 +0000 UTC m=+494.633743762" watchObservedRunningTime="2026-07-06 14:27:52.328188183 +0000 UTC m=+494.635480264"
Jul 06 14:27:53.287868 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:27:53.287829    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vszcz"
Jul 06 14:27:53.292440 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:27:53.292414    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vszcz"
Jul 06 14:27:53.308180 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:27:53.308149    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vszcz"
Jul 06 14:27:53.308949 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:27:53.308934    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vszcz"
Jul 06 14:27:53.432287 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:27:53.432253    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-bdr8s"]
Jul 06 14:27:53.435648 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:27:53.435632    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-bdr8s"
Jul 06 14:27:53.438192 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:27:53.438171    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-7h9gg\""
Jul 06 14:27:53.438316 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:27:53.438207    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 06 14:27:53.438316 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:27:53.438250    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 06 14:27:53.438755 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:27:53.438739    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 06 14:27:53.447409 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:27:53.447371    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-bdr8s"]
Jul 06 14:27:53.517858 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:27:53.517819    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-bdr8s"]
Jul 06 14:27:53.586683 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:27:53.586598    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dx2qb\" (UniqueName: \"kubernetes.io/projected/dab73874-52b3-4fd3-85fe-0f2ec588cc40-kube-api-access-dx2qb\") pod \"limitador-limitador-64c94fd64f-bdr8s\" (UID: \"dab73874-52b3-4fd3-85fe-0f2ec588cc40\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-bdr8s"
Jul 06 14:27:53.586683 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:27:53.586656    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/dab73874-52b3-4fd3-85fe-0f2ec588cc40-config-file\") pod \"limitador-limitador-64c94fd64f-bdr8s\" (UID: \"dab73874-52b3-4fd3-85fe-0f2ec588cc40\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-bdr8s"
Jul 06 14:27:53.687250 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:27:53.687217    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dx2qb\" (UniqueName: \"kubernetes.io/projected/dab73874-52b3-4fd3-85fe-0f2ec588cc40-kube-api-access-dx2qb\") pod \"limitador-limitador-64c94fd64f-bdr8s\" (UID: \"dab73874-52b3-4fd3-85fe-0f2ec588cc40\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-bdr8s"
Jul 06 14:27:53.687441 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:27:53.687276    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/dab73874-52b3-4fd3-85fe-0f2ec588cc40-config-file\") pod \"limitador-limitador-64c94fd64f-bdr8s\" (UID: \"dab73874-52b3-4fd3-85fe-0f2ec588cc40\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-bdr8s"
Jul 06 14:27:53.687885 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:27:53.687867    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/dab73874-52b3-4fd3-85fe-0f2ec588cc40-config-file\") pod \"limitador-limitador-64c94fd64f-bdr8s\" (UID: \"dab73874-52b3-4fd3-85fe-0f2ec588cc40\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-bdr8s"
Jul 06 14:27:53.695269 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:27:53.695244    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dx2qb\" (UniqueName: \"kubernetes.io/projected/dab73874-52b3-4fd3-85fe-0f2ec588cc40-kube-api-access-dx2qb\") pod \"limitador-limitador-64c94fd64f-bdr8s\" (UID: \"dab73874-52b3-4fd3-85fe-0f2ec588cc40\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-bdr8s"
Jul 06 14:27:53.751206 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:27:53.751176    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-bdr8s"
Jul 06 14:27:53.882705 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:27:53.882679    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-bdr8s"]
Jul 06 14:27:53.885207 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:27:53.885170    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddab73874_52b3_4fd3_85fe_0f2ec588cc40.slice/crio-be55759f6b646db59240825110327d8f5dbadca7c8797b7e4ccc623658714f0d WatchSource:0}: Error finding container be55759f6b646db59240825110327d8f5dbadca7c8797b7e4ccc623658714f0d: Status 404 returned error can't find the container with id be55759f6b646db59240825110327d8f5dbadca7c8797b7e4ccc623658714f0d
Jul 06 14:27:54.313078 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:27:54.313042    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-bdr8s" event={"ID":"dab73874-52b3-4fd3-85fe-0f2ec588cc40","Type":"ContainerStarted","Data":"be55759f6b646db59240825110327d8f5dbadca7c8797b7e4ccc623658714f0d"}
Jul 06 14:27:57.329120 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:27:57.329084    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-bdr8s" event={"ID":"dab73874-52b3-4fd3-85fe-0f2ec588cc40","Type":"ContainerStarted","Data":"a4a7498ef02d70f0d2b81405e6445a8de70da081c6f9e4f035f42f90183c1491"}
Jul 06 14:27:57.329536 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:27:57.329185    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-bdr8s"
Jul 06 14:27:57.364249 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:27:57.364198    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-bdr8s" podStartSLOduration=1.475246295 podStartE2EDuration="4.36417838s" totalImagesPullingTime="2.888932085s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:27:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:27:57.363031208 +0000 UTC m=+499.670323289" watchObservedRunningTime="2026-07-06 14:27:57.36417838 +0000 UTC m=+499.671470461"
Jul 06 14:28:08.334272 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:28:08.334242    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-bdr8s"
Jul 06 14:28:10.710243 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:28:10.710210    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-bdr8s"]
Jul 06 14:28:10.710653 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:28:10.710391    2559 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-bdr8s" podUID="dab73874-52b3-4fd3-85fe-0f2ec588cc40" containerName="limitador" containerID="cri-o://a4a7498ef02d70f0d2b81405e6445a8de70da081c6f9e4f035f42f90183c1491" gracePeriod=30
Jul 06 14:28:11.253973 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:28:11.253948    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-bdr8s"
Jul 06 14:28:11.328320 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:28:11.328232    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dx2qb\" (UniqueName: \"kubernetes.io/projected/dab73874-52b3-4fd3-85fe-0f2ec588cc40-kube-api-access-dx2qb\") pod \"dab73874-52b3-4fd3-85fe-0f2ec588cc40\" (UID: \"dab73874-52b3-4fd3-85fe-0f2ec588cc40\") "
Jul 06 14:28:11.328320 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:28:11.328268    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/dab73874-52b3-4fd3-85fe-0f2ec588cc40-config-file\") pod \"dab73874-52b3-4fd3-85fe-0f2ec588cc40\" (UID: \"dab73874-52b3-4fd3-85fe-0f2ec588cc40\") "
Jul 06 14:28:11.328652 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:28:11.328623    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dab73874-52b3-4fd3-85fe-0f2ec588cc40-config-file" (OuterVolumeSpecName: "config-file") pod "dab73874-52b3-4fd3-85fe-0f2ec588cc40" (UID: "dab73874-52b3-4fd3-85fe-0f2ec588cc40"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 14:28:11.330341 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:28:11.330319    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dab73874-52b3-4fd3-85fe-0f2ec588cc40-kube-api-access-dx2qb" (OuterVolumeSpecName: "kube-api-access-dx2qb") pod "dab73874-52b3-4fd3-85fe-0f2ec588cc40" (UID: "dab73874-52b3-4fd3-85fe-0f2ec588cc40"). InnerVolumeSpecName "kube-api-access-dx2qb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 14:28:11.387507 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:28:11.387430    2559 generic.go:356] "Generic (PLEG): container finished" podID="dab73874-52b3-4fd3-85fe-0f2ec588cc40" containerID="a4a7498ef02d70f0d2b81405e6445a8de70da081c6f9e4f035f42f90183c1491" exitCode=0
Jul 06 14:28:11.387507 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:28:11.387464    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-bdr8s" event={"ID":"dab73874-52b3-4fd3-85fe-0f2ec588cc40","Type":"ContainerDied","Data":"a4a7498ef02d70f0d2b81405e6445a8de70da081c6f9e4f035f42f90183c1491"}
Jul 06 14:28:11.387745 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:28:11.387536    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-bdr8s" event={"ID":"dab73874-52b3-4fd3-85fe-0f2ec588cc40","Type":"ContainerDied","Data":"be55759f6b646db59240825110327d8f5dbadca7c8797b7e4ccc623658714f0d"}
Jul 06 14:28:11.387745 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:28:11.387596    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-bdr8s"
Jul 06 14:28:11.387745 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:28:11.387608    2559 scope.go:117] "RemoveContainer" containerID="a4a7498ef02d70f0d2b81405e6445a8de70da081c6f9e4f035f42f90183c1491"
Jul 06 14:28:11.397217 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:28:11.397200    2559 scope.go:117] "RemoveContainer" containerID="a4a7498ef02d70f0d2b81405e6445a8de70da081c6f9e4f035f42f90183c1491"
Jul 06 14:28:11.397487 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:28:11.397466    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a4a7498ef02d70f0d2b81405e6445a8de70da081c6f9e4f035f42f90183c1491\": container with ID starting with a4a7498ef02d70f0d2b81405e6445a8de70da081c6f9e4f035f42f90183c1491 not found: ID does not exist" containerID="a4a7498ef02d70f0d2b81405e6445a8de70da081c6f9e4f035f42f90183c1491"
Jul 06 14:28:11.397531 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:28:11.397496    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a4a7498ef02d70f0d2b81405e6445a8de70da081c6f9e4f035f42f90183c1491"} err="failed to get container status \"a4a7498ef02d70f0d2b81405e6445a8de70da081c6f9e4f035f42f90183c1491\": rpc error: code = NotFound desc = could not find container \"a4a7498ef02d70f0d2b81405e6445a8de70da081c6f9e4f035f42f90183c1491\": container with ID starting with a4a7498ef02d70f0d2b81405e6445a8de70da081c6f9e4f035f42f90183c1491 not found: ID does not exist"
Jul 06 14:28:11.411719 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:28:11.411696    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-bdr8s"]
Jul 06 14:28:11.420268 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:28:11.420243    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-bdr8s"]
Jul 06 14:28:11.429034 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:28:11.429017    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dx2qb\" (UniqueName: \"kubernetes.io/projected/dab73874-52b3-4fd3-85fe-0f2ec588cc40-kube-api-access-dx2qb\") on node \"ip-10-0-133-112.ec2.internal\" DevicePath \"\""
Jul 06 14:28:11.429098 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:28:11.429037    2559 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/dab73874-52b3-4fd3-85fe-0f2ec588cc40-config-file\") on node \"ip-10-0-133-112.ec2.internal\" DevicePath \"\""
Jul 06 14:28:12.226394 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:28:12.226361    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dab73874-52b3-4fd3-85fe-0f2ec588cc40" path="/var/lib/kubelet/pods/dab73874-52b3-4fd3-85fe-0f2ec588cc40/volumes"
Jul 06 14:28:15.194814 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:28:15.194781    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/postgres-69fb4594cf-6rvg5"]
Jul 06 14:28:15.195247 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:28:15.195101    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dab73874-52b3-4fd3-85fe-0f2ec588cc40" containerName="limitador"
Jul 06 14:28:15.195247 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:28:15.195112    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="dab73874-52b3-4fd3-85fe-0f2ec588cc40" containerName="limitador"
Jul 06 14:28:15.195247 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:28:15.195174    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="dab73874-52b3-4fd3-85fe-0f2ec588cc40" containerName="limitador"
Jul 06 14:28:15.199455 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:28:15.199432    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-69fb4594cf-6rvg5"
Jul 06 14:28:15.202012 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:28:15.201988    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"postgres-creds\""
Jul 06 14:28:15.202123 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:28:15.202053    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-w7swt\""
Jul 06 14:28:15.208499 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:28:15.208476    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-69fb4594cf-6rvg5"]
Jul 06 14:28:15.256440 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:28:15.256406    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/6f446a12-03fd-44e7-8fe4-2a6346653d8c-data\") pod \"postgres-69fb4594cf-6rvg5\" (UID: \"6f446a12-03fd-44e7-8fe4-2a6346653d8c\") " pod="opendatahub/postgres-69fb4594cf-6rvg5"
Jul 06 14:28:15.256691 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:28:15.256469    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5fst9\" (UniqueName: \"kubernetes.io/projected/6f446a12-03fd-44e7-8fe4-2a6346653d8c-kube-api-access-5fst9\") pod \"postgres-69fb4594cf-6rvg5\" (UID: \"6f446a12-03fd-44e7-8fe4-2a6346653d8c\") " pod="opendatahub/postgres-69fb4594cf-6rvg5"
Jul 06 14:28:15.357787 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:28:15.357751    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5fst9\" (UniqueName: \"kubernetes.io/projected/6f446a12-03fd-44e7-8fe4-2a6346653d8c-kube-api-access-5fst9\") pod \"postgres-69fb4594cf-6rvg5\" (UID: \"6f446a12-03fd-44e7-8fe4-2a6346653d8c\") " pod="opendatahub/postgres-69fb4594cf-6rvg5"
Jul 06 14:28:15.357957 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:28:15.357814    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/6f446a12-03fd-44e7-8fe4-2a6346653d8c-data\") pod \"postgres-69fb4594cf-6rvg5\" (UID: \"6f446a12-03fd-44e7-8fe4-2a6346653d8c\") " pod="opendatahub/postgres-69fb4594cf-6rvg5"
Jul 06 14:28:15.358217 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:28:15.358198    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/6f446a12-03fd-44e7-8fe4-2a6346653d8c-data\") pod \"postgres-69fb4594cf-6rvg5\" (UID: \"6f446a12-03fd-44e7-8fe4-2a6346653d8c\") " pod="opendatahub/postgres-69fb4594cf-6rvg5"
Jul 06 14:28:15.366975 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:28:15.366951    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5fst9\" (UniqueName: \"kubernetes.io/projected/6f446a12-03fd-44e7-8fe4-2a6346653d8c-kube-api-access-5fst9\") pod \"postgres-69fb4594cf-6rvg5\" (UID: \"6f446a12-03fd-44e7-8fe4-2a6346653d8c\") " pod="opendatahub/postgres-69fb4594cf-6rvg5"
Jul 06 14:28:15.509919 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:28:15.509836    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-69fb4594cf-6rvg5"
Jul 06 14:28:15.657792 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:28:15.657767    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-69fb4594cf-6rvg5"]
Jul 06 14:28:15.659608 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:28:15.659580    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f446a12_03fd_44e7_8fe4_2a6346653d8c.slice/crio-0f8d445924e53d679c523e55566e5830c53215f832129f3ac067be4c4a37f3c4 WatchSource:0}: Error finding container 0f8d445924e53d679c523e55566e5830c53215f832129f3ac067be4c4a37f3c4: Status 404 returned error can't find the container with id 0f8d445924e53d679c523e55566e5830c53215f832129f3ac067be4c4a37f3c4
Jul 06 14:28:16.410827 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:28:16.410790    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-69fb4594cf-6rvg5" event={"ID":"6f446a12-03fd-44e7-8fe4-2a6346653d8c","Type":"ContainerStarted","Data":"0f8d445924e53d679c523e55566e5830c53215f832129f3ac067be4c4a37f3c4"}
Jul 06 14:28:21.436879 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:28:21.436841    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-69fb4594cf-6rvg5" event={"ID":"6f446a12-03fd-44e7-8fe4-2a6346653d8c","Type":"ContainerStarted","Data":"a1d3cbab7f89e74ec9f04812025a5af0207a76a1447522e158050a5b2828f671"}
Jul 06 14:28:21.437328 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:28:21.436978    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/postgres-69fb4594cf-6rvg5"
Jul 06 14:28:21.460086 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:28:21.460035    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/postgres-69fb4594cf-6rvg5" podStartSLOduration=1.580761953 podStartE2EDuration="6.460022101s" totalImagesPullingTime="4.879260148s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:28:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:28:21.458402786 +0000 UTC m=+523.765694868" watchObservedRunningTime="2026-07-06 14:28:21.460022101 +0000 UTC m=+523.767314184"
Jul 06 14:28:27.470421 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:28:27.470391    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/postgres-69fb4594cf-6rvg5"
Jul 06 14:28:34.595914 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:28:34.595877    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6bcfc54b8f-9cwhf"]
Jul 06 14:28:34.599609 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:28:34.599563    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6bcfc54b8f-9cwhf"
Jul 06 14:28:34.601916 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:28:34.601894    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 06 14:28:34.602044 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:28:34.601901    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 06 14:28:34.602320 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:28:34.602301    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-hz2hk\""
Jul 06 14:28:34.611117 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:28:34.611083    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6bcfc54b8f-9cwhf"]
Jul 06 14:28:34.722205 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:28:34.722170    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/94ca3335-0521-48de-af0d-33b3dd416153-webhook-cert\") pod \"maas-controller-6bcfc54b8f-9cwhf\" (UID: \"94ca3335-0521-48de-af0d-33b3dd416153\") " pod="opendatahub/maas-controller-6bcfc54b8f-9cwhf"
Jul 06 14:28:34.722387 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:28:34.722270    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6k4x2\" (UniqueName: \"kubernetes.io/projected/94ca3335-0521-48de-af0d-33b3dd416153-kube-api-access-6k4x2\") pod \"maas-controller-6bcfc54b8f-9cwhf\" (UID: \"94ca3335-0521-48de-af0d-33b3dd416153\") " pod="opendatahub/maas-controller-6bcfc54b8f-9cwhf"
Jul 06 14:28:34.823480 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:28:34.823443    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/94ca3335-0521-48de-af0d-33b3dd416153-webhook-cert\") pod \"maas-controller-6bcfc54b8f-9cwhf\" (UID: \"94ca3335-0521-48de-af0d-33b3dd416153\") " pod="opendatahub/maas-controller-6bcfc54b8f-9cwhf"
Jul 06 14:28:34.823688 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:28:34.823522    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6k4x2\" (UniqueName: \"kubernetes.io/projected/94ca3335-0521-48de-af0d-33b3dd416153-kube-api-access-6k4x2\") pod \"maas-controller-6bcfc54b8f-9cwhf\" (UID: \"94ca3335-0521-48de-af0d-33b3dd416153\") " pod="opendatahub/maas-controller-6bcfc54b8f-9cwhf"
Jul 06 14:28:34.826090 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:28:34.826065    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/94ca3335-0521-48de-af0d-33b3dd416153-webhook-cert\") pod \"maas-controller-6bcfc54b8f-9cwhf\" (UID: \"94ca3335-0521-48de-af0d-33b3dd416153\") " pod="opendatahub/maas-controller-6bcfc54b8f-9cwhf"
Jul 06 14:28:34.828668 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:28:34.828649    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-58b9bc5c65-9cs44"]
Jul 06 14:28:34.831856 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:28:34.831841    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-58b9bc5c65-9cs44"
Jul 06 14:28:34.838432 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:28:34.838412    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6k4x2\" (UniqueName: \"kubernetes.io/projected/94ca3335-0521-48de-af0d-33b3dd416153-kube-api-access-6k4x2\") pod \"maas-controller-6bcfc54b8f-9cwhf\" (UID: \"94ca3335-0521-48de-af0d-33b3dd416153\") " pod="opendatahub/maas-controller-6bcfc54b8f-9cwhf"
Jul 06 14:28:34.851413 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:28:34.851390    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-58b9bc5c65-9cs44"]
Jul 06 14:28:34.910423 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:28:34.910388    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6bcfc54b8f-9cwhf"
Jul 06 14:28:34.924742 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:28:34.924710    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gvscw\" (UniqueName: \"kubernetes.io/projected/42bbc67a-8c09-40ba-8c41-70807782ab78-kube-api-access-gvscw\") pod \"maas-controller-58b9bc5c65-9cs44\" (UID: \"42bbc67a-8c09-40ba-8c41-70807782ab78\") " pod="opendatahub/maas-controller-58b9bc5c65-9cs44"
Jul 06 14:28:34.924859 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:28:34.924808    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/42bbc67a-8c09-40ba-8c41-70807782ab78-webhook-cert\") pod \"maas-controller-58b9bc5c65-9cs44\" (UID: \"42bbc67a-8c09-40ba-8c41-70807782ab78\") " pod="opendatahub/maas-controller-58b9bc5c65-9cs44"
Jul 06 14:28:35.026138 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:28:35.026106    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gvscw\" (UniqueName: \"kubernetes.io/projected/42bbc67a-8c09-40ba-8c41-70807782ab78-kube-api-access-gvscw\") pod \"maas-controller-58b9bc5c65-9cs44\" (UID: \"42bbc67a-8c09-40ba-8c41-70807782ab78\") " pod="opendatahub/maas-controller-58b9bc5c65-9cs44"
Jul 06 14:28:35.026323 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:28:35.026170    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/42bbc67a-8c09-40ba-8c41-70807782ab78-webhook-cert\") pod \"maas-controller-58b9bc5c65-9cs44\" (UID: \"42bbc67a-8c09-40ba-8c41-70807782ab78\") " pod="opendatahub/maas-controller-58b9bc5c65-9cs44"
Jul 06 14:28:35.028731 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:28:35.028701    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/42bbc67a-8c09-40ba-8c41-70807782ab78-webhook-cert\") pod \"maas-controller-58b9bc5c65-9cs44\" (UID: \"42bbc67a-8c09-40ba-8c41-70807782ab78\") " pod="opendatahub/maas-controller-58b9bc5c65-9cs44"
Jul 06 14:28:35.054377 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:28:35.054346    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gvscw\" (UniqueName: \"kubernetes.io/projected/42bbc67a-8c09-40ba-8c41-70807782ab78-kube-api-access-gvscw\") pod \"maas-controller-58b9bc5c65-9cs44\" (UID: \"42bbc67a-8c09-40ba-8c41-70807782ab78\") " pod="opendatahub/maas-controller-58b9bc5c65-9cs44"
Jul 06 14:28:35.075250 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:28:35.075225    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6bcfc54b8f-9cwhf"]
Jul 06 14:28:35.077733 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:28:35.077704    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod94ca3335_0521_48de_af0d_33b3dd416153.slice/crio-e3848a22117d9815e371d1c5b78a741d70d091bb87221357aa76deff39983a36 WatchSource:0}: Error finding container e3848a22117d9815e371d1c5b78a741d70d091bb87221357aa76deff39983a36: Status 404 returned error can't find the container with id e3848a22117d9815e371d1c5b78a741d70d091bb87221357aa76deff39983a36
Jul 06 14:28:35.150522 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:28:35.150432    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-58b9bc5c65-9cs44"
Jul 06 14:28:35.284635 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:28:35.284609    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-58b9bc5c65-9cs44"]
Jul 06 14:28:35.286145 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:28:35.286122    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42bbc67a_8c09_40ba_8c41_70807782ab78.slice/crio-c36c1eb45d58671d6bd9bf8da4633fa9a275dcaaf9bfa5b2283f2721b38431b5 WatchSource:0}: Error finding container c36c1eb45d58671d6bd9bf8da4633fa9a275dcaaf9bfa5b2283f2721b38431b5: Status 404 returned error can't find the container with id c36c1eb45d58671d6bd9bf8da4633fa9a275dcaaf9bfa5b2283f2721b38431b5
Jul 06 14:28:35.499520 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:28:35.499480    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-58b9bc5c65-9cs44" event={"ID":"42bbc67a-8c09-40ba-8c41-70807782ab78","Type":"ContainerStarted","Data":"c36c1eb45d58671d6bd9bf8da4633fa9a275dcaaf9bfa5b2283f2721b38431b5"}
Jul 06 14:28:35.500601 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:28:35.500557    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6bcfc54b8f-9cwhf" event={"ID":"94ca3335-0521-48de-af0d-33b3dd416153","Type":"ContainerStarted","Data":"e3848a22117d9815e371d1c5b78a741d70d091bb87221357aa76deff39983a36"}
Jul 06 14:28:38.517374 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:28:38.517339    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-58b9bc5c65-9cs44" event={"ID":"42bbc67a-8c09-40ba-8c41-70807782ab78","Type":"ContainerStarted","Data":"6ffd8c95dc9c6c152550f98ef7c051f0757f448d9e50dbf9e43d15419463c105"}
Jul 06 14:28:38.517911 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:28:38.517433    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-58b9bc5c65-9cs44"
Jul 06 14:28:38.518924 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:28:38.518892    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6bcfc54b8f-9cwhf" event={"ID":"94ca3335-0521-48de-af0d-33b3dd416153","Type":"ContainerStarted","Data":"6d7195d5e7f94833a049471919afcdaeaf40f098aba08304c8c522ab76f4237b"}
Jul 06 14:28:38.519057 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:28:38.518952    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6bcfc54b8f-9cwhf"
Jul 06 14:28:38.548017 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:28:38.547970    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-58b9bc5c65-9cs44" podStartSLOduration=2.000865399 podStartE2EDuration="4.547956444s" totalImagesPullingTime="2.547091045s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:28:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:28:38.545805848 +0000 UTC m=+540.853097929" watchObservedRunningTime="2026-07-06 14:28:38.547956444 +0000 UTC m=+540.855248504"
Jul 06 14:28:38.593558 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:28:38.593503    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6bcfc54b8f-9cwhf" podStartSLOduration=1.84861192 podStartE2EDuration="4.593487014s" totalImagesPullingTime="2.744875094s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:28:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:28:38.592393593 +0000 UTC m=+540.899685676" watchObservedRunningTime="2026-07-06 14:28:38.593487014 +0000 UTC m=+540.900779095"
Jul 06 14:28:49.528983 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:28:49.528952    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6bcfc54b8f-9cwhf"
Jul 06 14:28:49.529610 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:28:49.529137    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-58b9bc5c65-9cs44"
Jul 06 14:28:49.613085 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:28:49.613049    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6bcfc54b8f-9cwhf"]
Jul 06 14:28:49.613431 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:28:49.613325    2559 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-6bcfc54b8f-9cwhf" podUID="94ca3335-0521-48de-af0d-33b3dd416153" containerName="manager" containerID="cri-o://6d7195d5e7f94833a049471919afcdaeaf40f098aba08304c8c522ab76f4237b" gracePeriod=30
Jul 06 14:28:49.862409 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:28:49.862379    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6bcfc54b8f-9cwhf"
Jul 06 14:28:49.958308 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:28:49.958275    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/94ca3335-0521-48de-af0d-33b3dd416153-webhook-cert\") pod \"94ca3335-0521-48de-af0d-33b3dd416153\" (UID: \"94ca3335-0521-48de-af0d-33b3dd416153\") "
Jul 06 14:28:49.958308 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:28:49.958324    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6k4x2\" (UniqueName: \"kubernetes.io/projected/94ca3335-0521-48de-af0d-33b3dd416153-kube-api-access-6k4x2\") pod \"94ca3335-0521-48de-af0d-33b3dd416153\" (UID: \"94ca3335-0521-48de-af0d-33b3dd416153\") "
Jul 06 14:28:49.960646 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:28:49.960614    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/94ca3335-0521-48de-af0d-33b3dd416153-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "94ca3335-0521-48de-af0d-33b3dd416153" (UID: "94ca3335-0521-48de-af0d-33b3dd416153"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 14:28:49.960769 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:28:49.960645    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/94ca3335-0521-48de-af0d-33b3dd416153-kube-api-access-6k4x2" (OuterVolumeSpecName: "kube-api-access-6k4x2") pod "94ca3335-0521-48de-af0d-33b3dd416153" (UID: "94ca3335-0521-48de-af0d-33b3dd416153"). InnerVolumeSpecName "kube-api-access-6k4x2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 14:28:50.059937 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:28:50.059861    2559 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/94ca3335-0521-48de-af0d-33b3dd416153-webhook-cert\") on node \"ip-10-0-133-112.ec2.internal\" DevicePath \"\""
Jul 06 14:28:50.059937 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:28:50.059890    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6k4x2\" (UniqueName: \"kubernetes.io/projected/94ca3335-0521-48de-af0d-33b3dd416153-kube-api-access-6k4x2\") on node \"ip-10-0-133-112.ec2.internal\" DevicePath \"\""
Jul 06 14:28:50.573450 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:28:50.573417    2559 generic.go:356] "Generic (PLEG): container finished" podID="94ca3335-0521-48de-af0d-33b3dd416153" containerID="6d7195d5e7f94833a049471919afcdaeaf40f098aba08304c8c522ab76f4237b" exitCode=0
Jul 06 14:28:50.573910 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:28:50.573472    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6bcfc54b8f-9cwhf"
Jul 06 14:28:50.573910 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:28:50.573507    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6bcfc54b8f-9cwhf" event={"ID":"94ca3335-0521-48de-af0d-33b3dd416153","Type":"ContainerDied","Data":"6d7195d5e7f94833a049471919afcdaeaf40f098aba08304c8c522ab76f4237b"}
Jul 06 14:28:50.573910 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:28:50.573558    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6bcfc54b8f-9cwhf" event={"ID":"94ca3335-0521-48de-af0d-33b3dd416153","Type":"ContainerDied","Data":"e3848a22117d9815e371d1c5b78a741d70d091bb87221357aa76deff39983a36"}
Jul 06 14:28:50.573910 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:28:50.573601    2559 scope.go:117] "RemoveContainer" containerID="6d7195d5e7f94833a049471919afcdaeaf40f098aba08304c8c522ab76f4237b"
Jul 06 14:28:50.581720 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:28:50.581700    2559 scope.go:117] "RemoveContainer" containerID="6d7195d5e7f94833a049471919afcdaeaf40f098aba08304c8c522ab76f4237b"
Jul 06 14:28:50.581985 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:28:50.581967    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6d7195d5e7f94833a049471919afcdaeaf40f098aba08304c8c522ab76f4237b\": container with ID starting with 6d7195d5e7f94833a049471919afcdaeaf40f098aba08304c8c522ab76f4237b not found: ID does not exist" containerID="6d7195d5e7f94833a049471919afcdaeaf40f098aba08304c8c522ab76f4237b"
Jul 06 14:28:50.582050 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:28:50.581991    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6d7195d5e7f94833a049471919afcdaeaf40f098aba08304c8c522ab76f4237b"} err="failed to get container status \"6d7195d5e7f94833a049471919afcdaeaf40f098aba08304c8c522ab76f4237b\": rpc error: code = NotFound desc = could not find container \"6d7195d5e7f94833a049471919afcdaeaf40f098aba08304c8c522ab76f4237b\": container with ID starting with 6d7195d5e7f94833a049471919afcdaeaf40f098aba08304c8c522ab76f4237b not found: ID does not exist"
Jul 06 14:28:50.591199 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:28:50.591171    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6bcfc54b8f-9cwhf"]
Jul 06 14:28:50.595175 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:28:50.595151    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-6bcfc54b8f-9cwhf"]
Jul 06 14:28:52.226651 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:28:52.226618    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="94ca3335-0521-48de-af0d-33b3dd416153" path="/var/lib/kubelet/pods/94ca3335-0521-48de-af0d-33b3dd416153/volumes"
Jul 06 14:29:11.139078 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:29:11.139039    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-c7fb74df8-wgl2x"]
Jul 06 14:29:11.139719 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:29:11.139533    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="94ca3335-0521-48de-af0d-33b3dd416153" containerName="manager"
Jul 06 14:29:11.139719 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:29:11.139552    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="94ca3335-0521-48de-af0d-33b3dd416153" containerName="manager"
Jul 06 14:29:11.139719 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:29:11.139699    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="94ca3335-0521-48de-af0d-33b3dd416153" containerName="manager"
Jul 06 14:29:11.144084 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:29:11.144061    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-c7fb74df8-wgl2x"
Jul 06 14:29:11.150142 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:29:11.150122    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-mwgcs\""
Jul 06 14:29:11.150290 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:29:11.150270    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-serving-cert\""
Jul 06 14:29:11.161063 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:29:11.161033    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-c7fb74df8-wgl2x"]
Jul 06 14:29:11.235460 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:29:11.235423    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8f1e0af1-095f-4f50-ab37-d5d06814aa49-maas-api-tls\") pod \"maas-api-c7fb74df8-wgl2x\" (UID: \"8f1e0af1-095f-4f50-ab37-d5d06814aa49\") " pod="opendatahub/maas-api-c7fb74df8-wgl2x"
Jul 06 14:29:11.235663 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:29:11.235475    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nhlbj\" (UniqueName: \"kubernetes.io/projected/8f1e0af1-095f-4f50-ab37-d5d06814aa49-kube-api-access-nhlbj\") pod \"maas-api-c7fb74df8-wgl2x\" (UID: \"8f1e0af1-095f-4f50-ab37-d5d06814aa49\") " pod="opendatahub/maas-api-c7fb74df8-wgl2x"
Jul 06 14:29:11.235663 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:29:11.235507    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8f1e0af1-095f-4f50-ab37-d5d06814aa49-openshift-service-ca\") pod \"maas-api-c7fb74df8-wgl2x\" (UID: \"8f1e0af1-095f-4f50-ab37-d5d06814aa49\") " pod="opendatahub/maas-api-c7fb74df8-wgl2x"
Jul 06 14:29:11.336953 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:29:11.336909    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8f1e0af1-095f-4f50-ab37-d5d06814aa49-maas-api-tls\") pod \"maas-api-c7fb74df8-wgl2x\" (UID: \"8f1e0af1-095f-4f50-ab37-d5d06814aa49\") " pod="opendatahub/maas-api-c7fb74df8-wgl2x"
Jul 06 14:29:11.337137 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:29:11.336973    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nhlbj\" (UniqueName: \"kubernetes.io/projected/8f1e0af1-095f-4f50-ab37-d5d06814aa49-kube-api-access-nhlbj\") pod \"maas-api-c7fb74df8-wgl2x\" (UID: \"8f1e0af1-095f-4f50-ab37-d5d06814aa49\") " pod="opendatahub/maas-api-c7fb74df8-wgl2x"
Jul 06 14:29:11.337137 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:29:11.337023    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8f1e0af1-095f-4f50-ab37-d5d06814aa49-openshift-service-ca\") pod \"maas-api-c7fb74df8-wgl2x\" (UID: \"8f1e0af1-095f-4f50-ab37-d5d06814aa49\") " pod="opendatahub/maas-api-c7fb74df8-wgl2x"
Jul 06 14:29:11.337754 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:29:11.337735    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8f1e0af1-095f-4f50-ab37-d5d06814aa49-openshift-service-ca\") pod \"maas-api-c7fb74df8-wgl2x\" (UID: \"8f1e0af1-095f-4f50-ab37-d5d06814aa49\") " pod="opendatahub/maas-api-c7fb74df8-wgl2x"
Jul 06 14:29:11.339483 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:29:11.339455    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8f1e0af1-095f-4f50-ab37-d5d06814aa49-maas-api-tls\") pod \"maas-api-c7fb74df8-wgl2x\" (UID: \"8f1e0af1-095f-4f50-ab37-d5d06814aa49\") " pod="opendatahub/maas-api-c7fb74df8-wgl2x"
Jul 06 14:29:11.349512 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:29:11.349483    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nhlbj\" (UniqueName: \"kubernetes.io/projected/8f1e0af1-095f-4f50-ab37-d5d06814aa49-kube-api-access-nhlbj\") pod \"maas-api-c7fb74df8-wgl2x\" (UID: \"8f1e0af1-095f-4f50-ab37-d5d06814aa49\") " pod="opendatahub/maas-api-c7fb74df8-wgl2x"
Jul 06 14:29:11.454070 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:29:11.454032    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-c7fb74df8-wgl2x"
Jul 06 14:29:11.604566 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:29:11.604532    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-c7fb74df8-wgl2x"]
Jul 06 14:29:11.611065 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:29:11.611036    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f1e0af1_095f_4f50_ab37_d5d06814aa49.slice/crio-df3f0360aa48af7c823fdba6ada285b1368454ec841c6a79da0632bfecd47089 WatchSource:0}: Error finding container df3f0360aa48af7c823fdba6ada285b1368454ec841c6a79da0632bfecd47089: Status 404 returned error can't find the container with id df3f0360aa48af7c823fdba6ada285b1368454ec841c6a79da0632bfecd47089
Jul 06 14:29:11.665799 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:29:11.665759    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-c7fb74df8-wgl2x" event={"ID":"8f1e0af1-095f-4f50-ab37-d5d06814aa49","Type":"ContainerStarted","Data":"df3f0360aa48af7c823fdba6ada285b1368454ec841c6a79da0632bfecd47089"}
Jul 06 14:29:11.903205 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:29:11.903125    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-clksh"]
Jul 06 14:29:11.906739 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:29:11.906718    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-clksh"
Jul 06 14:29:11.909374 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:29:11.909347    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Jul 06 14:29:11.917874 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:29:11.917846    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-clksh"]
Jul 06 14:29:11.942792 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:29:11.942756    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/10c09d8b-de7d-4798-a71c-971997b4fbe0-plugins-config-volume\") pod \"payload-pre-processing-7964457696-clksh\" (UID: \"10c09d8b-de7d-4798-a71c-971997b4fbe0\") " pod="openshift-ingress/payload-pre-processing-7964457696-clksh"
Jul 06 14:29:11.942975 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:29:11.942820    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nszsc\" (UniqueName: \"kubernetes.io/projected/10c09d8b-de7d-4798-a71c-971997b4fbe0-kube-api-access-nszsc\") pod \"payload-pre-processing-7964457696-clksh\" (UID: \"10c09d8b-de7d-4798-a71c-971997b4fbe0\") " pod="openshift-ingress/payload-pre-processing-7964457696-clksh"
Jul 06 14:29:11.981310 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:29:11.981275    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-56f8d"]
Jul 06 14:29:11.984094 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:29:11.984076    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-56f8d"
Jul 06 14:29:11.986590 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:29:11.986542    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-qv9l6\""
Jul 06 14:29:11.994024 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:29:11.993999    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-56f8d"]
Jul 06 14:29:12.043867 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:29:12.043833    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/10c09d8b-de7d-4798-a71c-971997b4fbe0-plugins-config-volume\") pod \"payload-pre-processing-7964457696-clksh\" (UID: \"10c09d8b-de7d-4798-a71c-971997b4fbe0\") " pod="openshift-ingress/payload-pre-processing-7964457696-clksh"
Jul 06 14:29:12.044056 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:29:12.043873    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f7bed500-2a8f-4fab-aaa4-1122ffdc412b-plugins-config-volume\") pod \"payload-processing-58f86c7798-56f8d\" (UID: \"f7bed500-2a8f-4fab-aaa4-1122ffdc412b\") " pod="openshift-ingress/payload-processing-58f86c7798-56f8d"
Jul 06 14:29:12.044056 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:29:12.043934    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6mbvh\" (UniqueName: \"kubernetes.io/projected/f7bed500-2a8f-4fab-aaa4-1122ffdc412b-kube-api-access-6mbvh\") pod \"payload-processing-58f86c7798-56f8d\" (UID: \"f7bed500-2a8f-4fab-aaa4-1122ffdc412b\") " pod="openshift-ingress/payload-processing-58f86c7798-56f8d"
Jul 06 14:29:12.044056 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:29:12.043968    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nszsc\" (UniqueName: \"kubernetes.io/projected/10c09d8b-de7d-4798-a71c-971997b4fbe0-kube-api-access-nszsc\") pod \"payload-pre-processing-7964457696-clksh\" (UID: \"10c09d8b-de7d-4798-a71c-971997b4fbe0\") " pod="openshift-ingress/payload-pre-processing-7964457696-clksh"
Jul 06 14:29:12.044627 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:29:12.044605    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/10c09d8b-de7d-4798-a71c-971997b4fbe0-plugins-config-volume\") pod \"payload-pre-processing-7964457696-clksh\" (UID: \"10c09d8b-de7d-4798-a71c-971997b4fbe0\") " pod="openshift-ingress/payload-pre-processing-7964457696-clksh"
Jul 06 14:29:12.054806 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:29:12.054777    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nszsc\" (UniqueName: \"kubernetes.io/projected/10c09d8b-de7d-4798-a71c-971997b4fbe0-kube-api-access-nszsc\") pod \"payload-pre-processing-7964457696-clksh\" (UID: \"10c09d8b-de7d-4798-a71c-971997b4fbe0\") " pod="openshift-ingress/payload-pre-processing-7964457696-clksh"
Jul 06 14:29:12.145009 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:29:12.144968    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6mbvh\" (UniqueName: \"kubernetes.io/projected/f7bed500-2a8f-4fab-aaa4-1122ffdc412b-kube-api-access-6mbvh\") pod \"payload-processing-58f86c7798-56f8d\" (UID: \"f7bed500-2a8f-4fab-aaa4-1122ffdc412b\") " pod="openshift-ingress/payload-processing-58f86c7798-56f8d"
Jul 06 14:29:12.145491 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:29:12.145046    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f7bed500-2a8f-4fab-aaa4-1122ffdc412b-plugins-config-volume\") pod \"payload-processing-58f86c7798-56f8d\" (UID: \"f7bed500-2a8f-4fab-aaa4-1122ffdc412b\") " pod="openshift-ingress/payload-processing-58f86c7798-56f8d"
Jul 06 14:29:12.145796 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:29:12.145775    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f7bed500-2a8f-4fab-aaa4-1122ffdc412b-plugins-config-volume\") pod \"payload-processing-58f86c7798-56f8d\" (UID: \"f7bed500-2a8f-4fab-aaa4-1122ffdc412b\") " pod="openshift-ingress/payload-processing-58f86c7798-56f8d"
Jul 06 14:29:12.153689 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:29:12.153638    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6mbvh\" (UniqueName: \"kubernetes.io/projected/f7bed500-2a8f-4fab-aaa4-1122ffdc412b-kube-api-access-6mbvh\") pod \"payload-processing-58f86c7798-56f8d\" (UID: \"f7bed500-2a8f-4fab-aaa4-1122ffdc412b\") " pod="openshift-ingress/payload-processing-58f86c7798-56f8d"
Jul 06 14:29:12.218446 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:29:12.218414    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-clksh"
Jul 06 14:29:12.295083 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:29:12.295049    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-56f8d"
Jul 06 14:29:12.387202 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:29:12.387160    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-clksh"]
Jul 06 14:29:12.391933 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:29:12.391906    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10c09d8b_de7d_4798_a71c_971997b4fbe0.slice/crio-85c6c2a0a0cb0d3e807404e6cbe97b09627f6fd3cb1541110cbda710ef072452 WatchSource:0}: Error finding container 85c6c2a0a0cb0d3e807404e6cbe97b09627f6fd3cb1541110cbda710ef072452: Status 404 returned error can't find the container with id 85c6c2a0a0cb0d3e807404e6cbe97b09627f6fd3cb1541110cbda710ef072452
Jul 06 14:29:12.471350 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:29:12.471309    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-56f8d"]
Jul 06 14:29:12.478256 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:29:12.478218    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7bed500_2a8f_4fab_aaa4_1122ffdc412b.slice/crio-f7da79ac878f405cd19aa97af8dbabfbc7e2070ed6ae67e687bcd348a2251e9c WatchSource:0}: Error finding container f7da79ac878f405cd19aa97af8dbabfbc7e2070ed6ae67e687bcd348a2251e9c: Status 404 returned error can't find the container with id f7da79ac878f405cd19aa97af8dbabfbc7e2070ed6ae67e687bcd348a2251e9c
Jul 06 14:29:12.672790 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:29:12.672753    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-56f8d" event={"ID":"f7bed500-2a8f-4fab-aaa4-1122ffdc412b","Type":"ContainerStarted","Data":"f7da79ac878f405cd19aa97af8dbabfbc7e2070ed6ae67e687bcd348a2251e9c"}
Jul 06 14:29:12.673977 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:29:12.673952    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-clksh" event={"ID":"10c09d8b-de7d-4798-a71c-971997b4fbe0","Type":"ContainerStarted","Data":"85c6c2a0a0cb0d3e807404e6cbe97b09627f6fd3cb1541110cbda710ef072452"}
Jul 06 14:29:13.705824 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:29:13.704875    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-c7fb74df8-wgl2x" event={"ID":"8f1e0af1-095f-4f50-ab37-d5d06814aa49","Type":"ContainerStarted","Data":"52c1a5a45205a66c38d3c15877390a77175084f467d31eb609dd25620117b399"}
Jul 06 14:29:13.705824 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:29:13.705755    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-c7fb74df8-wgl2x"
Jul 06 14:29:13.727430 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:29:13.727165    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-c7fb74df8-wgl2x" podStartSLOduration=0.998705969 podStartE2EDuration="2.727146929s" totalImagesPullingTime="1.72844096s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:29:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:29:13.725243046 +0000 UTC m=+576.032535131" watchObservedRunningTime="2026-07-06 14:29:13.727146929 +0000 UTC m=+576.034439010"
Jul 06 14:29:15.719667 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:29:15.719626    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-56f8d" event={"ID":"f7bed500-2a8f-4fab-aaa4-1122ffdc412b","Type":"ContainerStarted","Data":"7a1e4fad336f95c11257421d20bd534d4190f9768de75ace4b2a891b390e95f2"}
Jul 06 14:29:15.720112 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:29:15.719699    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-58f86c7798-56f8d"
Jul 06 14:29:15.721200 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:29:15.721177    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-clksh" event={"ID":"10c09d8b-de7d-4798-a71c-971997b4fbe0","Type":"ContainerStarted","Data":"db1faf97cc287d1cef3fe3e57b953652af539a873b92beb5bc36e942535b5179"}
Jul 06 14:29:15.738654 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:29:15.738607    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-58f86c7798-56f8d" podStartSLOduration=2.598770993 podStartE2EDuration="4.738591328s" totalImagesPullingTime="2.139820335s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:29:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:29:15.737297941 +0000 UTC m=+578.044590024" watchObservedRunningTime="2026-07-06 14:29:15.738591328 +0000 UTC m=+578.045883409"
Jul 06 14:29:15.753221 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:29:15.753164    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7964457696-clksh" podStartSLOduration=2.524650664 podStartE2EDuration="4.753146251s" totalImagesPullingTime="2.228495587s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:29:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:29:15.753105478 +0000 UTC m=+578.060397563" watchObservedRunningTime="2026-07-06 14:29:15.753146251 +0000 UTC m=+578.060438334"
Jul 06 14:29:16.726497 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:29:16.726464    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7964457696-clksh"
Jul 06 14:29:19.742518 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:29:19.742434    2559 generic.go:356] "Generic (PLEG): container finished" podID="065f6787-1fa0-40e0-ae75-6f87a4df4d10" containerID="2a3c184f557f77febb333066e328f8f0937a3bf7219fe2e0731e2c939ee65c22" exitCode=0
Jul 06 14:29:19.742518 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:29:19.742491    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vszcz" event={"ID":"065f6787-1fa0-40e0-ae75-6f87a4df4d10","Type":"ContainerDied","Data":"2a3c184f557f77febb333066e328f8f0937a3bf7219fe2e0731e2c939ee65c22"}
Jul 06 14:29:19.742966 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:29:19.742867    2559 scope.go:117] "RemoveContainer" containerID="2a3c184f557f77febb333066e328f8f0937a3bf7219fe2e0731e2c939ee65c22"
Jul 06 14:29:19.743854 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:29:19.743820    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 06 14:29:19.743915 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:29:19.743892    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 06 14:29:19.743949 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:29:19.743925    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 06 14:29:20.727277 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:29:20.727247    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-c7fb74df8-wgl2x"
Jul 06 14:29:20.749588 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:29:20.749547    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vszcz" event={"ID":"065f6787-1fa0-40e0-ae75-6f87a4df4d10","Type":"ContainerStarted","Data":"417969fafbb92d35c6a8df306390670011973bc29cc2a9daab4e9a102422a6e2"}
Jul 06 14:29:21.287256 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:29:21.287160    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vszcz"
Jul 06 14:29:21.288108 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:29:21.288073    2559 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-vszcz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.30:15021/healthz/ready\": dial tcp 10.132.0.30:15021: connect: connection refused" start-of-body=
Jul 06 14:29:21.288242 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:29:21.288132    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vszcz" podUID="065f6787-1fa0-40e0-ae75-6f87a4df4d10" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.30:15021/healthz/ready\": dial tcp 10.132.0.30:15021: connect: connection refused"
Jul 06 14:29:22.287264 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:29:22.287216    2559 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-vszcz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.30:15021/healthz/ready\": dial tcp 10.132.0.30:15021: connect: connection refused" start-of-body=
Jul 06 14:29:22.287841 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:29:22.287299    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vszcz" podUID="065f6787-1fa0-40e0-ae75-6f87a4df4d10" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.30:15021/healthz/ready\": dial tcp 10.132.0.30:15021: connect: connection refused"
Jul 06 14:29:23.287439 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:29:23.287404    2559 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-vszcz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.30:15021/healthz/ready\": dial tcp 10.132.0.30:15021: connect: connection refused" start-of-body=
Jul 06 14:29:23.288080 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:29:23.288045    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vszcz" podUID="065f6787-1fa0-40e0-ae75-6f87a4df4d10" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.30:15021/healthz/ready\": dial tcp 10.132.0.30:15021: connect: connection refused"
Jul 06 14:29:23.308776 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:29:23.308742    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vszcz"
Jul 06 14:29:24.288336 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:29:24.287895    2559 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-vszcz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.30:15021/healthz/ready\": dial tcp 10.132.0.30:15021: connect: connection refused" start-of-body=
Jul 06 14:29:24.288336 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:29:24.287959    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vszcz" podUID="065f6787-1fa0-40e0-ae75-6f87a4df4d10" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.30:15021/healthz/ready\": dial tcp 10.132.0.30:15021: connect: connection refused"
Jul 06 14:29:25.291369 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:29:25.291339    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vszcz"
Jul 06 14:29:25.292412 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:29:25.292394    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vszcz"
Jul 06 14:29:26.728355 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:29:26.728324    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-58f86c7798-56f8d"
Jul 06 14:29:27.733316 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:29:27.733281    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7964457696-clksh"
Jul 06 14:29:31.351684 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:29:31.351643    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-zpmrz"]
Jul 06 14:29:31.354417 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:29:31.354394    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-zpmrz"
Jul 06 14:29:31.357685 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:29:31.357656    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-pgwdp\""
Jul 06 14:29:31.357818 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:29:31.357690    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 06 14:29:31.357818 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:29:31.357661    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Jul 06 14:29:31.357818 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:29:31.357661    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 06 14:29:31.365634 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:29:31.365615    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-zpmrz"]
Jul 06 14:29:31.417616 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:29:31.417564    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0b873ff1-9a28-4917-97af-09a20796835f-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-zpmrz\" (UID: \"0b873ff1-9a28-4917-97af-09a20796835f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-zpmrz"
Jul 06 14:29:31.417775 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:29:31.417657    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kxv46\" (UniqueName: \"kubernetes.io/projected/0b873ff1-9a28-4917-97af-09a20796835f-kube-api-access-kxv46\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-zpmrz\" (UID: \"0b873ff1-9a28-4917-97af-09a20796835f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-zpmrz"
Jul 06 14:29:31.417775 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:29:31.417689    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0b873ff1-9a28-4917-97af-09a20796835f-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-zpmrz\" (UID: \"0b873ff1-9a28-4917-97af-09a20796835f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-zpmrz"
Jul 06 14:29:31.417775 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:29:31.417706    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0b873ff1-9a28-4917-97af-09a20796835f-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-zpmrz\" (UID: \"0b873ff1-9a28-4917-97af-09a20796835f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-zpmrz"
Jul 06 14:29:31.417902 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:29:31.417786    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0b873ff1-9a28-4917-97af-09a20796835f-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-zpmrz\" (UID: \"0b873ff1-9a28-4917-97af-09a20796835f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-zpmrz"
Jul 06 14:29:31.417902 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:29:31.417808    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0b873ff1-9a28-4917-97af-09a20796835f-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-zpmrz\" (UID: \"0b873ff1-9a28-4917-97af-09a20796835f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-zpmrz"
Jul 06 14:29:31.518746 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:29:31.518712    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0b873ff1-9a28-4917-97af-09a20796835f-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-zpmrz\" (UID: \"0b873ff1-9a28-4917-97af-09a20796835f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-zpmrz"
Jul 06 14:29:31.518746 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:29:31.518745    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0b873ff1-9a28-4917-97af-09a20796835f-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-zpmrz\" (UID: \"0b873ff1-9a28-4917-97af-09a20796835f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-zpmrz"
Jul 06 14:29:31.518978 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:29:31.518802    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0b873ff1-9a28-4917-97af-09a20796835f-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-zpmrz\" (UID: \"0b873ff1-9a28-4917-97af-09a20796835f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-zpmrz"
Jul 06 14:29:31.518978 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:29:31.518833    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0b873ff1-9a28-4917-97af-09a20796835f-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-zpmrz\" (UID: \"0b873ff1-9a28-4917-97af-09a20796835f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-zpmrz"
Jul 06 14:29:31.518978 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:29:31.518872    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0b873ff1-9a28-4917-97af-09a20796835f-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-zpmrz\" (UID: \"0b873ff1-9a28-4917-97af-09a20796835f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-zpmrz"
Jul 06 14:29:31.518978 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:29:31.518954    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kxv46\" (UniqueName: \"kubernetes.io/projected/0b873ff1-9a28-4917-97af-09a20796835f-kube-api-access-kxv46\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-zpmrz\" (UID: \"0b873ff1-9a28-4917-97af-09a20796835f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-zpmrz"
Jul 06 14:29:31.519192 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:29:31.519172    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0b873ff1-9a28-4917-97af-09a20796835f-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-zpmrz\" (UID: \"0b873ff1-9a28-4917-97af-09a20796835f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-zpmrz"
Jul 06 14:29:31.519232 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:29:31.519207    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0b873ff1-9a28-4917-97af-09a20796835f-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-zpmrz\" (UID: \"0b873ff1-9a28-4917-97af-09a20796835f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-zpmrz"
Jul 06 14:29:31.519277 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:29:31.519254    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0b873ff1-9a28-4917-97af-09a20796835f-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-zpmrz\" (UID: \"0b873ff1-9a28-4917-97af-09a20796835f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-zpmrz"
Jul 06 14:29:31.521107 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:29:31.521076    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0b873ff1-9a28-4917-97af-09a20796835f-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-zpmrz\" (UID: \"0b873ff1-9a28-4917-97af-09a20796835f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-zpmrz"
Jul 06 14:29:31.521368 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:29:31.521349    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0b873ff1-9a28-4917-97af-09a20796835f-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-zpmrz\" (UID: \"0b873ff1-9a28-4917-97af-09a20796835f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-zpmrz"
Jul 06 14:29:31.526853 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:29:31.526828    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kxv46\" (UniqueName: \"kubernetes.io/projected/0b873ff1-9a28-4917-97af-09a20796835f-kube-api-access-kxv46\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-zpmrz\" (UID: \"0b873ff1-9a28-4917-97af-09a20796835f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-zpmrz"
Jul 06 14:29:31.665255 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:29:31.665174    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-zpmrz"
Jul 06 14:29:31.801448 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:29:31.801417    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-zpmrz"]
Jul 06 14:29:31.808609 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:29:31.808564    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b873ff1_9a28_4917_97af_09a20796835f.slice/crio-9a73cdde82f7f98b235fc7d404932eeedc698112e6f1edd75371db5e88687383 WatchSource:0}: Error finding container 9a73cdde82f7f98b235fc7d404932eeedc698112e6f1edd75371db5e88687383: Status 404 returned error can't find the container with id 9a73cdde82f7f98b235fc7d404932eeedc698112e6f1edd75371db5e88687383
Jul 06 14:29:32.805830 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:29:32.805798    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-zpmrz" event={"ID":"0b873ff1-9a28-4917-97af-09a20796835f","Type":"ContainerStarted","Data":"9a73cdde82f7f98b235fc7d404932eeedc698112e6f1edd75371db5e88687383"}
Jul 06 14:29:33.811644 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:29:33.811601    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-zpmrz" event={"ID":"0b873ff1-9a28-4917-97af-09a20796835f","Type":"ContainerStarted","Data":"0f654d1b63790de8e715b6fa6f782df3c2dd2b168a2615e005a02065fc8c0c9d"}
Jul 06 14:29:33.833177 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:29:33.833125    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-zpmrz" podStartSLOduration=1.805348811 podStartE2EDuration="2.833109939s" totalImagesPullingTime="1.027761128s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:29:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:29:33.830697065 +0000 UTC m=+596.137989148" watchObservedRunningTime="2026-07-06 14:29:33.833109939 +0000 UTC m=+596.140402072"
Jul 06 14:29:38.131085 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:29:38.131056    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-g5w5m_e35b0247-7c4e-4bd1-bc28-c084dc40ffa1/console-operator/1.log"
Jul 06 14:29:38.132343 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:29:38.132318    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-g5w5m_e35b0247-7c4e-4bd1-bc28-c084dc40ffa1/console-operator/1.log"
Jul 06 14:29:41.665643 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:29:41.665607    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-zpmrz"
Jul 06 14:29:41.666082 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:29:41.665878    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-zpmrz"
Jul 06 14:29:41.678306 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:29:41.678284    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-zpmrz"
Jul 06 14:29:41.867031 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:29:41.867000    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-zpmrz"
Jul 06 14:30:00.163721 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:30:00.163625    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-29722470-zdppp"]
Jul 06 14:30:00.168671 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:30:00.168654    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29722470-zdppp"
Jul 06 14:30:00.178203 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:30:00.178180    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mhs8z\" (UniqueName: \"kubernetes.io/projected/daa61e06-6298-40c6-8e9d-ca593a1638b8-kube-api-access-mhs8z\") pod \"maas-api-key-cleanup-29722470-zdppp\" (UID: \"daa61e06-6298-40c6-8e9d-ca593a1638b8\") " pod="opendatahub/maas-api-key-cleanup-29722470-zdppp"
Jul 06 14:30:00.180878 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:30:00.180855    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29722470-zdppp"]
Jul 06 14:30:00.278780 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:30:00.278745    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mhs8z\" (UniqueName: \"kubernetes.io/projected/daa61e06-6298-40c6-8e9d-ca593a1638b8-kube-api-access-mhs8z\") pod \"maas-api-key-cleanup-29722470-zdppp\" (UID: \"daa61e06-6298-40c6-8e9d-ca593a1638b8\") " pod="opendatahub/maas-api-key-cleanup-29722470-zdppp"
Jul 06 14:30:00.292373 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:30:00.292350    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mhs8z\" (UniqueName: \"kubernetes.io/projected/daa61e06-6298-40c6-8e9d-ca593a1638b8-kube-api-access-mhs8z\") pod \"maas-api-key-cleanup-29722470-zdppp\" (UID: \"daa61e06-6298-40c6-8e9d-ca593a1638b8\") " pod="opendatahub/maas-api-key-cleanup-29722470-zdppp"
Jul 06 14:30:00.479089 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:30:00.479048    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29722470-zdppp"
Jul 06 14:30:00.627118 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:30:00.627083    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29722470-zdppp"]
Jul 06 14:30:00.635114 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:30:00.635094    2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 06 14:30:00.947662 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:30:00.947628    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29722470-zdppp" event={"ID":"daa61e06-6298-40c6-8e9d-ca593a1638b8","Type":"ContainerStarted","Data":"d6a0a343bb5d82974ab5665e0dec13788afe624192d60f261cf244a7c920e79f"}
Jul 06 14:30:02.347593 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:30:02.347540    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfp7wv"]
Jul 06 14:30:02.352385 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:30:02.352362    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfp7wv"
Jul 06 14:30:02.355047 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:30:02.355026    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Jul 06 14:30:02.371735 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:30:02.371700    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfp7wv"]
Jul 06 14:30:02.495291 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:30:02.495256    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b7f32cfe-8038-4da0-948e-95ecda736905-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfp7wv\" (UID: \"b7f32cfe-8038-4da0-948e-95ecda736905\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfp7wv"
Jul 06 14:30:02.495473 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:30:02.495305    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b7f32cfe-8038-4da0-948e-95ecda736905-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfp7wv\" (UID: \"b7f32cfe-8038-4da0-948e-95ecda736905\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfp7wv"
Jul 06 14:30:02.495473 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:30:02.495397    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b7f32cfe-8038-4da0-948e-95ecda736905-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfp7wv\" (UID: \"b7f32cfe-8038-4da0-948e-95ecda736905\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfp7wv"
Jul 06 14:30:02.495473 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:30:02.495436    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b7f32cfe-8038-4da0-948e-95ecda736905-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfp7wv\" (UID: \"b7f32cfe-8038-4da0-948e-95ecda736905\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfp7wv"
Jul 06 14:30:02.495473 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:30:02.495468    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x5jhh\" (UniqueName: \"kubernetes.io/projected/b7f32cfe-8038-4da0-948e-95ecda736905-kube-api-access-x5jhh\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfp7wv\" (UID: \"b7f32cfe-8038-4da0-948e-95ecda736905\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfp7wv"
Jul 06 14:30:02.495661 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:30:02.495493    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b7f32cfe-8038-4da0-948e-95ecda736905-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfp7wv\" (UID: \"b7f32cfe-8038-4da0-948e-95ecda736905\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfp7wv"
Jul 06 14:30:02.596797 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:30:02.596762    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b7f32cfe-8038-4da0-948e-95ecda736905-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfp7wv\" (UID: \"b7f32cfe-8038-4da0-948e-95ecda736905\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfp7wv"
Jul 06 14:30:02.596980 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:30:02.596811    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b7f32cfe-8038-4da0-948e-95ecda736905-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfp7wv\" (UID: \"b7f32cfe-8038-4da0-948e-95ecda736905\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfp7wv"
Jul 06 14:30:02.596980 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:30:02.596836    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x5jhh\" (UniqueName: \"kubernetes.io/projected/b7f32cfe-8038-4da0-948e-95ecda736905-kube-api-access-x5jhh\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfp7wv\" (UID: \"b7f32cfe-8038-4da0-948e-95ecda736905\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfp7wv"
Jul 06 14:30:02.596980 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:30:02.596952    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b7f32cfe-8038-4da0-948e-95ecda736905-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfp7wv\" (UID: \"b7f32cfe-8038-4da0-948e-95ecda736905\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfp7wv"
Jul 06 14:30:02.597104 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:30:02.597064    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b7f32cfe-8038-4da0-948e-95ecda736905-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfp7wv\" (UID: \"b7f32cfe-8038-4da0-948e-95ecda736905\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfp7wv"
Jul 06 14:30:02.597104 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:30:02.597093    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b7f32cfe-8038-4da0-948e-95ecda736905-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfp7wv\" (UID: \"b7f32cfe-8038-4da0-948e-95ecda736905\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfp7wv"
Jul 06 14:30:02.597420 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:30:02.597397    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b7f32cfe-8038-4da0-948e-95ecda736905-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfp7wv\" (UID: \"b7f32cfe-8038-4da0-948e-95ecda736905\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfp7wv"
Jul 06 14:30:02.597487 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:30:02.597433    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b7f32cfe-8038-4da0-948e-95ecda736905-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfp7wv\" (UID: \"b7f32cfe-8038-4da0-948e-95ecda736905\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfp7wv"
Jul 06 14:30:02.597487 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:30:02.597468    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b7f32cfe-8038-4da0-948e-95ecda736905-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfp7wv\" (UID: \"b7f32cfe-8038-4da0-948e-95ecda736905\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfp7wv"
Jul 06 14:30:02.599272 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:30:02.599219    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b7f32cfe-8038-4da0-948e-95ecda736905-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfp7wv\" (UID: \"b7f32cfe-8038-4da0-948e-95ecda736905\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfp7wv"
Jul 06 14:30:02.599393 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:30:02.599376    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b7f32cfe-8038-4da0-948e-95ecda736905-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfp7wv\" (UID: \"b7f32cfe-8038-4da0-948e-95ecda736905\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfp7wv"
Jul 06 14:30:02.605917 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:30:02.605899    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x5jhh\" (UniqueName: \"kubernetes.io/projected/b7f32cfe-8038-4da0-948e-95ecda736905-kube-api-access-x5jhh\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfp7wv\" (UID: \"b7f32cfe-8038-4da0-948e-95ecda736905\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfp7wv"
Jul 06 14:30:02.663460 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:30:02.663423    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfp7wv"
Jul 06 14:30:02.831032 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:30:02.830998    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfp7wv"]
Jul 06 14:30:02.833271 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:30:02.833241    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7f32cfe_8038_4da0_948e_95ecda736905.slice/crio-a5c4ed3f3dccd75c800e20c6ded48ac42e1c10baaad351e55394657980f097a7 WatchSource:0}: Error finding container a5c4ed3f3dccd75c800e20c6ded48ac42e1c10baaad351e55394657980f097a7: Status 404 returned error can't find the container with id a5c4ed3f3dccd75c800e20c6ded48ac42e1c10baaad351e55394657980f097a7
Jul 06 14:30:02.959705 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:30:02.959660    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfp7wv" event={"ID":"b7f32cfe-8038-4da0-948e-95ecda736905","Type":"ContainerStarted","Data":"a5c4ed3f3dccd75c800e20c6ded48ac42e1c10baaad351e55394657980f097a7"}
Jul 06 14:30:04.971506 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:30:04.971465    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29722470-zdppp" event={"ID":"daa61e06-6298-40c6-8e9d-ca593a1638b8","Type":"ContainerStarted","Data":"fd16d628821ab861dc2deb34eed9ecd6bea00686839d2a0fc08ae995d93a2b22"}
Jul 06 14:30:04.991912 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:30:04.991855    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-key-cleanup-29722470-zdppp" podStartSLOduration=1.605804654 podStartE2EDuration="4.991840071s" totalImagesPullingTime="3.386035417s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:30:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:30:04.989436355 +0000 UTC m=+627.296728438" watchObservedRunningTime="2026-07-06 14:30:04.991840071 +0000 UTC m=+627.299132152"
Jul 06 14:30:05.978138 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:30:05.978096    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfp7wv" event={"ID":"b7f32cfe-8038-4da0-948e-95ecda736905","Type":"ContainerStarted","Data":"588cb36731504d6084f43dc1842812e6112077a33de9a6e966650f78055d825c"}
Jul 06 14:30:05.979718 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:30:05.979690    2559 generic.go:356] "Generic (PLEG): container finished" podID="daa61e06-6298-40c6-8e9d-ca593a1638b8" containerID="fd16d628821ab861dc2deb34eed9ecd6bea00686839d2a0fc08ae995d93a2b22" exitCode=0
Jul 06 14:30:05.979838 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:30:05.979731    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29722470-zdppp" event={"ID":"daa61e06-6298-40c6-8e9d-ca593a1638b8","Type":"ContainerDied","Data":"fd16d628821ab861dc2deb34eed9ecd6bea00686839d2a0fc08ae995d93a2b22"}
Jul 06 14:30:06.001840 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:30:06.001786    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfp7wv" podStartSLOduration=1.754819322 podStartE2EDuration="4.001770768s" totalImagesPullingTime="2.246951446s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:30:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:30:05.999910969 +0000 UTC m=+628.307203074" watchObservedRunningTime="2026-07-06 14:30:06.001770768 +0000 UTC m=+628.309062849"
Jul 06 14:30:07.123322 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:30:07.123292    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29722470-zdppp"
Jul 06 14:30:07.139147 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:30:07.139119    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mhs8z\" (UniqueName: \"kubernetes.io/projected/daa61e06-6298-40c6-8e9d-ca593a1638b8-kube-api-access-mhs8z\") pod \"daa61e06-6298-40c6-8e9d-ca593a1638b8\" (UID: \"daa61e06-6298-40c6-8e9d-ca593a1638b8\") "
Jul 06 14:30:07.141791 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:30:07.141756    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/daa61e06-6298-40c6-8e9d-ca593a1638b8-kube-api-access-mhs8z" (OuterVolumeSpecName: "kube-api-access-mhs8z") pod "daa61e06-6298-40c6-8e9d-ca593a1638b8" (UID: "daa61e06-6298-40c6-8e9d-ca593a1638b8"). InnerVolumeSpecName "kube-api-access-mhs8z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 14:30:07.240232 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:30:07.240143    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mhs8z\" (UniqueName: \"kubernetes.io/projected/daa61e06-6298-40c6-8e9d-ca593a1638b8-kube-api-access-mhs8z\") on node \"ip-10-0-133-112.ec2.internal\" DevicePath \"\""
Jul 06 14:30:07.990882 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:30:07.990841    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29722470-zdppp"
Jul 06 14:30:07.991073 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:30:07.990840    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29722470-zdppp" event={"ID":"daa61e06-6298-40c6-8e9d-ca593a1638b8","Type":"ContainerDied","Data":"d6a0a343bb5d82974ab5665e0dec13788afe624192d60f261cf244a7c920e79f"}
Jul 06 14:30:07.991073 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:30:07.990988    2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d6a0a343bb5d82974ab5665e0dec13788afe624192d60f261cf244a7c920e79f"
Jul 06 14:30:12.663990 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:30:12.663949    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfp7wv"
Jul 06 14:30:12.664398 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:30:12.664012    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfp7wv"
Jul 06 14:30:12.676407 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:30:12.676380    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfp7wv"
Jul 06 14:30:13.025903 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:30:13.025869    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfp7wv"
Jul 06 14:33:01.837223 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:33:01.837136    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-58b9bc5c65-9cs44"]
Jul 06 14:33:01.837806 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:33:01.837393    2559 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-58b9bc5c65-9cs44" podUID="42bbc67a-8c09-40ba-8c41-70807782ab78" containerName="manager" containerID="cri-o://6ffd8c95dc9c6c152550f98ef7c051f0757f448d9e50dbf9e43d15419463c105" gracePeriod=30
Jul 06 14:33:02.089964 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:33:02.089904    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-58b9bc5c65-9cs44"
Jul 06 14:33:02.129998 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:33:02.129968    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gvscw\" (UniqueName: \"kubernetes.io/projected/42bbc67a-8c09-40ba-8c41-70807782ab78-kube-api-access-gvscw\") pod \"42bbc67a-8c09-40ba-8c41-70807782ab78\" (UID: \"42bbc67a-8c09-40ba-8c41-70807782ab78\") "
Jul 06 14:33:02.130174 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:33:02.130009    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/42bbc67a-8c09-40ba-8c41-70807782ab78-webhook-cert\") pod \"42bbc67a-8c09-40ba-8c41-70807782ab78\" (UID: \"42bbc67a-8c09-40ba-8c41-70807782ab78\") "
Jul 06 14:33:02.131960 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:33:02.131925    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/42bbc67a-8c09-40ba-8c41-70807782ab78-kube-api-access-gvscw" (OuterVolumeSpecName: "kube-api-access-gvscw") pod "42bbc67a-8c09-40ba-8c41-70807782ab78" (UID: "42bbc67a-8c09-40ba-8c41-70807782ab78"). InnerVolumeSpecName "kube-api-access-gvscw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 14:33:02.132064 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:33:02.132002    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/42bbc67a-8c09-40ba-8c41-70807782ab78-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "42bbc67a-8c09-40ba-8c41-70807782ab78" (UID: "42bbc67a-8c09-40ba-8c41-70807782ab78"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 14:33:02.231080 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:33:02.231046    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gvscw\" (UniqueName: \"kubernetes.io/projected/42bbc67a-8c09-40ba-8c41-70807782ab78-kube-api-access-gvscw\") on node \"ip-10-0-133-112.ec2.internal\" DevicePath \"\""
Jul 06 14:33:02.231080 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:33:02.231074    2559 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/42bbc67a-8c09-40ba-8c41-70807782ab78-webhook-cert\") on node \"ip-10-0-133-112.ec2.internal\" DevicePath \"\""
Jul 06 14:33:02.815714 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:33:02.815680    2559 generic.go:356] "Generic (PLEG): container finished" podID="42bbc67a-8c09-40ba-8c41-70807782ab78" containerID="6ffd8c95dc9c6c152550f98ef7c051f0757f448d9e50dbf9e43d15419463c105" exitCode=0
Jul 06 14:33:02.815906 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:33:02.815727    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-58b9bc5c65-9cs44"
Jul 06 14:33:02.815906 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:33:02.815767    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-58b9bc5c65-9cs44" event={"ID":"42bbc67a-8c09-40ba-8c41-70807782ab78","Type":"ContainerDied","Data":"6ffd8c95dc9c6c152550f98ef7c051f0757f448d9e50dbf9e43d15419463c105"}
Jul 06 14:33:02.815906 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:33:02.815827    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-58b9bc5c65-9cs44" event={"ID":"42bbc67a-8c09-40ba-8c41-70807782ab78","Type":"ContainerDied","Data":"c36c1eb45d58671d6bd9bf8da4633fa9a275dcaaf9bfa5b2283f2721b38431b5"}
Jul 06 14:33:02.815906 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:33:02.815854    2559 scope.go:117] "RemoveContainer" containerID="6ffd8c95dc9c6c152550f98ef7c051f0757f448d9e50dbf9e43d15419463c105"
Jul 06 14:33:02.824674 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:33:02.824647    2559 scope.go:117] "RemoveContainer" containerID="6ffd8c95dc9c6c152550f98ef7c051f0757f448d9e50dbf9e43d15419463c105"
Jul 06 14:33:02.824980 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:33:02.824958    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6ffd8c95dc9c6c152550f98ef7c051f0757f448d9e50dbf9e43d15419463c105\": container with ID starting with 6ffd8c95dc9c6c152550f98ef7c051f0757f448d9e50dbf9e43d15419463c105 not found: ID does not exist" containerID="6ffd8c95dc9c6c152550f98ef7c051f0757f448d9e50dbf9e43d15419463c105"
Jul 06 14:33:02.825041 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:33:02.824988    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6ffd8c95dc9c6c152550f98ef7c051f0757f448d9e50dbf9e43d15419463c105"} err="failed to get container status \"6ffd8c95dc9c6c152550f98ef7c051f0757f448d9e50dbf9e43d15419463c105\": rpc error: code = NotFound desc = could not find container \"6ffd8c95dc9c6c152550f98ef7c051f0757f448d9e50dbf9e43d15419463c105\": container with ID starting with 6ffd8c95dc9c6c152550f98ef7c051f0757f448d9e50dbf9e43d15419463c105 not found: ID does not exist"
Jul 06 14:33:02.834276 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:33:02.834253    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-58b9bc5c65-9cs44"]
Jul 06 14:33:02.838076 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:33:02.838053    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-58b9bc5c65-9cs44"]
Jul 06 14:33:03.682002 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:33:03.681961    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-58b9bc5c65-m78gp"]
Jul 06 14:33:03.682440 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:33:03.682420    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="daa61e06-6298-40c6-8e9d-ca593a1638b8" containerName="cleanup"
Jul 06 14:33:03.682440 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:33:03.682442    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="daa61e06-6298-40c6-8e9d-ca593a1638b8" containerName="cleanup"
Jul 06 14:33:03.682622 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:33:03.682459    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="42bbc67a-8c09-40ba-8c41-70807782ab78" containerName="manager"
Jul 06 14:33:03.682622 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:33:03.682466    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="42bbc67a-8c09-40ba-8c41-70807782ab78" containerName="manager"
Jul 06 14:33:03.682622 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:33:03.682542    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="daa61e06-6298-40c6-8e9d-ca593a1638b8" containerName="cleanup"
Jul 06 14:33:03.682622 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:33:03.682557    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="42bbc67a-8c09-40ba-8c41-70807782ab78" containerName="manager"
Jul 06 14:33:03.687039 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:33:03.687017    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-58b9bc5c65-m78gp"
Jul 06 14:33:03.690131 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:33:03.690109    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-hz2hk\""
Jul 06 14:33:03.690233 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:33:03.690109    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 06 14:33:03.690233 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:33:03.690148    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 06 14:33:03.696111 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:33:03.696088    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-58b9bc5c65-m78gp"]
Jul 06 14:33:03.744178 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:33:03.744142    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lvft4\" (UniqueName: \"kubernetes.io/projected/ba19bb7d-8ef5-4e75-884d-cb4a9a25d7e7-kube-api-access-lvft4\") pod \"maas-controller-58b9bc5c65-m78gp\" (UID: \"ba19bb7d-8ef5-4e75-884d-cb4a9a25d7e7\") " pod="opendatahub/maas-controller-58b9bc5c65-m78gp"
Jul 06 14:33:03.744178 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:33:03.744180    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ba19bb7d-8ef5-4e75-884d-cb4a9a25d7e7-webhook-cert\") pod \"maas-controller-58b9bc5c65-m78gp\" (UID: \"ba19bb7d-8ef5-4e75-884d-cb4a9a25d7e7\") " pod="opendatahub/maas-controller-58b9bc5c65-m78gp"
Jul 06 14:33:03.844836 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:33:03.844799    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lvft4\" (UniqueName: \"kubernetes.io/projected/ba19bb7d-8ef5-4e75-884d-cb4a9a25d7e7-kube-api-access-lvft4\") pod \"maas-controller-58b9bc5c65-m78gp\" (UID: \"ba19bb7d-8ef5-4e75-884d-cb4a9a25d7e7\") " pod="opendatahub/maas-controller-58b9bc5c65-m78gp"
Jul 06 14:33:03.845239 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:33:03.844842    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ba19bb7d-8ef5-4e75-884d-cb4a9a25d7e7-webhook-cert\") pod \"maas-controller-58b9bc5c65-m78gp\" (UID: \"ba19bb7d-8ef5-4e75-884d-cb4a9a25d7e7\") " pod="opendatahub/maas-controller-58b9bc5c65-m78gp"
Jul 06 14:33:03.847476 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:33:03.847435    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ba19bb7d-8ef5-4e75-884d-cb4a9a25d7e7-webhook-cert\") pod \"maas-controller-58b9bc5c65-m78gp\" (UID: \"ba19bb7d-8ef5-4e75-884d-cb4a9a25d7e7\") " pod="opendatahub/maas-controller-58b9bc5c65-m78gp"
Jul 06 14:33:03.853318 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:33:03.853295    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lvft4\" (UniqueName: \"kubernetes.io/projected/ba19bb7d-8ef5-4e75-884d-cb4a9a25d7e7-kube-api-access-lvft4\") pod \"maas-controller-58b9bc5c65-m78gp\" (UID: \"ba19bb7d-8ef5-4e75-884d-cb4a9a25d7e7\") " pod="opendatahub/maas-controller-58b9bc5c65-m78gp"
Jul 06 14:33:03.997712 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:33:03.997619    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-58b9bc5c65-m78gp"
Jul 06 14:33:04.133337 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:33:04.133307    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-58b9bc5c65-m78gp"]
Jul 06 14:33:04.135333 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:33:04.135305    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba19bb7d_8ef5_4e75_884d_cb4a9a25d7e7.slice/crio-a07fbdb92d298b2c83ddef1b4c8c029cab5d4b1eb420999cbac9a03d12e5cc85 WatchSource:0}: Error finding container a07fbdb92d298b2c83ddef1b4c8c029cab5d4b1eb420999cbac9a03d12e5cc85: Status 404 returned error can't find the container with id a07fbdb92d298b2c83ddef1b4c8c029cab5d4b1eb420999cbac9a03d12e5cc85
Jul 06 14:33:04.227065 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:33:04.227035    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="42bbc67a-8c09-40ba-8c41-70807782ab78" path="/var/lib/kubelet/pods/42bbc67a-8c09-40ba-8c41-70807782ab78/volumes"
Jul 06 14:33:04.828299 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:33:04.828272    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-58b9bc5c65-m78gp" event={"ID":"ba19bb7d-8ef5-4e75-884d-cb4a9a25d7e7","Type":"ContainerStarted","Data":"a07fbdb92d298b2c83ddef1b4c8c029cab5d4b1eb420999cbac9a03d12e5cc85"}
Jul 06 14:33:05.834220 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:33:05.834182    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-58b9bc5c65-m78gp" event={"ID":"ba19bb7d-8ef5-4e75-884d-cb4a9a25d7e7","Type":"ContainerStarted","Data":"7f92d3def7cd277270b54ef711f818308b415f8fb2bf70862d43f8b0dbc6a587"}
Jul 06 14:33:05.834614 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:33:05.834276    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-58b9bc5c65-m78gp"
Jul 06 14:33:05.854921 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:33:05.854870    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-58b9bc5c65-m78gp" podStartSLOduration=2.194807134 podStartE2EDuration="2.854855234s" totalImagesPullingTime="660.0481ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:33:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:33:05.854633838 +0000 UTC m=+808.161925917" watchObservedRunningTime="2026-07-06 14:33:05.854855234 +0000 UTC m=+808.162147315"
Jul 06 14:33:16.846018 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:33:16.845985    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-58b9bc5c65-m78gp"
Jul 06 14:34:38.160558 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:34:38.160529    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-g5w5m_e35b0247-7c4e-4bd1-bc28-c084dc40ffa1/console-operator/1.log"
Jul 06 14:34:38.161072 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:34:38.160619    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-g5w5m_e35b0247-7c4e-4bd1-bc28-c084dc40ffa1/console-operator/1.log"
Jul 06 14:39:38.195375 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:39:38.195338    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-g5w5m_e35b0247-7c4e-4bd1-bc28-c084dc40ffa1/console-operator/1.log"
Jul 06 14:39:38.200010 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:39:38.196379    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-g5w5m_e35b0247-7c4e-4bd1-bc28-c084dc40ffa1/console-operator/1.log"
Jul 06 14:44:38.225356 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:44:38.225234    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-g5w5m_e35b0247-7c4e-4bd1-bc28-c084dc40ffa1/console-operator/1.log"
Jul 06 14:44:38.229514 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:44:38.226291    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-g5w5m_e35b0247-7c4e-4bd1-bc28-c084dc40ffa1/console-operator/1.log"
Jul 06 14:45:00.152816 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:00.152721    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-29722485-97sgd"]
Jul 06 14:45:00.156195 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:00.156172    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29722485-97sgd"
Jul 06 14:45:00.229152 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:00.229119    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29722485-97sgd"]
Jul 06 14:45:00.256410 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:00.256374    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d4nlg\" (UniqueName: \"kubernetes.io/projected/97633ee4-b353-43f6-ad58-8b726df28551-kube-api-access-d4nlg\") pod \"maas-api-key-cleanup-29722485-97sgd\" (UID: \"97633ee4-b353-43f6-ad58-8b726df28551\") " pod="opendatahub/maas-api-key-cleanup-29722485-97sgd"
Jul 06 14:45:00.357531 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:00.357491    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d4nlg\" (UniqueName: \"kubernetes.io/projected/97633ee4-b353-43f6-ad58-8b726df28551-kube-api-access-d4nlg\") pod \"maas-api-key-cleanup-29722485-97sgd\" (UID: \"97633ee4-b353-43f6-ad58-8b726df28551\") " pod="opendatahub/maas-api-key-cleanup-29722485-97sgd"
Jul 06 14:45:00.368547 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:00.368512    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d4nlg\" (UniqueName: \"kubernetes.io/projected/97633ee4-b353-43f6-ad58-8b726df28551-kube-api-access-d4nlg\") pod \"maas-api-key-cleanup-29722485-97sgd\" (UID: \"97633ee4-b353-43f6-ad58-8b726df28551\") " pod="opendatahub/maas-api-key-cleanup-29722485-97sgd"
Jul 06 14:45:00.466272 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:00.466228    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29722485-97sgd"
Jul 06 14:45:00.619449 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:00.619411    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29722485-97sgd"]
Jul 06 14:45:00.622906 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:45:00.622875    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97633ee4_b353_43f6_ad58_8b726df28551.slice/crio-6ab87f0fb7eb0e9312791881dba383602c23c6837fd3cb231ac42ffde9505747 WatchSource:0}: Error finding container 6ab87f0fb7eb0e9312791881dba383602c23c6837fd3cb231ac42ffde9505747: Status 404 returned error can't find the container with id 6ab87f0fb7eb0e9312791881dba383602c23c6837fd3cb231ac42ffde9505747
Jul 06 14:45:01.271528 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:01.271489    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29722485-97sgd" event={"ID":"97633ee4-b353-43f6-ad58-8b726df28551","Type":"ContainerStarted","Data":"9fda21f404bacd75ee2e7f5dc8ae8d412fdece74e700da59968ca584e1ff0e0e"}
Jul 06 14:45:01.271978 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:01.271556    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29722485-97sgd" event={"ID":"97633ee4-b353-43f6-ad58-8b726df28551","Type":"ContainerStarted","Data":"6ab87f0fb7eb0e9312791881dba383602c23c6837fd3cb231ac42ffde9505747"}
Jul 06 14:45:01.290319 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:01.290262    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-key-cleanup-29722485-97sgd" podStartSLOduration=1.290243985 podStartE2EDuration="1.290243985s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:45:01.289105563 +0000 UTC m=+1523.596397646" watchObservedRunningTime="2026-07-06 14:45:01.290243985 +0000 UTC m=+1523.597536096"
Jul 06 14:45:02.278059 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:02.278020    2559 generic.go:356] "Generic (PLEG): container finished" podID="97633ee4-b353-43f6-ad58-8b726df28551" containerID="9fda21f404bacd75ee2e7f5dc8ae8d412fdece74e700da59968ca584e1ff0e0e" exitCode=0
Jul 06 14:45:02.278483 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:02.278106    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29722485-97sgd" event={"ID":"97633ee4-b353-43f6-ad58-8b726df28551","Type":"ContainerDied","Data":"9fda21f404bacd75ee2e7f5dc8ae8d412fdece74e700da59968ca584e1ff0e0e"}
Jul 06 14:45:03.421824 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:03.421798    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29722485-97sgd"
Jul 06 14:45:03.483002 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:03.482952    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d4nlg\" (UniqueName: \"kubernetes.io/projected/97633ee4-b353-43f6-ad58-8b726df28551-kube-api-access-d4nlg\") pod \"97633ee4-b353-43f6-ad58-8b726df28551\" (UID: \"97633ee4-b353-43f6-ad58-8b726df28551\") "
Jul 06 14:45:03.485140 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:03.485111    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/97633ee4-b353-43f6-ad58-8b726df28551-kube-api-access-d4nlg" (OuterVolumeSpecName: "kube-api-access-d4nlg") pod "97633ee4-b353-43f6-ad58-8b726df28551" (UID: "97633ee4-b353-43f6-ad58-8b726df28551"). InnerVolumeSpecName "kube-api-access-d4nlg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 14:45:03.584359 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:03.584272    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d4nlg\" (UniqueName: \"kubernetes.io/projected/97633ee4-b353-43f6-ad58-8b726df28551-kube-api-access-d4nlg\") on node \"ip-10-0-133-112.ec2.internal\" DevicePath \"\""
Jul 06 14:45:04.289859 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:04.289826    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29722485-97sgd"
Jul 06 14:45:04.289859 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:04.289856    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29722485-97sgd" event={"ID":"97633ee4-b353-43f6-ad58-8b726df28551","Type":"ContainerDied","Data":"6ab87f0fb7eb0e9312791881dba383602c23c6837fd3cb231ac42ffde9505747"}
Jul 06 14:45:04.290082 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:04.289898    2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6ab87f0fb7eb0e9312791881dba383602c23c6837fd3cb231ac42ffde9505747"
Jul 06 14:45:41.428558 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:41.428519    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-ed2d5841-openshift-default-7d7cdfbcb9-j2vkx"]
Jul 06 14:45:41.429213 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:41.428985    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="97633ee4-b353-43f6-ad58-8b726df28551" containerName="cleanup"
Jul 06 14:45:41.429213 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:41.429000    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="97633ee4-b353-43f6-ad58-8b726df28551" containerName="cleanup"
Jul 06 14:45:41.429213 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:41.429059    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="97633ee4-b353-43f6-ad58-8b726df28551" containerName="cleanup"
Jul 06 14:45:41.432740 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:41.432708    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-ed2d5841-openshift-default-7d7cdfbcb9-j2vkx"
Jul 06 14:45:41.443285 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:41.442262    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-ed2d5841-openshift-default-dockercfg-bp74g\""
Jul 06 14:45:41.451366 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:41.451318    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-ed2d5841-openshift-default-7d7cdfbcb9-j2vkx"]
Jul 06 14:45:41.544500 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:41.544458    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b1f242fe-9c00-4882-aec4-f4dd69f3c381-workload-certs\") pod \"e2e-ait-ed2d5841-openshift-default-7d7cdfbcb9-j2vkx\" (UID: \"b1f242fe-9c00-4882-aec4-f4dd69f3c381\") " pod="openshift-ingress/e2e-ait-ed2d5841-openshift-default-7d7cdfbcb9-j2vkx"
Jul 06 14:45:41.544719 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:41.544566    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b1f242fe-9c00-4882-aec4-f4dd69f3c381-istio-data\") pod \"e2e-ait-ed2d5841-openshift-default-7d7cdfbcb9-j2vkx\" (UID: \"b1f242fe-9c00-4882-aec4-f4dd69f3c381\") " pod="openshift-ingress/e2e-ait-ed2d5841-openshift-default-7d7cdfbcb9-j2vkx"
Jul 06 14:45:41.544719 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:41.544680    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b1f242fe-9c00-4882-aec4-f4dd69f3c381-istiod-ca-cert\") pod \"e2e-ait-ed2d5841-openshift-default-7d7cdfbcb9-j2vkx\" (UID: \"b1f242fe-9c00-4882-aec4-f4dd69f3c381\") " pod="openshift-ingress/e2e-ait-ed2d5841-openshift-default-7d7cdfbcb9-j2vkx"
Jul 06 14:45:41.544842 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:41.544724    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nw6ph\" (UniqueName: \"kubernetes.io/projected/b1f242fe-9c00-4882-aec4-f4dd69f3c381-kube-api-access-nw6ph\") pod \"e2e-ait-ed2d5841-openshift-default-7d7cdfbcb9-j2vkx\" (UID: \"b1f242fe-9c00-4882-aec4-f4dd69f3c381\") " pod="openshift-ingress/e2e-ait-ed2d5841-openshift-default-7d7cdfbcb9-j2vkx"
Jul 06 14:45:41.544842 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:41.544752    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b1f242fe-9c00-4882-aec4-f4dd69f3c381-istio-envoy\") pod \"e2e-ait-ed2d5841-openshift-default-7d7cdfbcb9-j2vkx\" (UID: \"b1f242fe-9c00-4882-aec4-f4dd69f3c381\") " pod="openshift-ingress/e2e-ait-ed2d5841-openshift-default-7d7cdfbcb9-j2vkx"
Jul 06 14:45:41.544842 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:41.544776    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b1f242fe-9c00-4882-aec4-f4dd69f3c381-istio-token\") pod \"e2e-ait-ed2d5841-openshift-default-7d7cdfbcb9-j2vkx\" (UID: \"b1f242fe-9c00-4882-aec4-f4dd69f3c381\") " pod="openshift-ingress/e2e-ait-ed2d5841-openshift-default-7d7cdfbcb9-j2vkx"
Jul 06 14:45:41.544992 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:41.544841    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1f242fe-9c00-4882-aec4-f4dd69f3c381-credential-socket\") pod \"e2e-ait-ed2d5841-openshift-default-7d7cdfbcb9-j2vkx\" (UID: \"b1f242fe-9c00-4882-aec4-f4dd69f3c381\") " pod="openshift-ingress/e2e-ait-ed2d5841-openshift-default-7d7cdfbcb9-j2vkx"
Jul 06 14:45:41.544992 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:41.544866    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b1f242fe-9c00-4882-aec4-f4dd69f3c381-istio-podinfo\") pod \"e2e-ait-ed2d5841-openshift-default-7d7cdfbcb9-j2vkx\" (UID: \"b1f242fe-9c00-4882-aec4-f4dd69f3c381\") " pod="openshift-ingress/e2e-ait-ed2d5841-openshift-default-7d7cdfbcb9-j2vkx"
Jul 06 14:45:41.544992 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:41.544906    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1f242fe-9c00-4882-aec4-f4dd69f3c381-workload-socket\") pod \"e2e-ait-ed2d5841-openshift-default-7d7cdfbcb9-j2vkx\" (UID: \"b1f242fe-9c00-4882-aec4-f4dd69f3c381\") " pod="openshift-ingress/e2e-ait-ed2d5841-openshift-default-7d7cdfbcb9-j2vkx"
Jul 06 14:45:41.646081 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:41.646041    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b1f242fe-9c00-4882-aec4-f4dd69f3c381-workload-certs\") pod \"e2e-ait-ed2d5841-openshift-default-7d7cdfbcb9-j2vkx\" (UID: \"b1f242fe-9c00-4882-aec4-f4dd69f3c381\") " pod="openshift-ingress/e2e-ait-ed2d5841-openshift-default-7d7cdfbcb9-j2vkx"
Jul 06 14:45:41.646303 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:41.646107    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b1f242fe-9c00-4882-aec4-f4dd69f3c381-istio-data\") pod \"e2e-ait-ed2d5841-openshift-default-7d7cdfbcb9-j2vkx\" (UID: \"b1f242fe-9c00-4882-aec4-f4dd69f3c381\") " pod="openshift-ingress/e2e-ait-ed2d5841-openshift-default-7d7cdfbcb9-j2vkx"
Jul 06 14:45:41.646303 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:41.646167    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b1f242fe-9c00-4882-aec4-f4dd69f3c381-istiod-ca-cert\") pod \"e2e-ait-ed2d5841-openshift-default-7d7cdfbcb9-j2vkx\" (UID: \"b1f242fe-9c00-4882-aec4-f4dd69f3c381\") " pod="openshift-ingress/e2e-ait-ed2d5841-openshift-default-7d7cdfbcb9-j2vkx"
Jul 06 14:45:41.646303 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:41.646207    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nw6ph\" (UniqueName: \"kubernetes.io/projected/b1f242fe-9c00-4882-aec4-f4dd69f3c381-kube-api-access-nw6ph\") pod \"e2e-ait-ed2d5841-openshift-default-7d7cdfbcb9-j2vkx\" (UID: \"b1f242fe-9c00-4882-aec4-f4dd69f3c381\") " pod="openshift-ingress/e2e-ait-ed2d5841-openshift-default-7d7cdfbcb9-j2vkx"
Jul 06 14:45:41.646303 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:41.646234    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b1f242fe-9c00-4882-aec4-f4dd69f3c381-istio-envoy\") pod \"e2e-ait-ed2d5841-openshift-default-7d7cdfbcb9-j2vkx\" (UID: \"b1f242fe-9c00-4882-aec4-f4dd69f3c381\") " pod="openshift-ingress/e2e-ait-ed2d5841-openshift-default-7d7cdfbcb9-j2vkx"
Jul 06 14:45:41.646303 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:41.646259    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b1f242fe-9c00-4882-aec4-f4dd69f3c381-istio-token\") pod \"e2e-ait-ed2d5841-openshift-default-7d7cdfbcb9-j2vkx\" (UID: \"b1f242fe-9c00-4882-aec4-f4dd69f3c381\") " pod="openshift-ingress/e2e-ait-ed2d5841-openshift-default-7d7cdfbcb9-j2vkx"
Jul 06 14:45:41.646303 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:41.646297    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1f242fe-9c00-4882-aec4-f4dd69f3c381-credential-socket\") pod \"e2e-ait-ed2d5841-openshift-default-7d7cdfbcb9-j2vkx\" (UID: \"b1f242fe-9c00-4882-aec4-f4dd69f3c381\") " pod="openshift-ingress/e2e-ait-ed2d5841-openshift-default-7d7cdfbcb9-j2vkx"
Jul 06 14:45:41.646636 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:41.646321    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b1f242fe-9c00-4882-aec4-f4dd69f3c381-istio-podinfo\") pod \"e2e-ait-ed2d5841-openshift-default-7d7cdfbcb9-j2vkx\" (UID: \"b1f242fe-9c00-4882-aec4-f4dd69f3c381\") " pod="openshift-ingress/e2e-ait-ed2d5841-openshift-default-7d7cdfbcb9-j2vkx"
Jul 06 14:45:41.646636 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:41.646361    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1f242fe-9c00-4882-aec4-f4dd69f3c381-workload-socket\") pod \"e2e-ait-ed2d5841-openshift-default-7d7cdfbcb9-j2vkx\" (UID: \"b1f242fe-9c00-4882-aec4-f4dd69f3c381\") " pod="openshift-ingress/e2e-ait-ed2d5841-openshift-default-7d7cdfbcb9-j2vkx"
Jul 06 14:45:41.646636 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:41.646541    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b1f242fe-9c00-4882-aec4-f4dd69f3c381-workload-certs\") pod \"e2e-ait-ed2d5841-openshift-default-7d7cdfbcb9-j2vkx\" (UID: \"b1f242fe-9c00-4882-aec4-f4dd69f3c381\") " pod="openshift-ingress/e2e-ait-ed2d5841-openshift-default-7d7cdfbcb9-j2vkx"
Jul 06 14:45:41.646800 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:41.646649    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b1f242fe-9c00-4882-aec4-f4dd69f3c381-istio-data\") pod \"e2e-ait-ed2d5841-openshift-default-7d7cdfbcb9-j2vkx\" (UID: \"b1f242fe-9c00-4882-aec4-f4dd69f3c381\") " pod="openshift-ingress/e2e-ait-ed2d5841-openshift-default-7d7cdfbcb9-j2vkx"
Jul 06 14:45:41.646800 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:41.646781    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1f242fe-9c00-4882-aec4-f4dd69f3c381-credential-socket\") pod \"e2e-ait-ed2d5841-openshift-default-7d7cdfbcb9-j2vkx\" (UID: \"b1f242fe-9c00-4882-aec4-f4dd69f3c381\") " pod="openshift-ingress/e2e-ait-ed2d5841-openshift-default-7d7cdfbcb9-j2vkx"
Jul 06 14:45:41.646897 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:41.646834    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1f242fe-9c00-4882-aec4-f4dd69f3c381-workload-socket\") pod \"e2e-ait-ed2d5841-openshift-default-7d7cdfbcb9-j2vkx\" (UID: \"b1f242fe-9c00-4882-aec4-f4dd69f3c381\") " pod="openshift-ingress/e2e-ait-ed2d5841-openshift-default-7d7cdfbcb9-j2vkx"
Jul 06 14:45:41.646986 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:41.646954    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b1f242fe-9c00-4882-aec4-f4dd69f3c381-istiod-ca-cert\") pod \"e2e-ait-ed2d5841-openshift-default-7d7cdfbcb9-j2vkx\" (UID: \"b1f242fe-9c00-4882-aec4-f4dd69f3c381\") " pod="openshift-ingress/e2e-ait-ed2d5841-openshift-default-7d7cdfbcb9-j2vkx"
Jul 06 14:45:41.648545 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:41.648519    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b1f242fe-9c00-4882-aec4-f4dd69f3c381-istio-envoy\") pod \"e2e-ait-ed2d5841-openshift-default-7d7cdfbcb9-j2vkx\" (UID: \"b1f242fe-9c00-4882-aec4-f4dd69f3c381\") " pod="openshift-ingress/e2e-ait-ed2d5841-openshift-default-7d7cdfbcb9-j2vkx"
Jul 06 14:45:41.648956 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:41.648940    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b1f242fe-9c00-4882-aec4-f4dd69f3c381-istio-podinfo\") pod \"e2e-ait-ed2d5841-openshift-default-7d7cdfbcb9-j2vkx\" (UID: \"b1f242fe-9c00-4882-aec4-f4dd69f3c381\") " pod="openshift-ingress/e2e-ait-ed2d5841-openshift-default-7d7cdfbcb9-j2vkx"
Jul 06 14:45:41.680047 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:41.679965    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b1f242fe-9c00-4882-aec4-f4dd69f3c381-istio-token\") pod \"e2e-ait-ed2d5841-openshift-default-7d7cdfbcb9-j2vkx\" (UID: \"b1f242fe-9c00-4882-aec4-f4dd69f3c381\") " pod="openshift-ingress/e2e-ait-ed2d5841-openshift-default-7d7cdfbcb9-j2vkx"
Jul 06 14:45:41.680047 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:41.680025    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nw6ph\" (UniqueName: \"kubernetes.io/projected/b1f242fe-9c00-4882-aec4-f4dd69f3c381-kube-api-access-nw6ph\") pod \"e2e-ait-ed2d5841-openshift-default-7d7cdfbcb9-j2vkx\" (UID: \"b1f242fe-9c00-4882-aec4-f4dd69f3c381\") " pod="openshift-ingress/e2e-ait-ed2d5841-openshift-default-7d7cdfbcb9-j2vkx"
Jul 06 14:45:41.754039 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:41.753995    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-ed2d5841-openshift-default-7d7cdfbcb9-j2vkx"
Jul 06 14:45:41.900803 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:41.900693    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-ed2d5841-openshift-default-7d7cdfbcb9-j2vkx"]
Jul 06 14:45:41.903658 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:45:41.903631    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1f242fe_9c00_4882_aec4_f4dd69f3c381.slice/crio-cd1518ec4ef54417eb84466b15133850057ffe146ace41be09a6725464f872a6 WatchSource:0}: Error finding container cd1518ec4ef54417eb84466b15133850057ffe146ace41be09a6725464f872a6: Status 404 returned error can't find the container with id cd1518ec4ef54417eb84466b15133850057ffe146ace41be09a6725464f872a6
Jul 06 14:45:41.905817 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:41.905772    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 06 14:45:41.905931 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:41.905852    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 06 14:45:41.905931 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:41.905920    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 06 14:45:42.478278 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:42.478242    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-ed2d5841-openshift-default-7d7cdfbcb9-j2vkx" event={"ID":"b1f242fe-9c00-4882-aec4-f4dd69f3c381","Type":"ContainerStarted","Data":"194c7504db75304850c24c346e44b21a7b2a9583b1af4763f006c2e4b5ad942d"}
Jul 06 14:45:42.478839 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:42.478305    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-ed2d5841-openshift-default-7d7cdfbcb9-j2vkx" event={"ID":"b1f242fe-9c00-4882-aec4-f4dd69f3c381","Type":"ContainerStarted","Data":"cd1518ec4ef54417eb84466b15133850057ffe146ace41be09a6725464f872a6"}
Jul 06 14:45:42.501727 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:42.501668    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-ed2d5841-openshift-default-7d7cdfbcb9-j2vkx" podStartSLOduration=1.5016452500000002 podStartE2EDuration="1.50164525s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:45:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:45:42.500981886 +0000 UTC m=+1564.808273969" watchObservedRunningTime="2026-07-06 14:45:42.50164525 +0000 UTC m=+1564.808937333"
Jul 06 14:45:42.754513 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:42.754411    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-ed2d5841-openshift-default-7d7cdfbcb9-j2vkx"
Jul 06 14:45:42.756005 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:42.755974    2559 patch_prober.go:28] interesting pod/e2e-ait-ed2d5841-openshift-default-7d7cdfbcb9-j2vkx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body=
Jul 06 14:45:42.756149 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:42.756029    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-ed2d5841-openshift-default-7d7cdfbcb9-j2vkx" podUID="b1f242fe-9c00-4882-aec4-f4dd69f3c381" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused"
Jul 06 14:45:42.975405 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:42.975362    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-ait-ed2d5841-766dbd8d5b-b9gjw"]
Jul 06 14:45:42.979051 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:42.979025    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-ed2d5841-766dbd8d5b-b9gjw"
Jul 06 14:45:42.982211 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:42.982185    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-ait-ed2d5841-serving-cert\""
Jul 06 14:45:42.991822 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:42.991792    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-ed2d5841-766dbd8d5b-b9gjw"]
Jul 06 14:45:43.159862 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:43.159765    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ec2e57cd-adba-4e53-aa2d-a42bbfcfd4a5-maas-api-tls\") pod \"maas-api-e2e-ait-ed2d5841-766dbd8d5b-b9gjw\" (UID: \"ec2e57cd-adba-4e53-aa2d-a42bbfcfd4a5\") " pod="opendatahub/maas-api-e2e-ait-ed2d5841-766dbd8d5b-b9gjw"
Jul 06 14:45:43.159862 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:43.159832    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zmpdf\" (UniqueName: \"kubernetes.io/projected/ec2e57cd-adba-4e53-aa2d-a42bbfcfd4a5-kube-api-access-zmpdf\") pod \"maas-api-e2e-ait-ed2d5841-766dbd8d5b-b9gjw\" (UID: \"ec2e57cd-adba-4e53-aa2d-a42bbfcfd4a5\") " pod="opendatahub/maas-api-e2e-ait-ed2d5841-766dbd8d5b-b9gjw"
Jul 06 14:45:43.160060 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:43.159917    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ec2e57cd-adba-4e53-aa2d-a42bbfcfd4a5-openshift-service-ca\") pod \"maas-api-e2e-ait-ed2d5841-766dbd8d5b-b9gjw\" (UID: \"ec2e57cd-adba-4e53-aa2d-a42bbfcfd4a5\") " pod="opendatahub/maas-api-e2e-ait-ed2d5841-766dbd8d5b-b9gjw"
Jul 06 14:45:43.261280 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:43.261238    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ec2e57cd-adba-4e53-aa2d-a42bbfcfd4a5-maas-api-tls\") pod \"maas-api-e2e-ait-ed2d5841-766dbd8d5b-b9gjw\" (UID: \"ec2e57cd-adba-4e53-aa2d-a42bbfcfd4a5\") " pod="opendatahub/maas-api-e2e-ait-ed2d5841-766dbd8d5b-b9gjw"
Jul 06 14:45:43.261492 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:43.261316    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zmpdf\" (UniqueName: \"kubernetes.io/projected/ec2e57cd-adba-4e53-aa2d-a42bbfcfd4a5-kube-api-access-zmpdf\") pod \"maas-api-e2e-ait-ed2d5841-766dbd8d5b-b9gjw\" (UID: \"ec2e57cd-adba-4e53-aa2d-a42bbfcfd4a5\") " pod="opendatahub/maas-api-e2e-ait-ed2d5841-766dbd8d5b-b9gjw"
Jul 06 14:45:43.261492 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:43.261366    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ec2e57cd-adba-4e53-aa2d-a42bbfcfd4a5-openshift-service-ca\") pod \"maas-api-e2e-ait-ed2d5841-766dbd8d5b-b9gjw\" (UID: \"ec2e57cd-adba-4e53-aa2d-a42bbfcfd4a5\") " pod="opendatahub/maas-api-e2e-ait-ed2d5841-766dbd8d5b-b9gjw"
Jul 06 14:45:43.262105 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:43.262084    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ec2e57cd-adba-4e53-aa2d-a42bbfcfd4a5-openshift-service-ca\") pod \"maas-api-e2e-ait-ed2d5841-766dbd8d5b-b9gjw\" (UID: \"ec2e57cd-adba-4e53-aa2d-a42bbfcfd4a5\") " pod="opendatahub/maas-api-e2e-ait-ed2d5841-766dbd8d5b-b9gjw"
Jul 06 14:45:43.263878 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:43.263853    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ec2e57cd-adba-4e53-aa2d-a42bbfcfd4a5-maas-api-tls\") pod \"maas-api-e2e-ait-ed2d5841-766dbd8d5b-b9gjw\" (UID: \"ec2e57cd-adba-4e53-aa2d-a42bbfcfd4a5\") " pod="opendatahub/maas-api-e2e-ait-ed2d5841-766dbd8d5b-b9gjw"
Jul 06 14:45:43.273804 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:43.273778    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zmpdf\" (UniqueName: \"kubernetes.io/projected/ec2e57cd-adba-4e53-aa2d-a42bbfcfd4a5-kube-api-access-zmpdf\") pod \"maas-api-e2e-ait-ed2d5841-766dbd8d5b-b9gjw\" (UID: \"ec2e57cd-adba-4e53-aa2d-a42bbfcfd4a5\") " pod="opendatahub/maas-api-e2e-ait-ed2d5841-766dbd8d5b-b9gjw"
Jul 06 14:45:43.292763 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:43.292723    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-ed2d5841-766dbd8d5b-b9gjw"
Jul 06 14:45:43.456245 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:43.456208    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-ed2d5841-766dbd8d5b-b9gjw"]
Jul 06 14:45:43.459259 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:45:43.459234    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec2e57cd_adba_4e53_aa2d_a42bbfcfd4a5.slice/crio-4e4da0aee2d97977db77f4b57b49c25dd704b2304617072b45ae8f9918ceda75 WatchSource:0}: Error finding container 4e4da0aee2d97977db77f4b57b49c25dd704b2304617072b45ae8f9918ceda75: Status 404 returned error can't find the container with id 4e4da0aee2d97977db77f4b57b49c25dd704b2304617072b45ae8f9918ceda75
Jul 06 14:45:43.460727 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:43.460712    2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 06 14:45:43.485467 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:43.485433    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-ed2d5841-766dbd8d5b-b9gjw" event={"ID":"ec2e57cd-adba-4e53-aa2d-a42bbfcfd4a5","Type":"ContainerStarted","Data":"4e4da0aee2d97977db77f4b57b49c25dd704b2304617072b45ae8f9918ceda75"}
Jul 06 14:45:43.759094 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:43.759016    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-ed2d5841-openshift-default-7d7cdfbcb9-j2vkx"
Jul 06 14:45:44.492412 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:44.492371    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-ed2d5841-766dbd8d5b-b9gjw" event={"ID":"ec2e57cd-adba-4e53-aa2d-a42bbfcfd4a5","Type":"ContainerStarted","Data":"53c26fa57bd5951e6f454f4067b4bd3cca41d6e85a979f7ec8edeb2fbb278711"}
Jul 06 14:45:44.492935 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:44.492622    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-ed2d5841-openshift-default-7d7cdfbcb9-j2vkx"
Jul 06 14:45:44.492935 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:44.492656    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-ait-ed2d5841-766dbd8d5b-b9gjw"
Jul 06 14:45:44.493728 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:44.493707    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-ed2d5841-openshift-default-7d7cdfbcb9-j2vkx"
Jul 06 14:45:44.567958 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:44.567890    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-ait-ed2d5841-766dbd8d5b-b9gjw" podStartSLOduration=2.189409917 podStartE2EDuration="2.567867637s" totalImagesPullingTime="378.45772ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:45:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:45:44.523257443 +0000 UTC m=+1566.830549534" watchObservedRunningTime="2026-07-06 14:45:44.567867637 +0000 UTC m=+1566.875159719"
Jul 06 14:45:47.920456 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:47.920419    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-ed2d5841-openshift-default-7d7cdfbcb9-j2vkx"]
Jul 06 14:45:47.920902 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:47.920640    2559 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-ed2d5841-openshift-default-7d7cdfbcb9-j2vkx" podUID="b1f242fe-9c00-4882-aec4-f4dd69f3c381" containerName="istio-proxy" containerID="cri-o://194c7504db75304850c24c346e44b21a7b2a9583b1af4763f006c2e4b5ad942d" gracePeriod=30
Jul 06 14:45:48.249757 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:48.249674    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-ed2d5841-766dbd8d5b-b9gjw"]
Jul 06 14:45:48.249923 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:48.249876    2559 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-ait-ed2d5841-766dbd8d5b-b9gjw" podUID="ec2e57cd-adba-4e53-aa2d-a42bbfcfd4a5" containerName="maas-api" containerID="cri-o://53c26fa57bd5951e6f454f4067b4bd3cca41d6e85a979f7ec8edeb2fbb278711" gracePeriod=30
Jul 06 14:45:48.255149 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:48.255124    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-ait-ed2d5841-766dbd8d5b-b9gjw"
Jul 06 14:45:48.516029 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:48.515945    2559 generic.go:356] "Generic (PLEG): container finished" podID="ec2e57cd-adba-4e53-aa2d-a42bbfcfd4a5" containerID="53c26fa57bd5951e6f454f4067b4bd3cca41d6e85a979f7ec8edeb2fbb278711" exitCode=0
Jul 06 14:45:48.516029 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:48.516000    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-ed2d5841-766dbd8d5b-b9gjw" event={"ID":"ec2e57cd-adba-4e53-aa2d-a42bbfcfd4a5","Type":"ContainerDied","Data":"53c26fa57bd5951e6f454f4067b4bd3cca41d6e85a979f7ec8edeb2fbb278711"}
Jul 06 14:45:48.540672 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:48.540643    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-ed2d5841-766dbd8d5b-b9gjw"
Jul 06 14:45:48.610460 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:48.610415    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zmpdf\" (UniqueName: \"kubernetes.io/projected/ec2e57cd-adba-4e53-aa2d-a42bbfcfd4a5-kube-api-access-zmpdf\") pod \"ec2e57cd-adba-4e53-aa2d-a42bbfcfd4a5\" (UID: \"ec2e57cd-adba-4e53-aa2d-a42bbfcfd4a5\") "
Jul 06 14:45:48.610664 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:48.610595    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ec2e57cd-adba-4e53-aa2d-a42bbfcfd4a5-maas-api-tls\") pod \"ec2e57cd-adba-4e53-aa2d-a42bbfcfd4a5\" (UID: \"ec2e57cd-adba-4e53-aa2d-a42bbfcfd4a5\") "
Jul 06 14:45:48.610664 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:48.610625    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ec2e57cd-adba-4e53-aa2d-a42bbfcfd4a5-openshift-service-ca\") pod \"ec2e57cd-adba-4e53-aa2d-a42bbfcfd4a5\" (UID: \"ec2e57cd-adba-4e53-aa2d-a42bbfcfd4a5\") "
Jul 06 14:45:48.611052 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:48.611004    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ec2e57cd-adba-4e53-aa2d-a42bbfcfd4a5-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ec2e57cd-adba-4e53-aa2d-a42bbfcfd4a5" (UID: "ec2e57cd-adba-4e53-aa2d-a42bbfcfd4a5"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 14:45:48.612654 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:48.612620    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ec2e57cd-adba-4e53-aa2d-a42bbfcfd4a5-kube-api-access-zmpdf" (OuterVolumeSpecName: "kube-api-access-zmpdf") pod "ec2e57cd-adba-4e53-aa2d-a42bbfcfd4a5" (UID: "ec2e57cd-adba-4e53-aa2d-a42bbfcfd4a5"). InnerVolumeSpecName "kube-api-access-zmpdf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 14:45:48.612769 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:48.612747    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ec2e57cd-adba-4e53-aa2d-a42bbfcfd4a5-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ec2e57cd-adba-4e53-aa2d-a42bbfcfd4a5" (UID: "ec2e57cd-adba-4e53-aa2d-a42bbfcfd4a5"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 14:45:48.711520 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:48.711472    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zmpdf\" (UniqueName: \"kubernetes.io/projected/ec2e57cd-adba-4e53-aa2d-a42bbfcfd4a5-kube-api-access-zmpdf\") on node \"ip-10-0-133-112.ec2.internal\" DevicePath \"\""
Jul 06 14:45:48.711520 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:48.711516    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ec2e57cd-adba-4e53-aa2d-a42bbfcfd4a5-maas-api-tls\") on node \"ip-10-0-133-112.ec2.internal\" DevicePath \"\""
Jul 06 14:45:48.711771 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:48.711534    2559 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ec2e57cd-adba-4e53-aa2d-a42bbfcfd4a5-openshift-service-ca\") on node \"ip-10-0-133-112.ec2.internal\" DevicePath \"\""
Jul 06 14:45:49.523278 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:49.523247    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-ed2d5841-766dbd8d5b-b9gjw"
Jul 06 14:45:49.523812 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:49.523279    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-ed2d5841-766dbd8d5b-b9gjw" event={"ID":"ec2e57cd-adba-4e53-aa2d-a42bbfcfd4a5","Type":"ContainerDied","Data":"4e4da0aee2d97977db77f4b57b49c25dd704b2304617072b45ae8f9918ceda75"}
Jul 06 14:45:49.523812 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:49.523338    2559 scope.go:117] "RemoveContainer" containerID="53c26fa57bd5951e6f454f4067b4bd3cca41d6e85a979f7ec8edeb2fbb278711"
Jul 06 14:45:49.547300 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:49.547264    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-ed2d5841-766dbd8d5b-b9gjw"]
Jul 06 14:45:49.552692 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:49.552650    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-ait-ed2d5841-766dbd8d5b-b9gjw"]
Jul 06 14:45:50.226979 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:50.226932    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ec2e57cd-adba-4e53-aa2d-a42bbfcfd4a5" path="/var/lib/kubelet/pods/ec2e57cd-adba-4e53-aa2d-a42bbfcfd4a5/volumes"
Jul 06 14:45:54.079536 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:54.079509    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-ed2d5841-openshift-default-7d7cdfbcb9-j2vkx"
Jul 06 14:45:54.155686 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:54.155603    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b1f242fe-9c00-4882-aec4-f4dd69f3c381-istio-data\") pod \"b1f242fe-9c00-4882-aec4-f4dd69f3c381\" (UID: \"b1f242fe-9c00-4882-aec4-f4dd69f3c381\") "
Jul 06 14:45:54.155686 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:54.155648    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1f242fe-9c00-4882-aec4-f4dd69f3c381-workload-socket\") pod \"b1f242fe-9c00-4882-aec4-f4dd69f3c381\" (UID: \"b1f242fe-9c00-4882-aec4-f4dd69f3c381\") "
Jul 06 14:45:54.155887 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:54.155708    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b1f242fe-9c00-4882-aec4-f4dd69f3c381-istiod-ca-cert\") pod \"b1f242fe-9c00-4882-aec4-f4dd69f3c381\" (UID: \"b1f242fe-9c00-4882-aec4-f4dd69f3c381\") "
Jul 06 14:45:54.155887 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:54.155744    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b1f242fe-9c00-4882-aec4-f4dd69f3c381-istio-envoy\") pod \"b1f242fe-9c00-4882-aec4-f4dd69f3c381\" (UID: \"b1f242fe-9c00-4882-aec4-f4dd69f3c381\") "
Jul 06 14:45:54.155887 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:54.155782    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nw6ph\" (UniqueName: \"kubernetes.io/projected/b1f242fe-9c00-4882-aec4-f4dd69f3c381-kube-api-access-nw6ph\") pod \"b1f242fe-9c00-4882-aec4-f4dd69f3c381\" (UID: \"b1f242fe-9c00-4882-aec4-f4dd69f3c381\") "
Jul 06 14:45:54.155887 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:54.155812    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b1f242fe-9c00-4882-aec4-f4dd69f3c381-workload-certs\") pod \"b1f242fe-9c00-4882-aec4-f4dd69f3c381\" (UID: \"b1f242fe-9c00-4882-aec4-f4dd69f3c381\") "
Jul 06 14:45:54.156095 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:54.155898    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1f242fe-9c00-4882-aec4-f4dd69f3c381-credential-socket\") pod \"b1f242fe-9c00-4882-aec4-f4dd69f3c381\" (UID: \"b1f242fe-9c00-4882-aec4-f4dd69f3c381\") "
Jul 06 14:45:54.156095 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:54.155917    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b1f242fe-9c00-4882-aec4-f4dd69f3c381-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b1f242fe-9c00-4882-aec4-f4dd69f3c381" (UID: "b1f242fe-9c00-4882-aec4-f4dd69f3c381"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 14:45:54.156095 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:54.155949    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b1f242fe-9c00-4882-aec4-f4dd69f3c381-istio-token\") pod \"b1f242fe-9c00-4882-aec4-f4dd69f3c381\" (UID: \"b1f242fe-9c00-4882-aec4-f4dd69f3c381\") "
Jul 06 14:45:54.156095 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:54.155977    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b1f242fe-9c00-4882-aec4-f4dd69f3c381-istio-podinfo\") pod \"b1f242fe-9c00-4882-aec4-f4dd69f3c381\" (UID: \"b1f242fe-9c00-4882-aec4-f4dd69f3c381\") "
Jul 06 14:45:54.156305 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:54.156150    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b1f242fe-9c00-4882-aec4-f4dd69f3c381-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b1f242fe-9c00-4882-aec4-f4dd69f3c381" (UID: "b1f242fe-9c00-4882-aec4-f4dd69f3c381"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 14:45:54.156305 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:54.156171    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b1f242fe-9c00-4882-aec4-f4dd69f3c381-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b1f242fe-9c00-4882-aec4-f4dd69f3c381" (UID: "b1f242fe-9c00-4882-aec4-f4dd69f3c381"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 14:45:54.156305 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:54.156175    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b1f242fe-9c00-4882-aec4-f4dd69f3c381-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b1f242fe-9c00-4882-aec4-f4dd69f3c381" (UID: "b1f242fe-9c00-4882-aec4-f4dd69f3c381"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 14:45:54.156305 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:54.156285    2559 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1f242fe-9c00-4882-aec4-f4dd69f3c381-workload-socket\") on node \"ip-10-0-133-112.ec2.internal\" DevicePath \"\""
Jul 06 14:45:54.156305 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:54.156300    2559 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b1f242fe-9c00-4882-aec4-f4dd69f3c381-istiod-ca-cert\") on node \"ip-10-0-133-112.ec2.internal\" DevicePath \"\""
Jul 06 14:45:54.156560 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:54.156315    2559 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b1f242fe-9c00-4882-aec4-f4dd69f3c381-workload-certs\") on node \"ip-10-0-133-112.ec2.internal\" DevicePath \"\""
Jul 06 14:45:54.156560 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:54.156323    2559 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1f242fe-9c00-4882-aec4-f4dd69f3c381-credential-socket\") on node \"ip-10-0-133-112.ec2.internal\" DevicePath \"\""
Jul 06 14:45:54.156560 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:54.156477    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b1f242fe-9c00-4882-aec4-f4dd69f3c381-istio-data" (OuterVolumeSpecName: "istio-data") pod "b1f242fe-9c00-4882-aec4-f4dd69f3c381" (UID: "b1f242fe-9c00-4882-aec4-f4dd69f3c381"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 14:45:54.158131 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:54.158103    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b1f242fe-9c00-4882-aec4-f4dd69f3c381-istio-token" (OuterVolumeSpecName: "istio-token") pod "b1f242fe-9c00-4882-aec4-f4dd69f3c381" (UID: "b1f242fe-9c00-4882-aec4-f4dd69f3c381"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 14:45:54.158255 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:54.158150    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b1f242fe-9c00-4882-aec4-f4dd69f3c381-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b1f242fe-9c00-4882-aec4-f4dd69f3c381" (UID: "b1f242fe-9c00-4882-aec4-f4dd69f3c381"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 14:45:54.158255 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:54.158186    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b1f242fe-9c00-4882-aec4-f4dd69f3c381-kube-api-access-nw6ph" (OuterVolumeSpecName: "kube-api-access-nw6ph") pod "b1f242fe-9c00-4882-aec4-f4dd69f3c381" (UID: "b1f242fe-9c00-4882-aec4-f4dd69f3c381"). InnerVolumeSpecName "kube-api-access-nw6ph". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 14:45:54.158255 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:54.158221    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b1f242fe-9c00-4882-aec4-f4dd69f3c381-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b1f242fe-9c00-4882-aec4-f4dd69f3c381" (UID: "b1f242fe-9c00-4882-aec4-f4dd69f3c381"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 06 14:45:54.256771 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:54.256744    2559 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b1f242fe-9c00-4882-aec4-f4dd69f3c381-istio-token\") on node \"ip-10-0-133-112.ec2.internal\" DevicePath \"\""
Jul 06 14:45:54.256771 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:54.256769    2559 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b1f242fe-9c00-4882-aec4-f4dd69f3c381-istio-podinfo\") on node \"ip-10-0-133-112.ec2.internal\" DevicePath \"\""
Jul 06 14:45:54.256771 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:54.256778    2559 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b1f242fe-9c00-4882-aec4-f4dd69f3c381-istio-data\") on node \"ip-10-0-133-112.ec2.internal\" DevicePath \"\""
Jul 06 14:45:54.256982 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:54.256786    2559 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b1f242fe-9c00-4882-aec4-f4dd69f3c381-istio-envoy\") on node \"ip-10-0-133-112.ec2.internal\" DevicePath \"\""
Jul 06 14:45:54.256982 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:54.256795    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nw6ph\" (UniqueName: \"kubernetes.io/projected/b1f242fe-9c00-4882-aec4-f4dd69f3c381-kube-api-access-nw6ph\") on node \"ip-10-0-133-112.ec2.internal\" DevicePath \"\""
Jul 06 14:45:54.550645 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:54.550606    2559 generic.go:356] "Generic (PLEG): container finished" podID="b1f242fe-9c00-4882-aec4-f4dd69f3c381" containerID="194c7504db75304850c24c346e44b21a7b2a9583b1af4763f006c2e4b5ad942d" exitCode=0
Jul 06 14:45:54.550827 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:54.550673    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-ed2d5841-openshift-default-7d7cdfbcb9-j2vkx"
Jul 06 14:45:54.550827 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:54.550697    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-ed2d5841-openshift-default-7d7cdfbcb9-j2vkx" event={"ID":"b1f242fe-9c00-4882-aec4-f4dd69f3c381","Type":"ContainerDied","Data":"194c7504db75304850c24c346e44b21a7b2a9583b1af4763f006c2e4b5ad942d"}
Jul 06 14:45:54.550827 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:54.550777    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-ed2d5841-openshift-default-7d7cdfbcb9-j2vkx" event={"ID":"b1f242fe-9c00-4882-aec4-f4dd69f3c381","Type":"ContainerDied","Data":"cd1518ec4ef54417eb84466b15133850057ffe146ace41be09a6725464f872a6"}
Jul 06 14:45:54.550827 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:54.550800    2559 scope.go:117] "RemoveContainer" containerID="194c7504db75304850c24c346e44b21a7b2a9583b1af4763f006c2e4b5ad942d"
Jul 06 14:45:54.559829 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:54.559811    2559 scope.go:117] "RemoveContainer" containerID="194c7504db75304850c24c346e44b21a7b2a9583b1af4763f006c2e4b5ad942d"
Jul 06 14:45:54.560080 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:45:54.560063    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"194c7504db75304850c24c346e44b21a7b2a9583b1af4763f006c2e4b5ad942d\": container with ID starting with 194c7504db75304850c24c346e44b21a7b2a9583b1af4763f006c2e4b5ad942d not found: ID does not exist" containerID="194c7504db75304850c24c346e44b21a7b2a9583b1af4763f006c2e4b5ad942d"
Jul 06 14:45:54.560149 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:54.560088    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"194c7504db75304850c24c346e44b21a7b2a9583b1af4763f006c2e4b5ad942d"} err="failed to get container status \"194c7504db75304850c24c346e44b21a7b2a9583b1af4763f006c2e4b5ad942d\": rpc error: code = NotFound desc = could not find container \"194c7504db75304850c24c346e44b21a7b2a9583b1af4763f006c2e4b5ad942d\": container with ID starting with 194c7504db75304850c24c346e44b21a7b2a9583b1af4763f006c2e4b5ad942d not found: ID does not exist"
Jul 06 14:45:54.577055 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:54.577030    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-ed2d5841-openshift-default-7d7cdfbcb9-j2vkx"]
Jul 06 14:45:54.596124 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:54.596100    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-ed2d5841-openshift-default-7d7cdfbcb9-j2vkx"]
Jul 06 14:45:55.185430 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:55.185399    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-df0d3551-openshift-default-cb545dccf-rwjrm"]
Jul 06 14:45:55.186526 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:55.186498    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ec2e57cd-adba-4e53-aa2d-a42bbfcfd4a5" containerName="maas-api"
Jul 06 14:45:55.186648 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:55.186530    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="ec2e57cd-adba-4e53-aa2d-a42bbfcfd4a5" containerName="maas-api"
Jul 06 14:45:55.186648 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:55.186550    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b1f242fe-9c00-4882-aec4-f4dd69f3c381" containerName="istio-proxy"
Jul 06 14:45:55.186648 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:55.186558    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="b1f242fe-9c00-4882-aec4-f4dd69f3c381" containerName="istio-proxy"
Jul 06 14:45:55.186771 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:55.186689    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="b1f242fe-9c00-4882-aec4-f4dd69f3c381" containerName="istio-proxy"
Jul 06 14:45:55.186771 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:55.186700    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="ec2e57cd-adba-4e53-aa2d-a42bbfcfd4a5" containerName="maas-api"
Jul 06 14:45:55.190852 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:55.190829    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-df0d3551-openshift-default-cb545dccf-rwjrm"
Jul 06 14:45:55.193446 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:55.193424    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-df0d3551-openshift-default-dockercfg-jt5h9\""
Jul 06 14:45:55.208525 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:55.208498    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-df0d3551-openshift-default-cb545dccf-rwjrm"]
Jul 06 14:45:55.265476 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:55.265438    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/721e13eb-83b2-4904-a851-3c7c7fc52e84-istio-token\") pod \"e2e-ait-df0d3551-openshift-default-cb545dccf-rwjrm\" (UID: \"721e13eb-83b2-4904-a851-3c7c7fc52e84\") " pod="openshift-ingress/e2e-ait-df0d3551-openshift-default-cb545dccf-rwjrm"
Jul 06 14:45:55.265724 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:55.265607    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/721e13eb-83b2-4904-a851-3c7c7fc52e84-credential-socket\") pod \"e2e-ait-df0d3551-openshift-default-cb545dccf-rwjrm\" (UID: \"721e13eb-83b2-4904-a851-3c7c7fc52e84\") " pod="openshift-ingress/e2e-ait-df0d3551-openshift-default-cb545dccf-rwjrm"
Jul 06 14:45:55.265724 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:55.265640    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sqqch\" (UniqueName: \"kubernetes.io/projected/721e13eb-83b2-4904-a851-3c7c7fc52e84-kube-api-access-sqqch\") pod \"e2e-ait-df0d3551-openshift-default-cb545dccf-rwjrm\" (UID: \"721e13eb-83b2-4904-a851-3c7c7fc52e84\") " pod="openshift-ingress/e2e-ait-df0d3551-openshift-default-cb545dccf-rwjrm"
Jul 06 14:45:55.265724 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:55.265685    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/721e13eb-83b2-4904-a851-3c7c7fc52e84-istio-envoy\") pod \"e2e-ait-df0d3551-openshift-default-cb545dccf-rwjrm\" (UID: \"721e13eb-83b2-4904-a851-3c7c7fc52e84\") " pod="openshift-ingress/e2e-ait-df0d3551-openshift-default-cb545dccf-rwjrm"
Jul 06 14:45:55.265724 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:55.265717    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/721e13eb-83b2-4904-a851-3c7c7fc52e84-workload-certs\") pod \"e2e-ait-df0d3551-openshift-default-cb545dccf-rwjrm\" (UID: \"721e13eb-83b2-4904-a851-3c7c7fc52e84\") " pod="openshift-ingress/e2e-ait-df0d3551-openshift-default-cb545dccf-rwjrm"
Jul 06 14:45:55.265905 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:55.265746    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/721e13eb-83b2-4904-a851-3c7c7fc52e84-istiod-ca-cert\") pod \"e2e-ait-df0d3551-openshift-default-cb545dccf-rwjrm\" (UID: \"721e13eb-83b2-4904-a851-3c7c7fc52e84\") " pod="openshift-ingress/e2e-ait-df0d3551-openshift-default-cb545dccf-rwjrm"
Jul 06 14:45:55.265905 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:55.265783    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/721e13eb-83b2-4904-a851-3c7c7fc52e84-istio-podinfo\") pod \"e2e-ait-df0d3551-openshift-default-cb545dccf-rwjrm\" (UID: \"721e13eb-83b2-4904-a851-3c7c7fc52e84\") " pod="openshift-ingress/e2e-ait-df0d3551-openshift-default-cb545dccf-rwjrm"
Jul 06 14:45:55.265905 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:55.265856    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/721e13eb-83b2-4904-a851-3c7c7fc52e84-istio-data\") pod \"e2e-ait-df0d3551-openshift-default-cb545dccf-rwjrm\" (UID: \"721e13eb-83b2-4904-a851-3c7c7fc52e84\") " pod="openshift-ingress/e2e-ait-df0d3551-openshift-default-cb545dccf-rwjrm"
Jul 06 14:45:55.266077 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:55.266049    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/721e13eb-83b2-4904-a851-3c7c7fc52e84-workload-socket\") pod \"e2e-ait-df0d3551-openshift-default-cb545dccf-rwjrm\" (UID: \"721e13eb-83b2-4904-a851-3c7c7fc52e84\") " pod="openshift-ingress/e2e-ait-df0d3551-openshift-default-cb545dccf-rwjrm"
Jul 06 14:45:55.367363 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:55.367328    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/721e13eb-83b2-4904-a851-3c7c7fc52e84-workload-socket\") pod \"e2e-ait-df0d3551-openshift-default-cb545dccf-rwjrm\" (UID: \"721e13eb-83b2-4904-a851-3c7c7fc52e84\") " pod="openshift-ingress/e2e-ait-df0d3551-openshift-default-cb545dccf-rwjrm"
Jul 06 14:45:55.367363 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:55.367377    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/721e13eb-83b2-4904-a851-3c7c7fc52e84-istio-token\") pod \"e2e-ait-df0d3551-openshift-default-cb545dccf-rwjrm\" (UID: \"721e13eb-83b2-4904-a851-3c7c7fc52e84\") " pod="openshift-ingress/e2e-ait-df0d3551-openshift-default-cb545dccf-rwjrm"
Jul 06 14:45:55.367663 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:55.367412    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/721e13eb-83b2-4904-a851-3c7c7fc52e84-credential-socket\") pod \"e2e-ait-df0d3551-openshift-default-cb545dccf-rwjrm\" (UID: \"721e13eb-83b2-4904-a851-3c7c7fc52e84\") " pod="openshift-ingress/e2e-ait-df0d3551-openshift-default-cb545dccf-rwjrm"
Jul 06 14:45:55.367663 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:55.367428    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sqqch\" (UniqueName: \"kubernetes.io/projected/721e13eb-83b2-4904-a851-3c7c7fc52e84-kube-api-access-sqqch\") pod \"e2e-ait-df0d3551-openshift-default-cb545dccf-rwjrm\" (UID: \"721e13eb-83b2-4904-a851-3c7c7fc52e84\") " pod="openshift-ingress/e2e-ait-df0d3551-openshift-default-cb545dccf-rwjrm"
Jul 06 14:45:55.367663 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:55.367453    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/721e13eb-83b2-4904-a851-3c7c7fc52e84-istio-envoy\") pod \"e2e-ait-df0d3551-openshift-default-cb545dccf-rwjrm\" (UID: \"721e13eb-83b2-4904-a851-3c7c7fc52e84\") " pod="openshift-ingress/e2e-ait-df0d3551-openshift-default-cb545dccf-rwjrm"
Jul 06 14:45:55.367663 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:55.367471    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/721e13eb-83b2-4904-a851-3c7c7fc52e84-workload-certs\") pod \"e2e-ait-df0d3551-openshift-default-cb545dccf-rwjrm\" (UID: \"721e13eb-83b2-4904-a851-3c7c7fc52e84\") " pod="openshift-ingress/e2e-ait-df0d3551-openshift-default-cb545dccf-rwjrm"
Jul 06 14:45:55.367663 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:55.367497    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/721e13eb-83b2-4904-a851-3c7c7fc52e84-istiod-ca-cert\") pod \"e2e-ait-df0d3551-openshift-default-cb545dccf-rwjrm\" (UID: \"721e13eb-83b2-4904-a851-3c7c7fc52e84\") " pod="openshift-ingress/e2e-ait-df0d3551-openshift-default-cb545dccf-rwjrm"
Jul 06 14:45:55.367663 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:55.367524    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/721e13eb-83b2-4904-a851-3c7c7fc52e84-istio-podinfo\") pod \"e2e-ait-df0d3551-openshift-default-cb545dccf-rwjrm\" (UID: \"721e13eb-83b2-4904-a851-3c7c7fc52e84\") " pod="openshift-ingress/e2e-ait-df0d3551-openshift-default-cb545dccf-rwjrm"
Jul 06 14:45:55.367980 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:55.367708    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/721e13eb-83b2-4904-a851-3c7c7fc52e84-istio-data\") pod \"e2e-ait-df0d3551-openshift-default-cb545dccf-rwjrm\" (UID: \"721e13eb-83b2-4904-a851-3c7c7fc52e84\") " pod="openshift-ingress/e2e-ait-df0d3551-openshift-default-cb545dccf-rwjrm"
Jul 06 14:45:55.367980 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:55.367895    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/721e13eb-83b2-4904-a851-3c7c7fc52e84-credential-socket\") pod \"e2e-ait-df0d3551-openshift-default-cb545dccf-rwjrm\" (UID: \"721e13eb-83b2-4904-a851-3c7c7fc52e84\") " pod="openshift-ingress/e2e-ait-df0d3551-openshift-default-cb545dccf-rwjrm"
Jul 06 14:45:55.368075 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:55.367975    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/721e13eb-83b2-4904-a851-3c7c7fc52e84-workload-socket\") pod \"e2e-ait-df0d3551-openshift-default-cb545dccf-rwjrm\" (UID: \"721e13eb-83b2-4904-a851-3c7c7fc52e84\") " pod="openshift-ingress/e2e-ait-df0d3551-openshift-default-cb545dccf-rwjrm"
Jul 06 14:45:55.368075 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:55.368008    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/721e13eb-83b2-4904-a851-3c7c7fc52e84-workload-certs\") pod \"e2e-ait-df0d3551-openshift-default-cb545dccf-rwjrm\" (UID: \"721e13eb-83b2-4904-a851-3c7c7fc52e84\") " pod="openshift-ingress/e2e-ait-df0d3551-openshift-default-cb545dccf-rwjrm"
Jul 06 14:45:55.368075 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:55.368061    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/721e13eb-83b2-4904-a851-3c7c7fc52e84-istio-data\") pod \"e2e-ait-df0d3551-openshift-default-cb545dccf-rwjrm\" (UID: \"721e13eb-83b2-4904-a851-3c7c7fc52e84\") " pod="openshift-ingress/e2e-ait-df0d3551-openshift-default-cb545dccf-rwjrm"
Jul 06 14:45:55.368264 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:55.368246    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/721e13eb-83b2-4904-a851-3c7c7fc52e84-istiod-ca-cert\") pod \"e2e-ait-df0d3551-openshift-default-cb545dccf-rwjrm\" (UID: \"721e13eb-83b2-4904-a851-3c7c7fc52e84\") " pod="openshift-ingress/e2e-ait-df0d3551-openshift-default-cb545dccf-rwjrm"
Jul 06 14:45:55.369852 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:55.369820    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/721e13eb-83b2-4904-a851-3c7c7fc52e84-istio-envoy\") pod \"e2e-ait-df0d3551-openshift-default-cb545dccf-rwjrm\" (UID: \"721e13eb-83b2-4904-a851-3c7c7fc52e84\") " pod="openshift-ingress/e2e-ait-df0d3551-openshift-default-cb545dccf-rwjrm"
Jul 06 14:45:55.370012 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:55.369995    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/721e13eb-83b2-4904-a851-3c7c7fc52e84-istio-podinfo\") pod \"e2e-ait-df0d3551-openshift-default-cb545dccf-rwjrm\" (UID: \"721e13eb-83b2-4904-a851-3c7c7fc52e84\") " pod="openshift-ingress/e2e-ait-df0d3551-openshift-default-cb545dccf-rwjrm"
Jul 06 14:45:55.378860 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:55.378835    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/721e13eb-83b2-4904-a851-3c7c7fc52e84-istio-token\") pod \"e2e-ait-df0d3551-openshift-default-cb545dccf-rwjrm\" (UID: \"721e13eb-83b2-4904-a851-3c7c7fc52e84\") " pod="openshift-ingress/e2e-ait-df0d3551-openshift-default-cb545dccf-rwjrm"
Jul 06 14:45:55.379787 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:55.379763    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sqqch\" (UniqueName: \"kubernetes.io/projected/721e13eb-83b2-4904-a851-3c7c7fc52e84-kube-api-access-sqqch\") pod \"e2e-ait-df0d3551-openshift-default-cb545dccf-rwjrm\" (UID: \"721e13eb-83b2-4904-a851-3c7c7fc52e84\") " pod="openshift-ingress/e2e-ait-df0d3551-openshift-default-cb545dccf-rwjrm"
Jul 06 14:45:55.502080 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:55.501992    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-df0d3551-openshift-default-cb545dccf-rwjrm"
Jul 06 14:45:55.670787 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:45:55.670759    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod721e13eb_83b2_4904_a851_3c7c7fc52e84.slice/crio-5c4030ae32a483ba3a332c1f84e4c8dc7d0edd47173bc4d9fd5958d4d0d3c1c0 WatchSource:0}: Error finding container 5c4030ae32a483ba3a332c1f84e4c8dc7d0edd47173bc4d9fd5958d4d0d3c1c0: Status 404 returned error can't find the container with id 5c4030ae32a483ba3a332c1f84e4c8dc7d0edd47173bc4d9fd5958d4d0d3c1c0
Jul 06 14:45:55.671775 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:55.671748    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-df0d3551-openshift-default-cb545dccf-rwjrm"]
Jul 06 14:45:55.673211 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:55.673180    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 06 14:45:55.673275 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:55.673243    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 06 14:45:55.673275 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:55.673271    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 06 14:45:56.227296 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:56.227254    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b1f242fe-9c00-4882-aec4-f4dd69f3c381" path="/var/lib/kubelet/pods/b1f242fe-9c00-4882-aec4-f4dd69f3c381/volumes"
Jul 06 14:45:56.482329 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:56.482245    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-ait-df0d3551-5fd49d77d7-b777j"]
Jul 06 14:45:56.486532 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:56.486505    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-df0d3551-5fd49d77d7-b777j"
Jul 06 14:45:56.497393 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:56.497371    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-ait-df0d3551-serving-cert\""
Jul 06 14:45:56.509322 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:56.509298    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-df0d3551-5fd49d77d7-b777j"]
Jul 06 14:45:56.565149 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:56.565113    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-df0d3551-openshift-default-cb545dccf-rwjrm" event={"ID":"721e13eb-83b2-4904-a851-3c7c7fc52e84","Type":"ContainerStarted","Data":"30ae12fcafa3905c67cc51dae371eb9299871ee097625c995c8fbbcf98f5d55d"}
Jul 06 14:45:56.565316 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:56.565171    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-df0d3551-openshift-default-cb545dccf-rwjrm" event={"ID":"721e13eb-83b2-4904-a851-3c7c7fc52e84","Type":"ContainerStarted","Data":"5c4030ae32a483ba3a332c1f84e4c8dc7d0edd47173bc4d9fd5958d4d0d3c1c0"}
Jul 06 14:45:56.579978 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:56.579947    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4cda6d94-068b-4253-b579-31a9788e78f0-openshift-service-ca\") pod \"maas-api-e2e-ait-df0d3551-5fd49d77d7-b777j\" (UID: \"4cda6d94-068b-4253-b579-31a9788e78f0\") " pod="opendatahub/maas-api-e2e-ait-df0d3551-5fd49d77d7-b777j"
Jul 06 14:45:56.580172 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:56.579998    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4cda6d94-068b-4253-b579-31a9788e78f0-maas-api-tls\") pod \"maas-api-e2e-ait-df0d3551-5fd49d77d7-b777j\" (UID: \"4cda6d94-068b-4253-b579-31a9788e78f0\") " pod="opendatahub/maas-api-e2e-ait-df0d3551-5fd49d77d7-b777j"
Jul 06 14:45:56.580172 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:56.580069    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gp2zm\" (UniqueName: \"kubernetes.io/projected/4cda6d94-068b-4253-b579-31a9788e78f0-kube-api-access-gp2zm\") pod \"maas-api-e2e-ait-df0d3551-5fd49d77d7-b777j\" (UID: \"4cda6d94-068b-4253-b579-31a9788e78f0\") " pod="opendatahub/maas-api-e2e-ait-df0d3551-5fd49d77d7-b777j"
Jul 06 14:45:56.594080 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:56.594006    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-df0d3551-openshift-default-cb545dccf-rwjrm" podStartSLOduration=1.593982657 podStartE2EDuration="1.593982657s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:45:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:45:56.593067289 +0000 UTC m=+1578.900359384" watchObservedRunningTime="2026-07-06 14:45:56.593982657 +0000 UTC m=+1578.901274740"
Jul 06 14:45:56.680829 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:56.680796    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4cda6d94-068b-4253-b579-31a9788e78f0-openshift-service-ca\") pod \"maas-api-e2e-ait-df0d3551-5fd49d77d7-b777j\" (UID: \"4cda6d94-068b-4253-b579-31a9788e78f0\") " pod="opendatahub/maas-api-e2e-ait-df0d3551-5fd49d77d7-b777j"
Jul 06 14:45:56.681046 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:56.680855    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4cda6d94-068b-4253-b579-31a9788e78f0-maas-api-tls\") pod \"maas-api-e2e-ait-df0d3551-5fd49d77d7-b777j\" (UID: \"4cda6d94-068b-4253-b579-31a9788e78f0\") " pod="opendatahub/maas-api-e2e-ait-df0d3551-5fd49d77d7-b777j"
Jul 06 14:45:56.681046 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:56.680928    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gp2zm\" (UniqueName: \"kubernetes.io/projected/4cda6d94-068b-4253-b579-31a9788e78f0-kube-api-access-gp2zm\") pod \"maas-api-e2e-ait-df0d3551-5fd49d77d7-b777j\" (UID: \"4cda6d94-068b-4253-b579-31a9788e78f0\") " pod="opendatahub/maas-api-e2e-ait-df0d3551-5fd49d77d7-b777j"
Jul 06 14:45:56.681309 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:45:56.681286    2559 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-ait-df0d3551-serving-cert: secret "maas-api-e2e-ait-df0d3551-serving-cert" not found
Jul 06 14:45:56.681402 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:45:56.681357    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4cda6d94-068b-4253-b579-31a9788e78f0-maas-api-tls podName:4cda6d94-068b-4253-b579-31a9788e78f0 nodeName:}" failed. No retries permitted until 2026-07-06 14:45:57.181338748 +0000 UTC m=+1579.488630808 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/4cda6d94-068b-4253-b579-31a9788e78f0-maas-api-tls") pod "maas-api-e2e-ait-df0d3551-5fd49d77d7-b777j" (UID: "4cda6d94-068b-4253-b579-31a9788e78f0") : secret "maas-api-e2e-ait-df0d3551-serving-cert" not found
Jul 06 14:45:56.681560 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:56.681533    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4cda6d94-068b-4253-b579-31a9788e78f0-openshift-service-ca\") pod \"maas-api-e2e-ait-df0d3551-5fd49d77d7-b777j\" (UID: \"4cda6d94-068b-4253-b579-31a9788e78f0\") " pod="opendatahub/maas-api-e2e-ait-df0d3551-5fd49d77d7-b777j"
Jul 06 14:45:56.690557 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:56.690537    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gp2zm\" (UniqueName: \"kubernetes.io/projected/4cda6d94-068b-4253-b579-31a9788e78f0-kube-api-access-gp2zm\") pod \"maas-api-e2e-ait-df0d3551-5fd49d77d7-b777j\" (UID: \"4cda6d94-068b-4253-b579-31a9788e78f0\") " pod="opendatahub/maas-api-e2e-ait-df0d3551-5fd49d77d7-b777j"
Jul 06 14:45:57.185348 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:57.185315    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4cda6d94-068b-4253-b579-31a9788e78f0-maas-api-tls\") pod \"maas-api-e2e-ait-df0d3551-5fd49d77d7-b777j\" (UID: \"4cda6d94-068b-4253-b579-31a9788e78f0\") " pod="opendatahub/maas-api-e2e-ait-df0d3551-5fd49d77d7-b777j"
Jul 06 14:45:57.187715 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:57.187692    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4cda6d94-068b-4253-b579-31a9788e78f0-maas-api-tls\") pod \"maas-api-e2e-ait-df0d3551-5fd49d77d7-b777j\" (UID: \"4cda6d94-068b-4253-b579-31a9788e78f0\") " pod="opendatahub/maas-api-e2e-ait-df0d3551-5fd49d77d7-b777j"
Jul 06 14:45:57.397911 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:57.397873    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-df0d3551-5fd49d77d7-b777j"
Jul 06 14:45:57.502619 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:57.502560    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-df0d3551-openshift-default-cb545dccf-rwjrm"
Jul 06 14:45:57.507990 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:57.507959    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-df0d3551-openshift-default-cb545dccf-rwjrm"
Jul 06 14:45:57.569343 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:57.569316    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-df0d3551-5fd49d77d7-b777j"]
Jul 06 14:45:57.570832 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:45:57.570803    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4cda6d94_068b_4253_b579_31a9788e78f0.slice/crio-b775521c3d4bafb2c309a715f0d393878553daa5539171f563682ace49fbc2e0 WatchSource:0}: Error finding container b775521c3d4bafb2c309a715f0d393878553daa5539171f563682ace49fbc2e0: Status 404 returned error can't find the container with id b775521c3d4bafb2c309a715f0d393878553daa5539171f563682ace49fbc2e0
Jul 06 14:45:57.571710 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:57.571688    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-df0d3551-openshift-default-cb545dccf-rwjrm"
Jul 06 14:45:57.572804 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:57.572780    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-df0d3551-openshift-default-cb545dccf-rwjrm"
Jul 06 14:45:58.577378 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:58.577342    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-df0d3551-5fd49d77d7-b777j" event={"ID":"4cda6d94-068b-4253-b579-31a9788e78f0","Type":"ContainerStarted","Data":"7f7a9826e59c5ffb8176a7d6098ef2c925ecf48717326b0c5063cc0423708c81"}
Jul 06 14:45:58.577812 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:58.577395    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-df0d3551-5fd49d77d7-b777j" event={"ID":"4cda6d94-068b-4253-b579-31a9788e78f0","Type":"ContainerStarted","Data":"b775521c3d4bafb2c309a715f0d393878553daa5539171f563682ace49fbc2e0"}
Jul 06 14:45:58.577812 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:58.577595    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-ait-df0d3551-5fd49d77d7-b777j"
Jul 06 14:45:58.599357 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:45:58.599310    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-ait-df0d3551-5fd49d77d7-b777j" podStartSLOduration=2.192880677 podStartE2EDuration="2.599295609s" totalImagesPullingTime="406.414932ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:45:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:45:58.597208779 +0000 UTC m=+1580.904500864" watchObservedRunningTime="2026-07-06 14:45:58.599295609 +0000 UTC m=+1580.906587691"
Jul 06 14:46:01.907271 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:01.907236    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-df0d3551-5fd49d77d7-b777j"]
Jul 06 14:46:01.907674 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:01.907419    2559 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-ait-df0d3551-5fd49d77d7-b777j" podUID="4cda6d94-068b-4253-b579-31a9788e78f0" containerName="maas-api" containerID="cri-o://7f7a9826e59c5ffb8176a7d6098ef2c925ecf48717326b0c5063cc0423708c81" gracePeriod=30
Jul 06 14:46:02.156109 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:02.156084    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-df0d3551-5fd49d77d7-b777j"
Jul 06 14:46:02.236681 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:02.236653    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gp2zm\" (UniqueName: \"kubernetes.io/projected/4cda6d94-068b-4253-b579-31a9788e78f0-kube-api-access-gp2zm\") pod \"4cda6d94-068b-4253-b579-31a9788e78f0\" (UID: \"4cda6d94-068b-4253-b579-31a9788e78f0\") "
Jul 06 14:46:02.236843 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:02.236702    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4cda6d94-068b-4253-b579-31a9788e78f0-openshift-service-ca\") pod \"4cda6d94-068b-4253-b579-31a9788e78f0\" (UID: \"4cda6d94-068b-4253-b579-31a9788e78f0\") "
Jul 06 14:46:02.236843 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:02.236748    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4cda6d94-068b-4253-b579-31a9788e78f0-maas-api-tls\") pod \"4cda6d94-068b-4253-b579-31a9788e78f0\" (UID: \"4cda6d94-068b-4253-b579-31a9788e78f0\") "
Jul 06 14:46:02.237107 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:02.237080    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4cda6d94-068b-4253-b579-31a9788e78f0-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "4cda6d94-068b-4253-b579-31a9788e78f0" (UID: "4cda6d94-068b-4253-b579-31a9788e78f0"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 14:46:02.238649 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:02.238626    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4cda6d94-068b-4253-b579-31a9788e78f0-kube-api-access-gp2zm" (OuterVolumeSpecName: "kube-api-access-gp2zm") pod "4cda6d94-068b-4253-b579-31a9788e78f0" (UID: "4cda6d94-068b-4253-b579-31a9788e78f0"). InnerVolumeSpecName "kube-api-access-gp2zm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 14:46:02.238794 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:02.238777    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4cda6d94-068b-4253-b579-31a9788e78f0-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "4cda6d94-068b-4253-b579-31a9788e78f0" (UID: "4cda6d94-068b-4253-b579-31a9788e78f0"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 14:46:02.338357 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:02.338327    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4cda6d94-068b-4253-b579-31a9788e78f0-maas-api-tls\") on node \"ip-10-0-133-112.ec2.internal\" DevicePath \"\""
Jul 06 14:46:02.338357 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:02.338354    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gp2zm\" (UniqueName: \"kubernetes.io/projected/4cda6d94-068b-4253-b579-31a9788e78f0-kube-api-access-gp2zm\") on node \"ip-10-0-133-112.ec2.internal\" DevicePath \"\""
Jul 06 14:46:02.338357 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:02.338364    2559 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4cda6d94-068b-4253-b579-31a9788e78f0-openshift-service-ca\") on node \"ip-10-0-133-112.ec2.internal\" DevicePath \"\""
Jul 06 14:46:02.601478 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:02.601390    2559 generic.go:356] "Generic (PLEG): container finished" podID="4cda6d94-068b-4253-b579-31a9788e78f0" containerID="7f7a9826e59c5ffb8176a7d6098ef2c925ecf48717326b0c5063cc0423708c81" exitCode=0
Jul 06 14:46:02.601478 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:02.601436    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-df0d3551-5fd49d77d7-b777j"
Jul 06 14:46:02.601478 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:02.601470    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-df0d3551-5fd49d77d7-b777j" event={"ID":"4cda6d94-068b-4253-b579-31a9788e78f0","Type":"ContainerDied","Data":"7f7a9826e59c5ffb8176a7d6098ef2c925ecf48717326b0c5063cc0423708c81"}
Jul 06 14:46:02.601751 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:02.601522    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-df0d3551-5fd49d77d7-b777j" event={"ID":"4cda6d94-068b-4253-b579-31a9788e78f0","Type":"ContainerDied","Data":"b775521c3d4bafb2c309a715f0d393878553daa5539171f563682ace49fbc2e0"}
Jul 06 14:46:02.601751 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:02.601543    2559 scope.go:117] "RemoveContainer" containerID="7f7a9826e59c5ffb8176a7d6098ef2c925ecf48717326b0c5063cc0423708c81"
Jul 06 14:46:02.610678 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:02.610660    2559 scope.go:117] "RemoveContainer" containerID="7f7a9826e59c5ffb8176a7d6098ef2c925ecf48717326b0c5063cc0423708c81"
Jul 06 14:46:02.610938 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:46:02.610921    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7f7a9826e59c5ffb8176a7d6098ef2c925ecf48717326b0c5063cc0423708c81\": container with ID starting with 7f7a9826e59c5ffb8176a7d6098ef2c925ecf48717326b0c5063cc0423708c81 not found: ID does not exist" containerID="7f7a9826e59c5ffb8176a7d6098ef2c925ecf48717326b0c5063cc0423708c81"
Jul 06 14:46:02.610995 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:02.610947    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7f7a9826e59c5ffb8176a7d6098ef2c925ecf48717326b0c5063cc0423708c81"} err="failed to get container status \"7f7a9826e59c5ffb8176a7d6098ef2c925ecf48717326b0c5063cc0423708c81\": rpc error: code = NotFound desc = could not find container \"7f7a9826e59c5ffb8176a7d6098ef2c925ecf48717326b0c5063cc0423708c81\": container with ID starting with 7f7a9826e59c5ffb8176a7d6098ef2c925ecf48717326b0c5063cc0423708c81 not found: ID does not exist"
Jul 06 14:46:02.628887 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:02.628862    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-df0d3551-5fd49d77d7-b777j"]
Jul 06 14:46:02.634300 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:02.634279    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-ait-df0d3551-5fd49d77d7-b777j"]
Jul 06 14:46:04.226666 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:04.226629    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4cda6d94-068b-4253-b579-31a9788e78f0" path="/var/lib/kubelet/pods/4cda6d94-068b-4253-b579-31a9788e78f0/volumes"
Jul 06 14:46:08.273701 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:08.273667    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-df0d3551-openshift-default-cb545dccf-rwjrm"]
Jul 06 14:46:08.274095 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:08.273884    2559 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-df0d3551-openshift-default-cb545dccf-rwjrm" podUID="721e13eb-83b2-4904-a851-3c7c7fc52e84" containerName="istio-proxy" containerID="cri-o://30ae12fcafa3905c67cc51dae371eb9299871ee097625c995c8fbbcf98f5d55d" gracePeriod=30
Jul 06 14:46:12.574233 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:12.574194    2559 patch_prober.go:28] interesting pod/e2e-ait-df0d3551-openshift-default-cb545dccf-rwjrm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 06 14:46:12.574715 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:12.574249    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-df0d3551-openshift-default-cb545dccf-rwjrm" podUID="721e13eb-83b2-4904-a851-3c7c7fc52e84" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 06 14:46:13.549652 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:13.549622    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-df0d3551-openshift-default-cb545dccf-rwjrm"
Jul 06 14:46:13.643800 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:13.643774    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/721e13eb-83b2-4904-a851-3c7c7fc52e84-istiod-ca-cert\") pod \"721e13eb-83b2-4904-a851-3c7c7fc52e84\" (UID: \"721e13eb-83b2-4904-a851-3c7c7fc52e84\") "
Jul 06 14:46:13.644192 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:13.643844    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/721e13eb-83b2-4904-a851-3c7c7fc52e84-credential-socket\") pod \"721e13eb-83b2-4904-a851-3c7c7fc52e84\" (UID: \"721e13eb-83b2-4904-a851-3c7c7fc52e84\") "
Jul 06 14:46:13.644192 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:13.643865    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/721e13eb-83b2-4904-a851-3c7c7fc52e84-workload-socket\") pod \"721e13eb-83b2-4904-a851-3c7c7fc52e84\" (UID: \"721e13eb-83b2-4904-a851-3c7c7fc52e84\") "
Jul 06 14:46:13.644192 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:13.643981    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/721e13eb-83b2-4904-a851-3c7c7fc52e84-istio-data\") pod \"721e13eb-83b2-4904-a851-3c7c7fc52e84\" (UID: \"721e13eb-83b2-4904-a851-3c7c7fc52e84\") "
Jul 06 14:46:13.644192 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:13.644033    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/721e13eb-83b2-4904-a851-3c7c7fc52e84-workload-certs\") pod \"721e13eb-83b2-4904-a851-3c7c7fc52e84\" (UID: \"721e13eb-83b2-4904-a851-3c7c7fc52e84\") "
Jul 06 14:46:13.644192 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:13.644100    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sqqch\" (UniqueName: \"kubernetes.io/projected/721e13eb-83b2-4904-a851-3c7c7fc52e84-kube-api-access-sqqch\") pod \"721e13eb-83b2-4904-a851-3c7c7fc52e84\" (UID: \"721e13eb-83b2-4904-a851-3c7c7fc52e84\") "
Jul 06 14:46:13.644192 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:13.644103    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/721e13eb-83b2-4904-a851-3c7c7fc52e84-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "721e13eb-83b2-4904-a851-3c7c7fc52e84" (UID: "721e13eb-83b2-4904-a851-3c7c7fc52e84"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 14:46:13.644192 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:13.644130    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/721e13eb-83b2-4904-a851-3c7c7fc52e84-istio-envoy\") pod \"721e13eb-83b2-4904-a851-3c7c7fc52e84\" (UID: \"721e13eb-83b2-4904-a851-3c7c7fc52e84\") "
Jul 06 14:46:13.644192 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:13.644174    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/721e13eb-83b2-4904-a851-3c7c7fc52e84-istio-podinfo\") pod \"721e13eb-83b2-4904-a851-3c7c7fc52e84\" (UID: \"721e13eb-83b2-4904-a851-3c7c7fc52e84\") "
Jul 06 14:46:13.644611 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:13.644218    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/721e13eb-83b2-4904-a851-3c7c7fc52e84-istio-token\") pod \"721e13eb-83b2-4904-a851-3c7c7fc52e84\" (UID: \"721e13eb-83b2-4904-a851-3c7c7fc52e84\") "
Jul 06 14:46:13.644611 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:13.644319    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/721e13eb-83b2-4904-a851-3c7c7fc52e84-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "721e13eb-83b2-4904-a851-3c7c7fc52e84" (UID: "721e13eb-83b2-4904-a851-3c7c7fc52e84"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 14:46:13.644611 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:13.644386    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/721e13eb-83b2-4904-a851-3c7c7fc52e84-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "721e13eb-83b2-4904-a851-3c7c7fc52e84" (UID: "721e13eb-83b2-4904-a851-3c7c7fc52e84"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 14:46:13.644791 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:13.644644    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/721e13eb-83b2-4904-a851-3c7c7fc52e84-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "721e13eb-83b2-4904-a851-3c7c7fc52e84" (UID: "721e13eb-83b2-4904-a851-3c7c7fc52e84"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 14:46:13.644791 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:13.644651    2559 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/721e13eb-83b2-4904-a851-3c7c7fc52e84-istiod-ca-cert\") on node \"ip-10-0-133-112.ec2.internal\" DevicePath \"\""
Jul 06 14:46:13.644791 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:13.644685    2559 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/721e13eb-83b2-4904-a851-3c7c7fc52e84-workload-socket\") on node \"ip-10-0-133-112.ec2.internal\" DevicePath \"\""
Jul 06 14:46:13.644791 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:13.644702    2559 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/721e13eb-83b2-4904-a851-3c7c7fc52e84-workload-certs\") on node \"ip-10-0-133-112.ec2.internal\" DevicePath \"\""
Jul 06 14:46:13.644991 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:13.644787    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/721e13eb-83b2-4904-a851-3c7c7fc52e84-istio-data" (OuterVolumeSpecName: "istio-data") pod "721e13eb-83b2-4904-a851-3c7c7fc52e84" (UID: "721e13eb-83b2-4904-a851-3c7c7fc52e84"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 14:46:13.646469 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:13.646443    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/721e13eb-83b2-4904-a851-3c7c7fc52e84-kube-api-access-sqqch" (OuterVolumeSpecName: "kube-api-access-sqqch") pod "721e13eb-83b2-4904-a851-3c7c7fc52e84" (UID: "721e13eb-83b2-4904-a851-3c7c7fc52e84"). InnerVolumeSpecName "kube-api-access-sqqch". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 14:46:13.646600 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:13.646477    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/721e13eb-83b2-4904-a851-3c7c7fc52e84-istio-token" (OuterVolumeSpecName: "istio-token") pod "721e13eb-83b2-4904-a851-3c7c7fc52e84" (UID: "721e13eb-83b2-4904-a851-3c7c7fc52e84"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 14:46:13.646792 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:13.646772    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/721e13eb-83b2-4904-a851-3c7c7fc52e84-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "721e13eb-83b2-4904-a851-3c7c7fc52e84" (UID: "721e13eb-83b2-4904-a851-3c7c7fc52e84"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 06 14:46:13.646872 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:13.646806    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/721e13eb-83b2-4904-a851-3c7c7fc52e84-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "721e13eb-83b2-4904-a851-3c7c7fc52e84" (UID: "721e13eb-83b2-4904-a851-3c7c7fc52e84"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 14:46:13.658634 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:13.658608    2559 generic.go:356] "Generic (PLEG): container finished" podID="721e13eb-83b2-4904-a851-3c7c7fc52e84" containerID="30ae12fcafa3905c67cc51dae371eb9299871ee097625c995c8fbbcf98f5d55d" exitCode=0
Jul 06 14:46:13.658730 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:13.658643    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-df0d3551-openshift-default-cb545dccf-rwjrm" event={"ID":"721e13eb-83b2-4904-a851-3c7c7fc52e84","Type":"ContainerDied","Data":"30ae12fcafa3905c67cc51dae371eb9299871ee097625c995c8fbbcf98f5d55d"}
Jul 06 14:46:13.658730 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:13.658654    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-df0d3551-openshift-default-cb545dccf-rwjrm"
Jul 06 14:46:13.658730 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:13.658685    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-df0d3551-openshift-default-cb545dccf-rwjrm" event={"ID":"721e13eb-83b2-4904-a851-3c7c7fc52e84","Type":"ContainerDied","Data":"5c4030ae32a483ba3a332c1f84e4c8dc7d0edd47173bc4d9fd5958d4d0d3c1c0"}
Jul 06 14:46:13.658730 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:13.658707    2559 scope.go:117] "RemoveContainer" containerID="30ae12fcafa3905c67cc51dae371eb9299871ee097625c995c8fbbcf98f5d55d"
Jul 06 14:46:13.667801 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:13.667786    2559 scope.go:117] "RemoveContainer" containerID="30ae12fcafa3905c67cc51dae371eb9299871ee097625c995c8fbbcf98f5d55d"
Jul 06 14:46:13.668036 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:46:13.668018    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"30ae12fcafa3905c67cc51dae371eb9299871ee097625c995c8fbbcf98f5d55d\": container with ID starting with 30ae12fcafa3905c67cc51dae371eb9299871ee097625c995c8fbbcf98f5d55d not found: ID does not exist" containerID="30ae12fcafa3905c67cc51dae371eb9299871ee097625c995c8fbbcf98f5d55d"
Jul 06 14:46:13.668095 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:13.668044    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"30ae12fcafa3905c67cc51dae371eb9299871ee097625c995c8fbbcf98f5d55d"} err="failed to get container status \"30ae12fcafa3905c67cc51dae371eb9299871ee097625c995c8fbbcf98f5d55d\": rpc error: code = NotFound desc = could not find container \"30ae12fcafa3905c67cc51dae371eb9299871ee097625c995c8fbbcf98f5d55d\": container with ID starting with 30ae12fcafa3905c67cc51dae371eb9299871ee097625c995c8fbbcf98f5d55d not found: ID does not exist"
Jul 06 14:46:13.684536 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:13.684510    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-df0d3551-openshift-default-cb545dccf-rwjrm"]
Jul 06 14:46:13.688190 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:13.688169    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-df0d3551-openshift-default-cb545dccf-rwjrm"]
Jul 06 14:46:13.745268 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:13.745231    2559 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/721e13eb-83b2-4904-a851-3c7c7fc52e84-istio-podinfo\") on node \"ip-10-0-133-112.ec2.internal\" DevicePath \"\""
Jul 06 14:46:13.745268 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:13.745264    2559 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/721e13eb-83b2-4904-a851-3c7c7fc52e84-istio-token\") on node \"ip-10-0-133-112.ec2.internal\" DevicePath \"\""
Jul 06 14:46:13.745268 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:13.745277    2559 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/721e13eb-83b2-4904-a851-3c7c7fc52e84-credential-socket\") on node \"ip-10-0-133-112.ec2.internal\" DevicePath \"\""
Jul 06 14:46:13.745493 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:13.745288    2559 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/721e13eb-83b2-4904-a851-3c7c7fc52e84-istio-data\") on node \"ip-10-0-133-112.ec2.internal\" DevicePath \"\""
Jul 06 14:46:13.745493 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:13.745300    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sqqch\" (UniqueName: \"kubernetes.io/projected/721e13eb-83b2-4904-a851-3c7c7fc52e84-kube-api-access-sqqch\") on node \"ip-10-0-133-112.ec2.internal\" DevicePath \"\""
Jul 06 14:46:13.745493 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:13.745310    2559 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/721e13eb-83b2-4904-a851-3c7c7fc52e84-istio-envoy\") on node \"ip-10-0-133-112.ec2.internal\" DevicePath \"\""
Jul 06 14:46:14.227637 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:14.227605    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="721e13eb-83b2-4904-a851-3c7c7fc52e84" path="/var/lib/kubelet/pods/721e13eb-83b2-4904-a851-3c7c7fc52e84/volumes"
Jul 06 14:46:15.119293 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:15.119263    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-3cf8ebc5-openshift-default-65fd87797f-4dv7p"]
Jul 06 14:46:15.119842 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:15.119823    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="721e13eb-83b2-4904-a851-3c7c7fc52e84" containerName="istio-proxy"
Jul 06 14:46:15.119842 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:15.119844    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="721e13eb-83b2-4904-a851-3c7c7fc52e84" containerName="istio-proxy"
Jul 06 14:46:15.119973 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:15.119856    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4cda6d94-068b-4253-b579-31a9788e78f0" containerName="maas-api"
Jul 06 14:46:15.119973 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:15.119865    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="4cda6d94-068b-4253-b579-31a9788e78f0" containerName="maas-api"
Jul 06 14:46:15.120069 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:15.119980    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="721e13eb-83b2-4904-a851-3c7c7fc52e84" containerName="istio-proxy"
Jul 06 14:46:15.120069 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:15.119993    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="4cda6d94-068b-4253-b579-31a9788e78f0" containerName="maas-api"
Jul 06 14:46:15.124484 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:15.124464    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-3cf8ebc5-openshift-default-65fd87797f-4dv7p"
Jul 06 14:46:15.126833 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:15.126815    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-3cf8ebc5-openshift-default-dockercfg-85fbs\""
Jul 06 14:46:15.134699 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:15.134676    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-3cf8ebc5-openshift-default-65fd87797f-4dv7p"]
Jul 06 14:46:15.259677 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:15.259588    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a48781ef-5a49-401d-aa25-4d03b5b2b16c-istio-envoy\") pod \"e2e-derive-3cf8ebc5-openshift-default-65fd87797f-4dv7p\" (UID: \"a48781ef-5a49-401d-aa25-4d03b5b2b16c\") " pod="openshift-ingress/e2e-derive-3cf8ebc5-openshift-default-65fd87797f-4dv7p"
Jul 06 14:46:15.259677 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:15.259641    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a48781ef-5a49-401d-aa25-4d03b5b2b16c-istio-token\") pod \"e2e-derive-3cf8ebc5-openshift-default-65fd87797f-4dv7p\" (UID: \"a48781ef-5a49-401d-aa25-4d03b5b2b16c\") " pod="openshift-ingress/e2e-derive-3cf8ebc5-openshift-default-65fd87797f-4dv7p"
Jul 06 14:46:15.259677 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:15.259673    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a48781ef-5a49-401d-aa25-4d03b5b2b16c-workload-socket\") pod \"e2e-derive-3cf8ebc5-openshift-default-65fd87797f-4dv7p\" (UID: \"a48781ef-5a49-401d-aa25-4d03b5b2b16c\") " pod="openshift-ingress/e2e-derive-3cf8ebc5-openshift-default-65fd87797f-4dv7p"
Jul 06 14:46:15.259914 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:15.259695    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-drgc7\" (UniqueName: \"kubernetes.io/projected/a48781ef-5a49-401d-aa25-4d03b5b2b16c-kube-api-access-drgc7\") pod \"e2e-derive-3cf8ebc5-openshift-default-65fd87797f-4dv7p\" (UID: \"a48781ef-5a49-401d-aa25-4d03b5b2b16c\") " pod="openshift-ingress/e2e-derive-3cf8ebc5-openshift-default-65fd87797f-4dv7p"
Jul 06 14:46:15.259914 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:15.259721    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a48781ef-5a49-401d-aa25-4d03b5b2b16c-workload-certs\") pod \"e2e-derive-3cf8ebc5-openshift-default-65fd87797f-4dv7p\" (UID: \"a48781ef-5a49-401d-aa25-4d03b5b2b16c\") " pod="openshift-ingress/e2e-derive-3cf8ebc5-openshift-default-65fd87797f-4dv7p"
Jul 06 14:46:15.259914 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:15.259754    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a48781ef-5a49-401d-aa25-4d03b5b2b16c-istiod-ca-cert\") pod \"e2e-derive-3cf8ebc5-openshift-default-65fd87797f-4dv7p\" (UID: \"a48781ef-5a49-401d-aa25-4d03b5b2b16c\") " pod="openshift-ingress/e2e-derive-3cf8ebc5-openshift-default-65fd87797f-4dv7p"
Jul 06 14:46:15.259914 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:15.259772    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a48781ef-5a49-401d-aa25-4d03b5b2b16c-credential-socket\") pod \"e2e-derive-3cf8ebc5-openshift-default-65fd87797f-4dv7p\" (UID: \"a48781ef-5a49-401d-aa25-4d03b5b2b16c\") " pod="openshift-ingress/e2e-derive-3cf8ebc5-openshift-default-65fd87797f-4dv7p"
Jul 06 14:46:15.259914 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:15.259805    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a48781ef-5a49-401d-aa25-4d03b5b2b16c-istio-podinfo\") pod \"e2e-derive-3cf8ebc5-openshift-default-65fd87797f-4dv7p\" (UID: \"a48781ef-5a49-401d-aa25-4d03b5b2b16c\") " pod="openshift-ingress/e2e-derive-3cf8ebc5-openshift-default-65fd87797f-4dv7p"
Jul 06 14:46:15.259914 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:15.259842    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a48781ef-5a49-401d-aa25-4d03b5b2b16c-istio-data\") pod \"e2e-derive-3cf8ebc5-openshift-default-65fd87797f-4dv7p\" (UID: \"a48781ef-5a49-401d-aa25-4d03b5b2b16c\") " pod="openshift-ingress/e2e-derive-3cf8ebc5-openshift-default-65fd87797f-4dv7p"
Jul 06 14:46:15.361018 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:15.360985    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a48781ef-5a49-401d-aa25-4d03b5b2b16c-istio-token\") pod \"e2e-derive-3cf8ebc5-openshift-default-65fd87797f-4dv7p\" (UID: \"a48781ef-5a49-401d-aa25-4d03b5b2b16c\") " pod="openshift-ingress/e2e-derive-3cf8ebc5-openshift-default-65fd87797f-4dv7p"
Jul 06 14:46:15.361225 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:15.361035    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a48781ef-5a49-401d-aa25-4d03b5b2b16c-workload-socket\") pod \"e2e-derive-3cf8ebc5-openshift-default-65fd87797f-4dv7p\" (UID: \"a48781ef-5a49-401d-aa25-4d03b5b2b16c\") " pod="openshift-ingress/e2e-derive-3cf8ebc5-openshift-default-65fd87797f-4dv7p"
Jul 06 14:46:15.361225 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:15.361062    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-drgc7\" (UniqueName: \"kubernetes.io/projected/a48781ef-5a49-401d-aa25-4d03b5b2b16c-kube-api-access-drgc7\") pod \"e2e-derive-3cf8ebc5-openshift-default-65fd87797f-4dv7p\" (UID: \"a48781ef-5a49-401d-aa25-4d03b5b2b16c\") " pod="openshift-ingress/e2e-derive-3cf8ebc5-openshift-default-65fd87797f-4dv7p"
Jul 06 14:46:15.361225 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:15.361100    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a48781ef-5a49-401d-aa25-4d03b5b2b16c-workload-certs\") pod \"e2e-derive-3cf8ebc5-openshift-default-65fd87797f-4dv7p\" (UID: \"a48781ef-5a49-401d-aa25-4d03b5b2b16c\") " pod="openshift-ingress/e2e-derive-3cf8ebc5-openshift-default-65fd87797f-4dv7p"
Jul 06 14:46:15.361225 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:15.361123    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a48781ef-5a49-401d-aa25-4d03b5b2b16c-istiod-ca-cert\") pod \"e2e-derive-3cf8ebc5-openshift-default-65fd87797f-4dv7p\" (UID: \"a48781ef-5a49-401d-aa25-4d03b5b2b16c\") " pod="openshift-ingress/e2e-derive-3cf8ebc5-openshift-default-65fd87797f-4dv7p"
Jul 06 14:46:15.361225 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:15.361149    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a48781ef-5a49-401d-aa25-4d03b5b2b16c-credential-socket\") pod \"e2e-derive-3cf8ebc5-openshift-default-65fd87797f-4dv7p\" (UID: \"a48781ef-5a49-401d-aa25-4d03b5b2b16c\") " pod="openshift-ingress/e2e-derive-3cf8ebc5-openshift-default-65fd87797f-4dv7p"
Jul 06 14:46:15.361225 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:15.361195    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a48781ef-5a49-401d-aa25-4d03b5b2b16c-istio-podinfo\") pod \"e2e-derive-3cf8ebc5-openshift-default-65fd87797f-4dv7p\" (UID: \"a48781ef-5a49-401d-aa25-4d03b5b2b16c\") " pod="openshift-ingress/e2e-derive-3cf8ebc5-openshift-default-65fd87797f-4dv7p"
Jul 06 14:46:15.361225 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:15.361223    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a48781ef-5a49-401d-aa25-4d03b5b2b16c-istio-data\") pod \"e2e-derive-3cf8ebc5-openshift-default-65fd87797f-4dv7p\" (UID: \"a48781ef-5a49-401d-aa25-4d03b5b2b16c\") " pod="openshift-ingress/e2e-derive-3cf8ebc5-openshift-default-65fd87797f-4dv7p"
Jul 06 14:46:15.361629 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:15.361268    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a48781ef-5a49-401d-aa25-4d03b5b2b16c-istio-envoy\") pod \"e2e-derive-3cf8ebc5-openshift-default-65fd87797f-4dv7p\" (UID: \"a48781ef-5a49-401d-aa25-4d03b5b2b16c\") " pod="openshift-ingress/e2e-derive-3cf8ebc5-openshift-default-65fd87797f-4dv7p"
Jul 06 14:46:15.361629 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:15.361463    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a48781ef-5a49-401d-aa25-4d03b5b2b16c-workload-socket\") pod \"e2e-derive-3cf8ebc5-openshift-default-65fd87797f-4dv7p\" (UID: \"a48781ef-5a49-401d-aa25-4d03b5b2b16c\") " pod="openshift-ingress/e2e-derive-3cf8ebc5-openshift-default-65fd87797f-4dv7p"
Jul 06 14:46:15.361742 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:15.361640    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a48781ef-5a49-401d-aa25-4d03b5b2b16c-credential-socket\") pod \"e2e-derive-3cf8ebc5-openshift-default-65fd87797f-4dv7p\" (UID: \"a48781ef-5a49-401d-aa25-4d03b5b2b16c\") " pod="openshift-ingress/e2e-derive-3cf8ebc5-openshift-default-65fd87797f-4dv7p"
Jul 06 14:46:15.361742 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:15.361669    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a48781ef-5a49-401d-aa25-4d03b5b2b16c-workload-certs\") pod \"e2e-derive-3cf8ebc5-openshift-default-65fd87797f-4dv7p\" (UID: \"a48781ef-5a49-401d-aa25-4d03b5b2b16c\") " pod="openshift-ingress/e2e-derive-3cf8ebc5-openshift-default-65fd87797f-4dv7p"
Jul 06 14:46:15.361742 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:15.361696    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a48781ef-5a49-401d-aa25-4d03b5b2b16c-istio-data\") pod \"e2e-derive-3cf8ebc5-openshift-default-65fd87797f-4dv7p\" (UID: \"a48781ef-5a49-401d-aa25-4d03b5b2b16c\") " pod="openshift-ingress/e2e-derive-3cf8ebc5-openshift-default-65fd87797f-4dv7p"
Jul 06 14:46:15.361867 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:15.361848    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a48781ef-5a49-401d-aa25-4d03b5b2b16c-istiod-ca-cert\") pod \"e2e-derive-3cf8ebc5-openshift-default-65fd87797f-4dv7p\" (UID: \"a48781ef-5a49-401d-aa25-4d03b5b2b16c\") " pod="openshift-ingress/e2e-derive-3cf8ebc5-openshift-default-65fd87797f-4dv7p"
Jul 06 14:46:15.363415 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:15.363399    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a48781ef-5a49-401d-aa25-4d03b5b2b16c-istio-envoy\") pod \"e2e-derive-3cf8ebc5-openshift-default-65fd87797f-4dv7p\" (UID: \"a48781ef-5a49-401d-aa25-4d03b5b2b16c\") " pod="openshift-ingress/e2e-derive-3cf8ebc5-openshift-default-65fd87797f-4dv7p"
Jul 06 14:46:15.363817 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:15.363798    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a48781ef-5a49-401d-aa25-4d03b5b2b16c-istio-podinfo\") pod \"e2e-derive-3cf8ebc5-openshift-default-65fd87797f-4dv7p\" (UID: \"a48781ef-5a49-401d-aa25-4d03b5b2b16c\") " pod="openshift-ingress/e2e-derive-3cf8ebc5-openshift-default-65fd87797f-4dv7p"
Jul 06 14:46:15.375285 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:15.375252    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-drgc7\" (UniqueName: \"kubernetes.io/projected/a48781ef-5a49-401d-aa25-4d03b5b2b16c-kube-api-access-drgc7\") pod \"e2e-derive-3cf8ebc5-openshift-default-65fd87797f-4dv7p\" (UID: \"a48781ef-5a49-401d-aa25-4d03b5b2b16c\") " pod="openshift-ingress/e2e-derive-3cf8ebc5-openshift-default-65fd87797f-4dv7p"
Jul 06 14:46:15.375383 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:15.375307    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a48781ef-5a49-401d-aa25-4d03b5b2b16c-istio-token\") pod \"e2e-derive-3cf8ebc5-openshift-default-65fd87797f-4dv7p\" (UID: \"a48781ef-5a49-401d-aa25-4d03b5b2b16c\") " pod="openshift-ingress/e2e-derive-3cf8ebc5-openshift-default-65fd87797f-4dv7p"
Jul 06 14:46:15.437015 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:15.436982    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-3cf8ebc5-openshift-default-65fd87797f-4dv7p"
Jul 06 14:46:15.577999 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:15.577970    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-3cf8ebc5-openshift-default-65fd87797f-4dv7p"]
Jul 06 14:46:15.579116 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:46:15.579084    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda48781ef_5a49_401d_aa25_4d03b5b2b16c.slice/crio-8174a246e0d0c5892821cc26ebcaee0593aba86acec0da575d15d950dc271098 WatchSource:0}: Error finding container 8174a246e0d0c5892821cc26ebcaee0593aba86acec0da575d15d950dc271098: Status 404 returned error can't find the container with id 8174a246e0d0c5892821cc26ebcaee0593aba86acec0da575d15d950dc271098
Jul 06 14:46:15.581280 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:15.581245    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 06 14:46:15.581389 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:15.581310    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 06 14:46:15.581389 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:15.581354    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 06 14:46:15.673391 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:15.673353    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-3cf8ebc5-openshift-default-65fd87797f-4dv7p" event={"ID":"a48781ef-5a49-401d-aa25-4d03b5b2b16c","Type":"ContainerStarted","Data":"06c2ead1237dbd1b1f9e271d444850bdfe4c649b97df78b414b9a1833535357f"}
Jul 06 14:46:15.673630 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:15.673457    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-3cf8ebc5-openshift-default-65fd87797f-4dv7p" event={"ID":"a48781ef-5a49-401d-aa25-4d03b5b2b16c","Type":"ContainerStarted","Data":"8174a246e0d0c5892821cc26ebcaee0593aba86acec0da575d15d950dc271098"}
Jul 06 14:46:15.706622 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:15.706543    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-3cf8ebc5-openshift-default-65fd87797f-4dv7p" podStartSLOduration=0.706519244 podStartE2EDuration="706.519244ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:46:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:46:15.703871606 +0000 UTC m=+1598.011163689" watchObservedRunningTime="2026-07-06 14:46:15.706519244 +0000 UTC m=+1598.013811328"
Jul 06 14:46:16.437646 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:16.437606    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-3cf8ebc5-openshift-default-65fd87797f-4dv7p"
Jul 06 14:46:16.443719 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:16.443675    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-3cf8ebc5-openshift-default-65fd87797f-4dv7p"
Jul 06 14:46:16.572966 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:16.572931    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-derive-3cf8ebc5-866474fcbc-fsdjp"]
Jul 06 14:46:16.576883 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:16.576858    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-3cf8ebc5-866474fcbc-fsdjp"
Jul 06 14:46:16.579625 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:16.579597    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-derive-3cf8ebc5-serving-cert\""
Jul 06 14:46:16.591068 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:16.591035    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-derive-3cf8ebc5-866474fcbc-fsdjp"]
Jul 06 14:46:16.674823 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:16.674787    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a2f3a8d3-2879-4672-86a2-afcbe60263d4-maas-api-tls\") pod \"maas-api-e2e-derive-3cf8ebc5-866474fcbc-fsdjp\" (UID: \"a2f3a8d3-2879-4672-86a2-afcbe60263d4\") " pod="opendatahub/maas-api-e2e-derive-3cf8ebc5-866474fcbc-fsdjp"
Jul 06 14:46:16.675036 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:16.674832    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6l2tl\" (UniqueName: \"kubernetes.io/projected/a2f3a8d3-2879-4672-86a2-afcbe60263d4-kube-api-access-6l2tl\") pod \"maas-api-e2e-derive-3cf8ebc5-866474fcbc-fsdjp\" (UID: \"a2f3a8d3-2879-4672-86a2-afcbe60263d4\") " pod="opendatahub/maas-api-e2e-derive-3cf8ebc5-866474fcbc-fsdjp"
Jul 06 14:46:16.675036 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:16.674948    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a2f3a8d3-2879-4672-86a2-afcbe60263d4-openshift-service-ca\") pod \"maas-api-e2e-derive-3cf8ebc5-866474fcbc-fsdjp\" (UID: \"a2f3a8d3-2879-4672-86a2-afcbe60263d4\") " pod="opendatahub/maas-api-e2e-derive-3cf8ebc5-866474fcbc-fsdjp"
Jul 06 14:46:16.679549 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:16.679516    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-3cf8ebc5-openshift-default-65fd87797f-4dv7p"
Jul 06 14:46:16.680527 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:16.680504    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-3cf8ebc5-openshift-default-65fd87797f-4dv7p"
Jul 06 14:46:16.776057 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:16.776019    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a2f3a8d3-2879-4672-86a2-afcbe60263d4-openshift-service-ca\") pod \"maas-api-e2e-derive-3cf8ebc5-866474fcbc-fsdjp\" (UID: \"a2f3a8d3-2879-4672-86a2-afcbe60263d4\") " pod="opendatahub/maas-api-e2e-derive-3cf8ebc5-866474fcbc-fsdjp"
Jul 06 14:46:16.776244 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:16.776169    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a2f3a8d3-2879-4672-86a2-afcbe60263d4-maas-api-tls\") pod \"maas-api-e2e-derive-3cf8ebc5-866474fcbc-fsdjp\" (UID: \"a2f3a8d3-2879-4672-86a2-afcbe60263d4\") " pod="opendatahub/maas-api-e2e-derive-3cf8ebc5-866474fcbc-fsdjp"
Jul 06 14:46:16.776244 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:16.776198    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6l2tl\" (UniqueName: \"kubernetes.io/projected/a2f3a8d3-2879-4672-86a2-afcbe60263d4-kube-api-access-6l2tl\") pod \"maas-api-e2e-derive-3cf8ebc5-866474fcbc-fsdjp\" (UID: \"a2f3a8d3-2879-4672-86a2-afcbe60263d4\") " pod="opendatahub/maas-api-e2e-derive-3cf8ebc5-866474fcbc-fsdjp"
Jul 06 14:46:16.776813 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:16.776790    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a2f3a8d3-2879-4672-86a2-afcbe60263d4-openshift-service-ca\") pod \"maas-api-e2e-derive-3cf8ebc5-866474fcbc-fsdjp\" (UID: \"a2f3a8d3-2879-4672-86a2-afcbe60263d4\") " pod="opendatahub/maas-api-e2e-derive-3cf8ebc5-866474fcbc-fsdjp"
Jul 06 14:46:16.779070 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:16.779033    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a2f3a8d3-2879-4672-86a2-afcbe60263d4-maas-api-tls\") pod \"maas-api-e2e-derive-3cf8ebc5-866474fcbc-fsdjp\" (UID: \"a2f3a8d3-2879-4672-86a2-afcbe60263d4\") " pod="opendatahub/maas-api-e2e-derive-3cf8ebc5-866474fcbc-fsdjp"
Jul 06 14:46:16.789846 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:16.789814    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6l2tl\" (UniqueName: \"kubernetes.io/projected/a2f3a8d3-2879-4672-86a2-afcbe60263d4-kube-api-access-6l2tl\") pod \"maas-api-e2e-derive-3cf8ebc5-866474fcbc-fsdjp\" (UID: \"a2f3a8d3-2879-4672-86a2-afcbe60263d4\") " pod="opendatahub/maas-api-e2e-derive-3cf8ebc5-866474fcbc-fsdjp"
Jul 06 14:46:16.888249 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:16.888216    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-3cf8ebc5-866474fcbc-fsdjp"
Jul 06 14:46:17.059085 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:17.059058    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-derive-3cf8ebc5-866474fcbc-fsdjp"]
Jul 06 14:46:17.061633 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:46:17.061599    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2f3a8d3_2879_4672_86a2_afcbe60263d4.slice/crio-b6ba596479cba3641bdc27781395ec3a1e01406eefd58a7d490c53305aa8728e WatchSource:0}: Error finding container b6ba596479cba3641bdc27781395ec3a1e01406eefd58a7d490c53305aa8728e: Status 404 returned error can't find the container with id b6ba596479cba3641bdc27781395ec3a1e01406eefd58a7d490c53305aa8728e
Jul 06 14:46:17.685905 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:17.685856    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-3cf8ebc5-866474fcbc-fsdjp" event={"ID":"a2f3a8d3-2879-4672-86a2-afcbe60263d4","Type":"ContainerStarted","Data":"07460cd43c3a8b61b8d13da665981829b9f2958f805dd4e2e68fa0436477e448"}
Jul 06 14:46:17.686374 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:17.685922    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-3cf8ebc5-866474fcbc-fsdjp" event={"ID":"a2f3a8d3-2879-4672-86a2-afcbe60263d4","Type":"ContainerStarted","Data":"b6ba596479cba3641bdc27781395ec3a1e01406eefd58a7d490c53305aa8728e"}
Jul 06 14:46:17.686374 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:17.685950    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-derive-3cf8ebc5-866474fcbc-fsdjp"
Jul 06 14:46:17.706989 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:17.706925    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-derive-3cf8ebc5-866474fcbc-fsdjp" podStartSLOduration=1.2278648460000001 podStartE2EDuration="1.706907737s" totalImagesPullingTime="479.042891ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:46:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:46:17.705682625 +0000 UTC m=+1600.012974706" watchObservedRunningTime="2026-07-06 14:46:17.706907737 +0000 UTC m=+1600.014199819"
Jul 06 14:46:20.992296 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:20.992253    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-3cf8ebc5-openshift-default-65fd87797f-4dv7p"]
Jul 06 14:46:20.992747 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:20.992464    2559 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-3cf8ebc5-openshift-default-65fd87797f-4dv7p" podUID="a48781ef-5a49-401d-aa25-4d03b5b2b16c" containerName="istio-proxy" containerID="cri-o://06c2ead1237dbd1b1f9e271d444850bdfe4c649b97df78b414b9a1833535357f" gracePeriod=30
Jul 06 14:46:21.361996 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:21.361902    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-derive-3cf8ebc5-866474fcbc-fsdjp"]
Jul 06 14:46:21.362161 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:21.362100    2559 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-derive-3cf8ebc5-866474fcbc-fsdjp" podUID="a2f3a8d3-2879-4672-86a2-afcbe60263d4" containerName="maas-api" containerID="cri-o://07460cd43c3a8b61b8d13da665981829b9f2958f805dd4e2e68fa0436477e448" gracePeriod=30
Jul 06 14:46:21.618284 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:21.618213    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-3cf8ebc5-866474fcbc-fsdjp"
Jul 06 14:46:21.706148 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:21.706114    2559 generic.go:356] "Generic (PLEG): container finished" podID="a2f3a8d3-2879-4672-86a2-afcbe60263d4" containerID="07460cd43c3a8b61b8d13da665981829b9f2958f805dd4e2e68fa0436477e448" exitCode=0
Jul 06 14:46:21.706388 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:21.706157    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-3cf8ebc5-866474fcbc-fsdjp"
Jul 06 14:46:21.706388 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:21.706195    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-3cf8ebc5-866474fcbc-fsdjp" event={"ID":"a2f3a8d3-2879-4672-86a2-afcbe60263d4","Type":"ContainerDied","Data":"07460cd43c3a8b61b8d13da665981829b9f2958f805dd4e2e68fa0436477e448"}
Jul 06 14:46:21.706388 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:21.706242    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-3cf8ebc5-866474fcbc-fsdjp" event={"ID":"a2f3a8d3-2879-4672-86a2-afcbe60263d4","Type":"ContainerDied","Data":"b6ba596479cba3641bdc27781395ec3a1e01406eefd58a7d490c53305aa8728e"}
Jul 06 14:46:21.706388 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:21.706264    2559 scope.go:117] "RemoveContainer" containerID="07460cd43c3a8b61b8d13da665981829b9f2958f805dd4e2e68fa0436477e448"
Jul 06 14:46:21.715090 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:21.715068    2559 scope.go:117] "RemoveContainer" containerID="07460cd43c3a8b61b8d13da665981829b9f2958f805dd4e2e68fa0436477e448"
Jul 06 14:46:21.715347 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:46:21.715328    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"07460cd43c3a8b61b8d13da665981829b9f2958f805dd4e2e68fa0436477e448\": container with ID starting with 07460cd43c3a8b61b8d13da665981829b9f2958f805dd4e2e68fa0436477e448 not found: ID does not exist" containerID="07460cd43c3a8b61b8d13da665981829b9f2958f805dd4e2e68fa0436477e448"
Jul 06 14:46:21.715421 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:21.715361    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"07460cd43c3a8b61b8d13da665981829b9f2958f805dd4e2e68fa0436477e448"} err="failed to get container status \"07460cd43c3a8b61b8d13da665981829b9f2958f805dd4e2e68fa0436477e448\": rpc error: code = NotFound desc = could not find container \"07460cd43c3a8b61b8d13da665981829b9f2958f805dd4e2e68fa0436477e448\": container with ID starting with 07460cd43c3a8b61b8d13da665981829b9f2958f805dd4e2e68fa0436477e448 not found: ID does not exist"
Jul 06 14:46:21.725776 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:21.725739    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6l2tl\" (UniqueName: \"kubernetes.io/projected/a2f3a8d3-2879-4672-86a2-afcbe60263d4-kube-api-access-6l2tl\") pod \"a2f3a8d3-2879-4672-86a2-afcbe60263d4\" (UID: \"a2f3a8d3-2879-4672-86a2-afcbe60263d4\") "
Jul 06 14:46:21.725943 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:21.725784    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a2f3a8d3-2879-4672-86a2-afcbe60263d4-openshift-service-ca\") pod \"a2f3a8d3-2879-4672-86a2-afcbe60263d4\" (UID: \"a2f3a8d3-2879-4672-86a2-afcbe60263d4\") "
Jul 06 14:46:21.725943 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:21.725811    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a2f3a8d3-2879-4672-86a2-afcbe60263d4-maas-api-tls\") pod \"a2f3a8d3-2879-4672-86a2-afcbe60263d4\" (UID: \"a2f3a8d3-2879-4672-86a2-afcbe60263d4\") "
Jul 06 14:46:21.726192 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:21.726165    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a2f3a8d3-2879-4672-86a2-afcbe60263d4-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a2f3a8d3-2879-4672-86a2-afcbe60263d4" (UID: "a2f3a8d3-2879-4672-86a2-afcbe60263d4"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 14:46:21.727933 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:21.727912    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a2f3a8d3-2879-4672-86a2-afcbe60263d4-kube-api-access-6l2tl" (OuterVolumeSpecName: "kube-api-access-6l2tl") pod "a2f3a8d3-2879-4672-86a2-afcbe60263d4" (UID: "a2f3a8d3-2879-4672-86a2-afcbe60263d4"). InnerVolumeSpecName "kube-api-access-6l2tl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 14:46:21.728025 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:21.728005    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a2f3a8d3-2879-4672-86a2-afcbe60263d4-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a2f3a8d3-2879-4672-86a2-afcbe60263d4" (UID: "a2f3a8d3-2879-4672-86a2-afcbe60263d4"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 14:46:21.827045 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:21.827008    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6l2tl\" (UniqueName: \"kubernetes.io/projected/a2f3a8d3-2879-4672-86a2-afcbe60263d4-kube-api-access-6l2tl\") on node \"ip-10-0-133-112.ec2.internal\" DevicePath \"\""
Jul 06 14:46:21.827045 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:21.827039    2559 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a2f3a8d3-2879-4672-86a2-afcbe60263d4-openshift-service-ca\") on node \"ip-10-0-133-112.ec2.internal\" DevicePath \"\""
Jul 06 14:46:21.827045 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:21.827051    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a2f3a8d3-2879-4672-86a2-afcbe60263d4-maas-api-tls\") on node \"ip-10-0-133-112.ec2.internal\" DevicePath \"\""
Jul 06 14:46:22.035841 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:22.035809    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-derive-3cf8ebc5-866474fcbc-fsdjp"]
Jul 06 14:46:22.042362 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:22.042331    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-derive-3cf8ebc5-866474fcbc-fsdjp"]
Jul 06 14:46:22.228508 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:22.228471    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a2f3a8d3-2879-4672-86a2-afcbe60263d4" path="/var/lib/kubelet/pods/a2f3a8d3-2879-4672-86a2-afcbe60263d4/volumes"
Jul 06 14:46:26.253204 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:26.253175    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-3cf8ebc5-openshift-default-65fd87797f-4dv7p"
Jul 06 14:46:26.369841 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:26.369742    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a48781ef-5a49-401d-aa25-4d03b5b2b16c-istiod-ca-cert\") pod \"a48781ef-5a49-401d-aa25-4d03b5b2b16c\" (UID: \"a48781ef-5a49-401d-aa25-4d03b5b2b16c\") "
Jul 06 14:46:26.369841 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:26.369786    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-drgc7\" (UniqueName: \"kubernetes.io/projected/a48781ef-5a49-401d-aa25-4d03b5b2b16c-kube-api-access-drgc7\") pod \"a48781ef-5a49-401d-aa25-4d03b5b2b16c\" (UID: \"a48781ef-5a49-401d-aa25-4d03b5b2b16c\") "
Jul 06 14:46:26.369841 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:26.369807    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a48781ef-5a49-401d-aa25-4d03b5b2b16c-istio-token\") pod \"a48781ef-5a49-401d-aa25-4d03b5b2b16c\" (UID: \"a48781ef-5a49-401d-aa25-4d03b5b2b16c\") "
Jul 06 14:46:26.370140 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:26.369885    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a48781ef-5a49-401d-aa25-4d03b5b2b16c-credential-socket\") pod \"a48781ef-5a49-401d-aa25-4d03b5b2b16c\" (UID: \"a48781ef-5a49-401d-aa25-4d03b5b2b16c\") "
Jul 06 14:46:26.370140 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:26.369941    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a48781ef-5a49-401d-aa25-4d03b5b2b16c-istio-envoy\") pod \"a48781ef-5a49-401d-aa25-4d03b5b2b16c\" (UID: \"a48781ef-5a49-401d-aa25-4d03b5b2b16c\") "
Jul 06 14:46:26.370140 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:26.369971    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a48781ef-5a49-401d-aa25-4d03b5b2b16c-workload-certs\") pod \"a48781ef-5a49-401d-aa25-4d03b5b2b16c\" (UID: \"a48781ef-5a49-401d-aa25-4d03b5b2b16c\") "
Jul 06 14:46:26.370140 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:26.369997    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a48781ef-5a49-401d-aa25-4d03b5b2b16c-workload-socket\") pod \"a48781ef-5a49-401d-aa25-4d03b5b2b16c\" (UID: \"a48781ef-5a49-401d-aa25-4d03b5b2b16c\") "
Jul 06 14:46:26.370140 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:26.370031    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a48781ef-5a49-401d-aa25-4d03b5b2b16c-istio-podinfo\") pod \"a48781ef-5a49-401d-aa25-4d03b5b2b16c\" (UID: \"a48781ef-5a49-401d-aa25-4d03b5b2b16c\") "
Jul 06 14:46:26.370140 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:26.370062    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a48781ef-5a49-401d-aa25-4d03b5b2b16c-istio-data\") pod \"a48781ef-5a49-401d-aa25-4d03b5b2b16c\" (UID: \"a48781ef-5a49-401d-aa25-4d03b5b2b16c\") "
Jul 06 14:46:26.370432 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:26.370248    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a48781ef-5a49-401d-aa25-4d03b5b2b16c-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a48781ef-5a49-401d-aa25-4d03b5b2b16c" (UID: "a48781ef-5a49-401d-aa25-4d03b5b2b16c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 14:46:26.370432 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:26.370257    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a48781ef-5a49-401d-aa25-4d03b5b2b16c-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a48781ef-5a49-401d-aa25-4d03b5b2b16c" (UID: "a48781ef-5a49-401d-aa25-4d03b5b2b16c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 14:46:26.370432 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:26.370257    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a48781ef-5a49-401d-aa25-4d03b5b2b16c-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a48781ef-5a49-401d-aa25-4d03b5b2b16c" (UID: "a48781ef-5a49-401d-aa25-4d03b5b2b16c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 14:46:26.370685 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:26.370477    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a48781ef-5a49-401d-aa25-4d03b5b2b16c-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a48781ef-5a49-401d-aa25-4d03b5b2b16c" (UID: "a48781ef-5a49-401d-aa25-4d03b5b2b16c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 14:46:26.370685 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:26.370488    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a48781ef-5a49-401d-aa25-4d03b5b2b16c-istio-data" (OuterVolumeSpecName: "istio-data") pod "a48781ef-5a49-401d-aa25-4d03b5b2b16c" (UID: "a48781ef-5a49-401d-aa25-4d03b5b2b16c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 14:46:26.370796 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:26.370741    2559 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a48781ef-5a49-401d-aa25-4d03b5b2b16c-workload-certs\") on node \"ip-10-0-133-112.ec2.internal\" DevicePath \"\""
Jul 06 14:46:26.370796 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:26.370766    2559 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a48781ef-5a49-401d-aa25-4d03b5b2b16c-istiod-ca-cert\") on node \"ip-10-0-133-112.ec2.internal\" DevicePath \"\""
Jul 06 14:46:26.370796 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:26.370781    2559 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a48781ef-5a49-401d-aa25-4d03b5b2b16c-credential-socket\") on node \"ip-10-0-133-112.ec2.internal\" DevicePath \"\""
Jul 06 14:46:26.372291 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:26.372262    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a48781ef-5a49-401d-aa25-4d03b5b2b16c-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a48781ef-5a49-401d-aa25-4d03b5b2b16c" (UID: "a48781ef-5a49-401d-aa25-4d03b5b2b16c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 14:46:26.372291 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:26.372283    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a48781ef-5a49-401d-aa25-4d03b5b2b16c-kube-api-access-drgc7" (OuterVolumeSpecName: "kube-api-access-drgc7") pod "a48781ef-5a49-401d-aa25-4d03b5b2b16c" (UID: "a48781ef-5a49-401d-aa25-4d03b5b2b16c"). InnerVolumeSpecName "kube-api-access-drgc7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 14:46:26.372640 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:26.372347    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a48781ef-5a49-401d-aa25-4d03b5b2b16c-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a48781ef-5a49-401d-aa25-4d03b5b2b16c" (UID: "a48781ef-5a49-401d-aa25-4d03b5b2b16c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 06 14:46:26.372640 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:26.372394    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a48781ef-5a49-401d-aa25-4d03b5b2b16c-istio-token" (OuterVolumeSpecName: "istio-token") pod "a48781ef-5a49-401d-aa25-4d03b5b2b16c" (UID: "a48781ef-5a49-401d-aa25-4d03b5b2b16c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 14:46:26.472297 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:26.472257    2559 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a48781ef-5a49-401d-aa25-4d03b5b2b16c-istio-envoy\") on node \"ip-10-0-133-112.ec2.internal\" DevicePath \"\""
Jul 06 14:46:26.472297 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:26.472291    2559 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a48781ef-5a49-401d-aa25-4d03b5b2b16c-workload-socket\") on node \"ip-10-0-133-112.ec2.internal\" DevicePath \"\""
Jul 06 14:46:26.472297 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:26.472301    2559 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a48781ef-5a49-401d-aa25-4d03b5b2b16c-istio-podinfo\") on node \"ip-10-0-133-112.ec2.internal\" DevicePath \"\""
Jul 06 14:46:26.472542 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:26.472309    2559 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a48781ef-5a49-401d-aa25-4d03b5b2b16c-istio-data\") on node \"ip-10-0-133-112.ec2.internal\" DevicePath \"\""
Jul 06 14:46:26.472542 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:26.472318    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-drgc7\" (UniqueName: \"kubernetes.io/projected/a48781ef-5a49-401d-aa25-4d03b5b2b16c-kube-api-access-drgc7\") on node \"ip-10-0-133-112.ec2.internal\" DevicePath \"\""
Jul 06 14:46:26.472542 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:26.472328    2559 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a48781ef-5a49-401d-aa25-4d03b5b2b16c-istio-token\") on node \"ip-10-0-133-112.ec2.internal\" DevicePath \"\""
Jul 06 14:46:26.733211 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:26.733175    2559 generic.go:356] "Generic (PLEG): container finished" podID="a48781ef-5a49-401d-aa25-4d03b5b2b16c" containerID="06c2ead1237dbd1b1f9e271d444850bdfe4c649b97df78b414b9a1833535357f" exitCode=0
Jul 06 14:46:26.733385 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:26.733227    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-3cf8ebc5-openshift-default-65fd87797f-4dv7p"
Jul 06 14:46:26.733385 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:26.733242    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-3cf8ebc5-openshift-default-65fd87797f-4dv7p" event={"ID":"a48781ef-5a49-401d-aa25-4d03b5b2b16c","Type":"ContainerDied","Data":"06c2ead1237dbd1b1f9e271d444850bdfe4c649b97df78b414b9a1833535357f"}
Jul 06 14:46:26.733385 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:26.733287    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-3cf8ebc5-openshift-default-65fd87797f-4dv7p" event={"ID":"a48781ef-5a49-401d-aa25-4d03b5b2b16c","Type":"ContainerDied","Data":"8174a246e0d0c5892821cc26ebcaee0593aba86acec0da575d15d950dc271098"}
Jul 06 14:46:26.733385 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:26.733309    2559 scope.go:117] "RemoveContainer" containerID="06c2ead1237dbd1b1f9e271d444850bdfe4c649b97df78b414b9a1833535357f"
Jul 06 14:46:26.742883 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:26.742864    2559 scope.go:117] "RemoveContainer" containerID="06c2ead1237dbd1b1f9e271d444850bdfe4c649b97df78b414b9a1833535357f"
Jul 06 14:46:26.743205 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:46:26.743186    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"06c2ead1237dbd1b1f9e271d444850bdfe4c649b97df78b414b9a1833535357f\": container with ID starting with 06c2ead1237dbd1b1f9e271d444850bdfe4c649b97df78b414b9a1833535357f not found: ID does not exist" containerID="06c2ead1237dbd1b1f9e271d444850bdfe4c649b97df78b414b9a1833535357f"
Jul 06 14:46:26.743266 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:26.743215    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"06c2ead1237dbd1b1f9e271d444850bdfe4c649b97df78b414b9a1833535357f"} err="failed to get container status \"06c2ead1237dbd1b1f9e271d444850bdfe4c649b97df78b414b9a1833535357f\": rpc error: code = NotFound desc = could not find container \"06c2ead1237dbd1b1f9e271d444850bdfe4c649b97df78b414b9a1833535357f\": container with ID starting with 06c2ead1237dbd1b1f9e271d444850bdfe4c649b97df78b414b9a1833535357f not found: ID does not exist"
Jul 06 14:46:26.760560 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:26.760520    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-3cf8ebc5-openshift-default-65fd87797f-4dv7p"]
Jul 06 14:46:26.764907 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:26.764875    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-3cf8ebc5-openshift-default-65fd87797f-4dv7p"]
Jul 06 14:46:28.227900 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:46:28.227861    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a48781ef-5a49-401d-aa25-4d03b5b2b16c" path="/var/lib/kubelet/pods/a48781ef-5a49-401d-aa25-4d03b5b2b16c/volumes"
Jul 06 14:48:10.515476 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:10.515391    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-4dff23b7-openshift-default-c8d66bb55-s5cl4"]
Jul 06 14:48:10.515986 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:10.515836    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a48781ef-5a49-401d-aa25-4d03b5b2b16c" containerName="istio-proxy"
Jul 06 14:48:10.515986 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:10.515851    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="a48781ef-5a49-401d-aa25-4d03b5b2b16c" containerName="istio-proxy"
Jul 06 14:48:10.515986 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:10.515877    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a2f3a8d3-2879-4672-86a2-afcbe60263d4" containerName="maas-api"
Jul 06 14:48:10.515986 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:10.515883    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="a2f3a8d3-2879-4672-86a2-afcbe60263d4" containerName="maas-api"
Jul 06 14:48:10.515986 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:10.515953    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="a2f3a8d3-2879-4672-86a2-afcbe60263d4" containerName="maas-api"
Jul 06 14:48:10.515986 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:10.515962    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="a48781ef-5a49-401d-aa25-4d03b5b2b16c" containerName="istio-proxy"
Jul 06 14:48:10.519087 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:10.519066    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-4dff23b7-openshift-default-c8d66bb55-s5cl4"
Jul 06 14:48:10.522552 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:10.522530    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-4dff23b7-openshift-default-dockercfg-g5v8v\""
Jul 06 14:48:10.547658 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:10.547615    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-4dff23b7-openshift-default-c8d66bb55-s5cl4"]
Jul 06 14:48:10.635444 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:10.635362    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f170ceef-fe2c-4890-baf9-a4589f08e27b-credential-socket\") pod \"e2e-mt-4dff23b7-openshift-default-c8d66bb55-s5cl4\" (UID: \"f170ceef-fe2c-4890-baf9-a4589f08e27b\") " pod="openshift-ingress/e2e-mt-4dff23b7-openshift-default-c8d66bb55-s5cl4"
Jul 06 14:48:10.635684 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:10.635464    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f170ceef-fe2c-4890-baf9-a4589f08e27b-istiod-ca-cert\") pod \"e2e-mt-4dff23b7-openshift-default-c8d66bb55-s5cl4\" (UID: \"f170ceef-fe2c-4890-baf9-a4589f08e27b\") " pod="openshift-ingress/e2e-mt-4dff23b7-openshift-default-c8d66bb55-s5cl4"
Jul 06 14:48:10.635684 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:10.635522    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f170ceef-fe2c-4890-baf9-a4589f08e27b-istio-data\") pod \"e2e-mt-4dff23b7-openshift-default-c8d66bb55-s5cl4\" (UID: \"f170ceef-fe2c-4890-baf9-a4589f08e27b\") " pod="openshift-ingress/e2e-mt-4dff23b7-openshift-default-c8d66bb55-s5cl4"
Jul 06 14:48:10.635684 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:10.635588    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v5k8d\" (UniqueName: \"kubernetes.io/projected/f170ceef-fe2c-4890-baf9-a4589f08e27b-kube-api-access-v5k8d\") pod \"e2e-mt-4dff23b7-openshift-default-c8d66bb55-s5cl4\" (UID: \"f170ceef-fe2c-4890-baf9-a4589f08e27b\") " pod="openshift-ingress/e2e-mt-4dff23b7-openshift-default-c8d66bb55-s5cl4"
Jul 06 14:48:10.635684 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:10.635642    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f170ceef-fe2c-4890-baf9-a4589f08e27b-istio-token\") pod \"e2e-mt-4dff23b7-openshift-default-c8d66bb55-s5cl4\" (UID: \"f170ceef-fe2c-4890-baf9-a4589f08e27b\") " pod="openshift-ingress/e2e-mt-4dff23b7-openshift-default-c8d66bb55-s5cl4"
Jul 06 14:48:10.635920 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:10.635695    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f170ceef-fe2c-4890-baf9-a4589f08e27b-workload-socket\") pod \"e2e-mt-4dff23b7-openshift-default-c8d66bb55-s5cl4\" (UID: \"f170ceef-fe2c-4890-baf9-a4589f08e27b\") " pod="openshift-ingress/e2e-mt-4dff23b7-openshift-default-c8d66bb55-s5cl4"
Jul 06 14:48:10.635920 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:10.635724    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f170ceef-fe2c-4890-baf9-a4589f08e27b-istio-envoy\") pod \"e2e-mt-4dff23b7-openshift-default-c8d66bb55-s5cl4\" (UID: \"f170ceef-fe2c-4890-baf9-a4589f08e27b\") " pod="openshift-ingress/e2e-mt-4dff23b7-openshift-default-c8d66bb55-s5cl4"
Jul 06 14:48:10.635920 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:10.635840    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f170ceef-fe2c-4890-baf9-a4589f08e27b-istio-podinfo\") pod \"e2e-mt-4dff23b7-openshift-default-c8d66bb55-s5cl4\" (UID: \"f170ceef-fe2c-4890-baf9-a4589f08e27b\") " pod="openshift-ingress/e2e-mt-4dff23b7-openshift-default-c8d66bb55-s5cl4"
Jul 06 14:48:10.635920 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:10.635875    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f170ceef-fe2c-4890-baf9-a4589f08e27b-workload-certs\") pod \"e2e-mt-4dff23b7-openshift-default-c8d66bb55-s5cl4\" (UID: \"f170ceef-fe2c-4890-baf9-a4589f08e27b\") " pod="openshift-ingress/e2e-mt-4dff23b7-openshift-default-c8d66bb55-s5cl4"
Jul 06 14:48:10.736530 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:10.736487    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v5k8d\" (UniqueName: \"kubernetes.io/projected/f170ceef-fe2c-4890-baf9-a4589f08e27b-kube-api-access-v5k8d\") pod \"e2e-mt-4dff23b7-openshift-default-c8d66bb55-s5cl4\" (UID: \"f170ceef-fe2c-4890-baf9-a4589f08e27b\") " pod="openshift-ingress/e2e-mt-4dff23b7-openshift-default-c8d66bb55-s5cl4"
Jul 06 14:48:10.736764 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:10.736542    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f170ceef-fe2c-4890-baf9-a4589f08e27b-istio-token\") pod \"e2e-mt-4dff23b7-openshift-default-c8d66bb55-s5cl4\" (UID: \"f170ceef-fe2c-4890-baf9-a4589f08e27b\") " pod="openshift-ingress/e2e-mt-4dff23b7-openshift-default-c8d66bb55-s5cl4"
Jul 06 14:48:10.736764 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:10.736605    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f170ceef-fe2c-4890-baf9-a4589f08e27b-workload-socket\") pod \"e2e-mt-4dff23b7-openshift-default-c8d66bb55-s5cl4\" (UID: \"f170ceef-fe2c-4890-baf9-a4589f08e27b\") " pod="openshift-ingress/e2e-mt-4dff23b7-openshift-default-c8d66bb55-s5cl4"
Jul 06 14:48:10.736764 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:10.736625    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f170ceef-fe2c-4890-baf9-a4589f08e27b-istio-envoy\") pod \"e2e-mt-4dff23b7-openshift-default-c8d66bb55-s5cl4\" (UID: \"f170ceef-fe2c-4890-baf9-a4589f08e27b\") " pod="openshift-ingress/e2e-mt-4dff23b7-openshift-default-c8d66bb55-s5cl4"
Jul 06 14:48:10.736764 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:10.736676    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f170ceef-fe2c-4890-baf9-a4589f08e27b-istio-podinfo\") pod \"e2e-mt-4dff23b7-openshift-default-c8d66bb55-s5cl4\" (UID: \"f170ceef-fe2c-4890-baf9-a4589f08e27b\") " pod="openshift-ingress/e2e-mt-4dff23b7-openshift-default-c8d66bb55-s5cl4"
Jul 06 14:48:10.736764 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:10.736703    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f170ceef-fe2c-4890-baf9-a4589f08e27b-workload-certs\") pod \"e2e-mt-4dff23b7-openshift-default-c8d66bb55-s5cl4\" (UID: \"f170ceef-fe2c-4890-baf9-a4589f08e27b\") " pod="openshift-ingress/e2e-mt-4dff23b7-openshift-default-c8d66bb55-s5cl4"
Jul 06 14:48:10.736764 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:10.736743    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f170ceef-fe2c-4890-baf9-a4589f08e27b-credential-socket\") pod \"e2e-mt-4dff23b7-openshift-default-c8d66bb55-s5cl4\" (UID: \"f170ceef-fe2c-4890-baf9-a4589f08e27b\") " pod="openshift-ingress/e2e-mt-4dff23b7-openshift-default-c8d66bb55-s5cl4"
Jul 06 14:48:10.737104 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:10.736770    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f170ceef-fe2c-4890-baf9-a4589f08e27b-istiod-ca-cert\") pod \"e2e-mt-4dff23b7-openshift-default-c8d66bb55-s5cl4\" (UID: \"f170ceef-fe2c-4890-baf9-a4589f08e27b\") " pod="openshift-ingress/e2e-mt-4dff23b7-openshift-default-c8d66bb55-s5cl4"
Jul 06 14:48:10.737104 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:10.736805    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f170ceef-fe2c-4890-baf9-a4589f08e27b-istio-data\") pod \"e2e-mt-4dff23b7-openshift-default-c8d66bb55-s5cl4\" (UID: \"f170ceef-fe2c-4890-baf9-a4589f08e27b\") " pod="openshift-ingress/e2e-mt-4dff23b7-openshift-default-c8d66bb55-s5cl4"
Jul 06 14:48:10.737104 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:10.737058    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f170ceef-fe2c-4890-baf9-a4589f08e27b-workload-socket\") pod \"e2e-mt-4dff23b7-openshift-default-c8d66bb55-s5cl4\" (UID: \"f170ceef-fe2c-4890-baf9-a4589f08e27b\") " pod="openshift-ingress/e2e-mt-4dff23b7-openshift-default-c8d66bb55-s5cl4"
Jul 06 14:48:10.737104 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:10.737095    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f170ceef-fe2c-4890-baf9-a4589f08e27b-workload-certs\") pod \"e2e-mt-4dff23b7-openshift-default-c8d66bb55-s5cl4\" (UID: \"f170ceef-fe2c-4890-baf9-a4589f08e27b\") " pod="openshift-ingress/e2e-mt-4dff23b7-openshift-default-c8d66bb55-s5cl4"
Jul 06 14:48:10.737281 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:10.737221    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f170ceef-fe2c-4890-baf9-a4589f08e27b-credential-socket\") pod \"e2e-mt-4dff23b7-openshift-default-c8d66bb55-s5cl4\" (UID: \"f170ceef-fe2c-4890-baf9-a4589f08e27b\") " pod="openshift-ingress/e2e-mt-4dff23b7-openshift-default-c8d66bb55-s5cl4"
Jul 06 14:48:10.737281 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:10.737264    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f170ceef-fe2c-4890-baf9-a4589f08e27b-istio-data\") pod \"e2e-mt-4dff23b7-openshift-default-c8d66bb55-s5cl4\" (UID: \"f170ceef-fe2c-4890-baf9-a4589f08e27b\") " pod="openshift-ingress/e2e-mt-4dff23b7-openshift-default-c8d66bb55-s5cl4"
Jul 06 14:48:10.737607 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:10.737584    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f170ceef-fe2c-4890-baf9-a4589f08e27b-istiod-ca-cert\") pod \"e2e-mt-4dff23b7-openshift-default-c8d66bb55-s5cl4\" (UID: \"f170ceef-fe2c-4890-baf9-a4589f08e27b\") " pod="openshift-ingress/e2e-mt-4dff23b7-openshift-default-c8d66bb55-s5cl4"
Jul 06 14:48:10.739113 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:10.739090    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f170ceef-fe2c-4890-baf9-a4589f08e27b-istio-envoy\") pod \"e2e-mt-4dff23b7-openshift-default-c8d66bb55-s5cl4\" (UID: \"f170ceef-fe2c-4890-baf9-a4589f08e27b\") " pod="openshift-ingress/e2e-mt-4dff23b7-openshift-default-c8d66bb55-s5cl4"
Jul 06 14:48:10.739361 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:10.739342    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f170ceef-fe2c-4890-baf9-a4589f08e27b-istio-podinfo\") pod \"e2e-mt-4dff23b7-openshift-default-c8d66bb55-s5cl4\" (UID: \"f170ceef-fe2c-4890-baf9-a4589f08e27b\") " pod="openshift-ingress/e2e-mt-4dff23b7-openshift-default-c8d66bb55-s5cl4"
Jul 06 14:48:10.745925 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:10.745894    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f170ceef-fe2c-4890-baf9-a4589f08e27b-istio-token\") pod \"e2e-mt-4dff23b7-openshift-default-c8d66bb55-s5cl4\" (UID: \"f170ceef-fe2c-4890-baf9-a4589f08e27b\") " pod="openshift-ingress/e2e-mt-4dff23b7-openshift-default-c8d66bb55-s5cl4"
Jul 06 14:48:10.746131 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:10.746106    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v5k8d\" (UniqueName: \"kubernetes.io/projected/f170ceef-fe2c-4890-baf9-a4589f08e27b-kube-api-access-v5k8d\") pod \"e2e-mt-4dff23b7-openshift-default-c8d66bb55-s5cl4\" (UID: \"f170ceef-fe2c-4890-baf9-a4589f08e27b\") " pod="openshift-ingress/e2e-mt-4dff23b7-openshift-default-c8d66bb55-s5cl4"
Jul 06 14:48:10.829855 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:10.829754    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-4dff23b7-openshift-default-c8d66bb55-s5cl4"
Jul 06 14:48:10.986389 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:10.986357    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-4dff23b7-openshift-default-c8d66bb55-s5cl4"]
Jul 06 14:48:10.988612 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:48:10.988560    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf170ceef_fe2c_4890_baf9_a4589f08e27b.slice/crio-72676ea783e2e7d0a93183c31c18628a2b07c99d05648613961353918833ac43 WatchSource:0}: Error finding container 72676ea783e2e7d0a93183c31c18628a2b07c99d05648613961353918833ac43: Status 404 returned error can't find the container with id 72676ea783e2e7d0a93183c31c18628a2b07c99d05648613961353918833ac43
Jul 06 14:48:10.991157 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:10.991119    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 06 14:48:10.991276 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:10.991193    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 06 14:48:10.991276 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:10.991227    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 06 14:48:11.250460 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:11.250421    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4dff23b7-openshift-default-c8d66bb55-s5cl4" event={"ID":"f170ceef-fe2c-4890-baf9-a4589f08e27b","Type":"ContainerStarted","Data":"13b054ab1d71b84029a30d9f5e8d07185a37dd5067d91fddd9a54af9f965a17c"}
Jul 06 14:48:11.250674 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:11.250480    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4dff23b7-openshift-default-c8d66bb55-s5cl4" event={"ID":"f170ceef-fe2c-4890-baf9-a4589f08e27b","Type":"ContainerStarted","Data":"72676ea783e2e7d0a93183c31c18628a2b07c99d05648613961353918833ac43"}
Jul 06 14:48:11.273288 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:11.273222    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-4dff23b7-openshift-default-c8d66bb55-s5cl4" podStartSLOduration=1.273198662 podStartE2EDuration="1.273198662s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:48:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:48:11.27163459 +0000 UTC m=+1713.578926672" watchObservedRunningTime="2026-07-06 14:48:11.273198662 +0000 UTC m=+1713.580490744"
Jul 06 14:48:11.830166 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:11.830128    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-4dff23b7-openshift-default-c8d66bb55-s5cl4"
Jul 06 14:48:12.835094 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:12.835066    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-4dff23b7-openshift-default-c8d66bb55-s5cl4"
Jul 06 14:48:13.259239 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:13.259202    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-4dff23b7-openshift-default-c8d66bb55-s5cl4"
Jul 06 14:48:13.260099 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:13.260079    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-4dff23b7-openshift-default-c8d66bb55-s5cl4"
Jul 06 14:48:13.560174 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:13.560081    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-mt-4dff23b7-c767c4f76-n58jj"]
Jul 06 14:48:13.564167 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:13.564140    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-4dff23b7-c767c4f76-n58jj"
Jul 06 14:48:13.567178 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:13.567157    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-mt-4dff23b7-serving-cert\""
Jul 06 14:48:13.576309 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:13.576285    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-4dff23b7-c767c4f76-n58jj"]
Jul 06 14:48:13.665583 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:13.665529    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b6ed843f-e102-4fa5-8abf-b180e831e7d7-openshift-service-ca\") pod \"maas-api-e2e-mt-4dff23b7-c767c4f76-n58jj\" (UID: \"b6ed843f-e102-4fa5-8abf-b180e831e7d7\") " pod="opendatahub/maas-api-e2e-mt-4dff23b7-c767c4f76-n58jj"
Jul 06 14:48:13.665750 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:13.665660    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sldw6\" (UniqueName: \"kubernetes.io/projected/b6ed843f-e102-4fa5-8abf-b180e831e7d7-kube-api-access-sldw6\") pod \"maas-api-e2e-mt-4dff23b7-c767c4f76-n58jj\" (UID: \"b6ed843f-e102-4fa5-8abf-b180e831e7d7\") " pod="opendatahub/maas-api-e2e-mt-4dff23b7-c767c4f76-n58jj"
Jul 06 14:48:13.665750 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:13.665693    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b6ed843f-e102-4fa5-8abf-b180e831e7d7-maas-api-tls\") pod \"maas-api-e2e-mt-4dff23b7-c767c4f76-n58jj\" (UID: \"b6ed843f-e102-4fa5-8abf-b180e831e7d7\") " pod="opendatahub/maas-api-e2e-mt-4dff23b7-c767c4f76-n58jj"
Jul 06 14:48:13.766399 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:13.766360    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b6ed843f-e102-4fa5-8abf-b180e831e7d7-openshift-service-ca\") pod \"maas-api-e2e-mt-4dff23b7-c767c4f76-n58jj\" (UID: \"b6ed843f-e102-4fa5-8abf-b180e831e7d7\") " pod="opendatahub/maas-api-e2e-mt-4dff23b7-c767c4f76-n58jj"
Jul 06 14:48:13.766639 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:13.766461    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sldw6\" (UniqueName: \"kubernetes.io/projected/b6ed843f-e102-4fa5-8abf-b180e831e7d7-kube-api-access-sldw6\") pod \"maas-api-e2e-mt-4dff23b7-c767c4f76-n58jj\" (UID: \"b6ed843f-e102-4fa5-8abf-b180e831e7d7\") " pod="opendatahub/maas-api-e2e-mt-4dff23b7-c767c4f76-n58jj"
Jul 06 14:48:13.766639 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:13.766587    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b6ed843f-e102-4fa5-8abf-b180e831e7d7-maas-api-tls\") pod \"maas-api-e2e-mt-4dff23b7-c767c4f76-n58jj\" (UID: \"b6ed843f-e102-4fa5-8abf-b180e831e7d7\") " pod="opendatahub/maas-api-e2e-mt-4dff23b7-c767c4f76-n58jj"
Jul 06 14:48:13.766761 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:48:13.766689    2559 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-mt-4dff23b7-serving-cert: secret "maas-api-e2e-mt-4dff23b7-serving-cert" not found
Jul 06 14:48:13.766800 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:48:13.766765    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b6ed843f-e102-4fa5-8abf-b180e831e7d7-maas-api-tls podName:b6ed843f-e102-4fa5-8abf-b180e831e7d7 nodeName:}" failed. No retries permitted until 2026-07-06 14:48:14.266744358 +0000 UTC m=+1716.574036422 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/b6ed843f-e102-4fa5-8abf-b180e831e7d7-maas-api-tls") pod "maas-api-e2e-mt-4dff23b7-c767c4f76-n58jj" (UID: "b6ed843f-e102-4fa5-8abf-b180e831e7d7") : secret "maas-api-e2e-mt-4dff23b7-serving-cert" not found
Jul 06 14:48:13.767157 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:13.767139    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b6ed843f-e102-4fa5-8abf-b180e831e7d7-openshift-service-ca\") pod \"maas-api-e2e-mt-4dff23b7-c767c4f76-n58jj\" (UID: \"b6ed843f-e102-4fa5-8abf-b180e831e7d7\") " pod="opendatahub/maas-api-e2e-mt-4dff23b7-c767c4f76-n58jj"
Jul 06 14:48:13.782596 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:13.782547    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sldw6\" (UniqueName: \"kubernetes.io/projected/b6ed843f-e102-4fa5-8abf-b180e831e7d7-kube-api-access-sldw6\") pod \"maas-api-e2e-mt-4dff23b7-c767c4f76-n58jj\" (UID: \"b6ed843f-e102-4fa5-8abf-b180e831e7d7\") " pod="opendatahub/maas-api-e2e-mt-4dff23b7-c767c4f76-n58jj"
Jul 06 14:48:14.271881 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:14.271852    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b6ed843f-e102-4fa5-8abf-b180e831e7d7-maas-api-tls\") pod \"maas-api-e2e-mt-4dff23b7-c767c4f76-n58jj\" (UID: \"b6ed843f-e102-4fa5-8abf-b180e831e7d7\") " pod="opendatahub/maas-api-e2e-mt-4dff23b7-c767c4f76-n58jj"
Jul 06 14:48:14.272281 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:48:14.272000    2559 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-mt-4dff23b7-serving-cert: secret "maas-api-e2e-mt-4dff23b7-serving-cert" not found
Jul 06 14:48:14.272281 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:48:14.272060    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b6ed843f-e102-4fa5-8abf-b180e831e7d7-maas-api-tls podName:b6ed843f-e102-4fa5-8abf-b180e831e7d7 nodeName:}" failed. No retries permitted until 2026-07-06 14:48:15.272044456 +0000 UTC m=+1717.579336517 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/b6ed843f-e102-4fa5-8abf-b180e831e7d7-maas-api-tls") pod "maas-api-e2e-mt-4dff23b7-c767c4f76-n58jj" (UID: "b6ed843f-e102-4fa5-8abf-b180e831e7d7") : secret "maas-api-e2e-mt-4dff23b7-serving-cert" not found
Jul 06 14:48:15.281189 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:15.281157    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b6ed843f-e102-4fa5-8abf-b180e831e7d7-maas-api-tls\") pod \"maas-api-e2e-mt-4dff23b7-c767c4f76-n58jj\" (UID: \"b6ed843f-e102-4fa5-8abf-b180e831e7d7\") " pod="opendatahub/maas-api-e2e-mt-4dff23b7-c767c4f76-n58jj"
Jul 06 14:48:15.281622 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:48:15.281305    2559 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-mt-4dff23b7-serving-cert: secret "maas-api-e2e-mt-4dff23b7-serving-cert" not found
Jul 06 14:48:15.281622 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:48:15.281372    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b6ed843f-e102-4fa5-8abf-b180e831e7d7-maas-api-tls podName:b6ed843f-e102-4fa5-8abf-b180e831e7d7 nodeName:}" failed. No retries permitted until 2026-07-06 14:48:17.281357289 +0000 UTC m=+1719.588649350 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/b6ed843f-e102-4fa5-8abf-b180e831e7d7-maas-api-tls") pod "maas-api-e2e-mt-4dff23b7-c767c4f76-n58jj" (UID: "b6ed843f-e102-4fa5-8abf-b180e831e7d7") : secret "maas-api-e2e-mt-4dff23b7-serving-cert" not found
Jul 06 14:48:17.302173 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:17.302139    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b6ed843f-e102-4fa5-8abf-b180e831e7d7-maas-api-tls\") pod \"maas-api-e2e-mt-4dff23b7-c767c4f76-n58jj\" (UID: \"b6ed843f-e102-4fa5-8abf-b180e831e7d7\") " pod="opendatahub/maas-api-e2e-mt-4dff23b7-c767c4f76-n58jj"
Jul 06 14:48:17.302566 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:48:17.302283    2559 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-mt-4dff23b7-serving-cert: secret "maas-api-e2e-mt-4dff23b7-serving-cert" not found
Jul 06 14:48:17.302566 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:48:17.302343    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b6ed843f-e102-4fa5-8abf-b180e831e7d7-maas-api-tls podName:b6ed843f-e102-4fa5-8abf-b180e831e7d7 nodeName:}" failed. No retries permitted until 2026-07-06 14:48:21.302328481 +0000 UTC m=+1723.609620541 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/b6ed843f-e102-4fa5-8abf-b180e831e7d7-maas-api-tls") pod "maas-api-e2e-mt-4dff23b7-c767c4f76-n58jj" (UID: "b6ed843f-e102-4fa5-8abf-b180e831e7d7") : secret "maas-api-e2e-mt-4dff23b7-serving-cert" not found
Jul 06 14:48:21.341051 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:21.341010    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b6ed843f-e102-4fa5-8abf-b180e831e7d7-maas-api-tls\") pod \"maas-api-e2e-mt-4dff23b7-c767c4f76-n58jj\" (UID: \"b6ed843f-e102-4fa5-8abf-b180e831e7d7\") " pod="opendatahub/maas-api-e2e-mt-4dff23b7-c767c4f76-n58jj"
Jul 06 14:48:21.341444 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:48:21.341168    2559 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-mt-4dff23b7-serving-cert: secret "maas-api-e2e-mt-4dff23b7-serving-cert" not found
Jul 06 14:48:21.341444 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:48:21.341242    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b6ed843f-e102-4fa5-8abf-b180e831e7d7-maas-api-tls podName:b6ed843f-e102-4fa5-8abf-b180e831e7d7 nodeName:}" failed. No retries permitted until 2026-07-06 14:48:29.341220924 +0000 UTC m=+1731.648512993 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/b6ed843f-e102-4fa5-8abf-b180e831e7d7-maas-api-tls") pod "maas-api-e2e-mt-4dff23b7-c767c4f76-n58jj" (UID: "b6ed843f-e102-4fa5-8abf-b180e831e7d7") : secret "maas-api-e2e-mt-4dff23b7-serving-cert" not found
Jul 06 14:48:27.118319 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:27.118271    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-4dff23b7-c767c4f76-n58jj"]
Jul 06 14:48:27.118847 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:48:27.118720    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[maas-api-tls], unattached volumes=[], failed to process volumes=[]: context canceled" pod="opendatahub/maas-api-e2e-mt-4dff23b7-c767c4f76-n58jj" podUID="b6ed843f-e102-4fa5-8abf-b180e831e7d7"
Jul 06 14:48:27.340718 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:27.340629    2559 generic.go:356] "Generic (PLEG): container finished" podID="f170ceef-fe2c-4890-baf9-a4589f08e27b" containerID="13b054ab1d71b84029a30d9f5e8d07185a37dd5067d91fddd9a54af9f965a17c" exitCode=0
Jul 06 14:48:27.340718 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:27.340705    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4dff23b7-openshift-default-c8d66bb55-s5cl4" event={"ID":"f170ceef-fe2c-4890-baf9-a4589f08e27b","Type":"ContainerDied","Data":"13b054ab1d71b84029a30d9f5e8d07185a37dd5067d91fddd9a54af9f965a17c"}
Jul 06 14:48:27.340915 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:27.340717    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-4dff23b7-c767c4f76-n58jj"
Jul 06 14:48:27.341237 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:27.341219    2559 scope.go:117] "RemoveContainer" containerID="13b054ab1d71b84029a30d9f5e8d07185a37dd5067d91fddd9a54af9f965a17c"
Jul 06 14:48:27.342518 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:27.342487    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 06 14:48:27.342611 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:27.342592    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 06 14:48:27.342675 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:27.342660    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 06 14:48:27.347989 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:27.347966    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-4dff23b7-c767c4f76-n58jj"
Jul 06 14:48:27.500721 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:27.500683    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b6ed843f-e102-4fa5-8abf-b180e831e7d7-openshift-service-ca\") pod \"b6ed843f-e102-4fa5-8abf-b180e831e7d7\" (UID: \"b6ed843f-e102-4fa5-8abf-b180e831e7d7\") "
Jul 06 14:48:27.500721 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:27.500724    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sldw6\" (UniqueName: \"kubernetes.io/projected/b6ed843f-e102-4fa5-8abf-b180e831e7d7-kube-api-access-sldw6\") pod \"b6ed843f-e102-4fa5-8abf-b180e831e7d7\" (UID: \"b6ed843f-e102-4fa5-8abf-b180e831e7d7\") "
Jul 06 14:48:27.501184 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:27.501142    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b6ed843f-e102-4fa5-8abf-b180e831e7d7-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b6ed843f-e102-4fa5-8abf-b180e831e7d7" (UID: "b6ed843f-e102-4fa5-8abf-b180e831e7d7"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 14:48:27.503238 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:27.503214    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b6ed843f-e102-4fa5-8abf-b180e831e7d7-kube-api-access-sldw6" (OuterVolumeSpecName: "kube-api-access-sldw6") pod "b6ed843f-e102-4fa5-8abf-b180e831e7d7" (UID: "b6ed843f-e102-4fa5-8abf-b180e831e7d7"). InnerVolumeSpecName "kube-api-access-sldw6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 14:48:27.602415 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:27.602326    2559 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b6ed843f-e102-4fa5-8abf-b180e831e7d7-openshift-service-ca\") on node \"ip-10-0-133-112.ec2.internal\" DevicePath \"\""
Jul 06 14:48:27.602415 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:27.602361    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sldw6\" (UniqueName: \"kubernetes.io/projected/b6ed843f-e102-4fa5-8abf-b180e831e7d7-kube-api-access-sldw6\") on node \"ip-10-0-133-112.ec2.internal\" DevicePath \"\""
Jul 06 14:48:28.259768 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:28.259733    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-4dff23b7-openshift-default-c8d66bb55-s5cl4"
Jul 06 14:48:28.347368 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:28.347329    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4dff23b7-openshift-default-c8d66bb55-s5cl4" event={"ID":"f170ceef-fe2c-4890-baf9-a4589f08e27b","Type":"ContainerStarted","Data":"ed50ed4dee52ef802815cd39793b1031c35429f2a5eec51e9446bcc8f20d84fe"}
Jul 06 14:48:28.347559 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:28.347342    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-4dff23b7-c767c4f76-n58jj"
Jul 06 14:48:28.426220 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:28.426180    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-4dff23b7-c767c4f76-n58jj"]
Jul 06 14:48:28.431906 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:28.431863    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-mt-4dff23b7-c767c4f76-n58jj"]
Jul 06 14:48:28.612827 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:28.612733    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b6ed843f-e102-4fa5-8abf-b180e831e7d7-maas-api-tls\") on node \"ip-10-0-133-112.ec2.internal\" DevicePath \"\""
Jul 06 14:48:28.830898 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:28.830860    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-4dff23b7-openshift-default-c8d66bb55-s5cl4"
Jul 06 14:48:28.831914 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:28.831880    2559 patch_prober.go:28] interesting pod/e2e-mt-4dff23b7-openshift-default-c8d66bb55-s5cl4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body=
Jul 06 14:48:28.832052 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:28.831956    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-4dff23b7-openshift-default-c8d66bb55-s5cl4" podUID="f170ceef-fe2c-4890-baf9-a4589f08e27b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused"
Jul 06 14:48:29.830799 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:29.830754    2559 patch_prober.go:28] interesting pod/e2e-mt-4dff23b7-openshift-default-c8d66bb55-s5cl4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body=
Jul 06 14:48:29.831325 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:29.830823    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-4dff23b7-openshift-default-c8d66bb55-s5cl4" podUID="f170ceef-fe2c-4890-baf9-a4589f08e27b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused"
Jul 06 14:48:30.226772 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:30.226730    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b6ed843f-e102-4fa5-8abf-b180e831e7d7" path="/var/lib/kubelet/pods/b6ed843f-e102-4fa5-8abf-b180e831e7d7/volumes"
Jul 06 14:48:30.831143 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:30.831102    2559 patch_prober.go:28] interesting pod/e2e-mt-4dff23b7-openshift-default-c8d66bb55-s5cl4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body=
Jul 06 14:48:30.831556 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:30.831206    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-4dff23b7-openshift-default-c8d66bb55-s5cl4" podUID="f170ceef-fe2c-4890-baf9-a4589f08e27b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused"
Jul 06 14:48:31.830436 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:31.830395    2559 patch_prober.go:28] interesting pod/e2e-mt-4dff23b7-openshift-default-c8d66bb55-s5cl4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body=
Jul 06 14:48:31.830669 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:31.830474    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-4dff23b7-openshift-default-c8d66bb55-s5cl4" podUID="f170ceef-fe2c-4890-baf9-a4589f08e27b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused"
Jul 06 14:48:32.835133 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:32.835101    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-4dff23b7-openshift-default-c8d66bb55-s5cl4"
Jul 06 14:48:32.835528 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:32.835462    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-4dff23b7-openshift-default-c8d66bb55-s5cl4"
Jul 06 14:48:32.836211 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:32.836193    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-4dff23b7-openshift-default-c8d66bb55-s5cl4"
Jul 06 14:48:33.567275 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:33.567234    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-4dff23b7-openshift-default-c8d66bb55-s5cl4"]
Jul 06 14:48:34.376060 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:34.376024    2559 kubelet_pods.go:1082] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/e2e-mt-4dff23b7-openshift-default-c8d66bb55-s5cl4" secret="" err="secret \"e2e-mt-4dff23b7-openshift-default-dockercfg-g5v8v\" not found"
Jul 06 14:48:35.380493 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:35.380443    2559 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-4dff23b7-openshift-default-c8d66bb55-s5cl4" podUID="f170ceef-fe2c-4890-baf9-a4589f08e27b" containerName="istio-proxy" containerID="cri-o://ed50ed4dee52ef802815cd39793b1031c35429f2a5eec51e9446bcc8f20d84fe" gracePeriod=30
Jul 06 14:48:40.632836 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:40.632804    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-4dff23b7-openshift-default-c8d66bb55-s5cl4"
Jul 06 14:48:40.724288 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:40.724250    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f170ceef-fe2c-4890-baf9-a4589f08e27b-istio-envoy\") pod \"f170ceef-fe2c-4890-baf9-a4589f08e27b\" (UID: \"f170ceef-fe2c-4890-baf9-a4589f08e27b\") "
Jul 06 14:48:40.724509 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:40.724485    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f170ceef-fe2c-4890-baf9-a4589f08e27b-workload-socket\") pod \"f170ceef-fe2c-4890-baf9-a4589f08e27b\" (UID: \"f170ceef-fe2c-4890-baf9-a4589f08e27b\") "
Jul 06 14:48:40.724606 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:40.724559    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f170ceef-fe2c-4890-baf9-a4589f08e27b-istiod-ca-cert\") pod \"f170ceef-fe2c-4890-baf9-a4589f08e27b\" (UID: \"f170ceef-fe2c-4890-baf9-a4589f08e27b\") "
Jul 06 14:48:40.724677 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:40.724609    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f170ceef-fe2c-4890-baf9-a4589f08e27b-istio-data\") pod \"f170ceef-fe2c-4890-baf9-a4589f08e27b\" (UID: \"f170ceef-fe2c-4890-baf9-a4589f08e27b\") "
Jul 06 14:48:40.724732 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:40.724695    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f170ceef-fe2c-4890-baf9-a4589f08e27b-credential-socket\") pod \"f170ceef-fe2c-4890-baf9-a4589f08e27b\" (UID: \"f170ceef-fe2c-4890-baf9-a4589f08e27b\") "
Jul 06 14:48:40.724791 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:40.724750    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v5k8d\" (UniqueName: \"kubernetes.io/projected/f170ceef-fe2c-4890-baf9-a4589f08e27b-kube-api-access-v5k8d\") pod \"f170ceef-fe2c-4890-baf9-a4589f08e27b\" (UID: \"f170ceef-fe2c-4890-baf9-a4589f08e27b\") "
Jul 06 14:48:40.724791 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:40.724783    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f170ceef-fe2c-4890-baf9-a4589f08e27b-workload-certs\") pod \"f170ceef-fe2c-4890-baf9-a4589f08e27b\" (UID: \"f170ceef-fe2c-4890-baf9-a4589f08e27b\") "
Jul 06 14:48:40.724894 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:40.724789    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f170ceef-fe2c-4890-baf9-a4589f08e27b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f170ceef-fe2c-4890-baf9-a4589f08e27b" (UID: "f170ceef-fe2c-4890-baf9-a4589f08e27b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 14:48:40.724894 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:40.724812    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f170ceef-fe2c-4890-baf9-a4589f08e27b-istio-token\") pod \"f170ceef-fe2c-4890-baf9-a4589f08e27b\" (UID: \"f170ceef-fe2c-4890-baf9-a4589f08e27b\") "
Jul 06 14:48:40.724894 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:40.724870    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f170ceef-fe2c-4890-baf9-a4589f08e27b-istio-podinfo\") pod \"f170ceef-fe2c-4890-baf9-a4589f08e27b\" (UID: \"f170ceef-fe2c-4890-baf9-a4589f08e27b\") "
Jul 06 14:48:40.726207 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:40.725076    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f170ceef-fe2c-4890-baf9-a4589f08e27b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f170ceef-fe2c-4890-baf9-a4589f08e27b" (UID: "f170ceef-fe2c-4890-baf9-a4589f08e27b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 14:48:40.726207 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:40.725366    2559 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f170ceef-fe2c-4890-baf9-a4589f08e27b-workload-socket\") on node \"ip-10-0-133-112.ec2.internal\" DevicePath \"\""
Jul 06 14:48:40.726207 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:40.725387    2559 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f170ceef-fe2c-4890-baf9-a4589f08e27b-istiod-ca-cert\") on node \"ip-10-0-133-112.ec2.internal\" DevicePath \"\""
Jul 06 14:48:40.726207 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:40.725438    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f170ceef-fe2c-4890-baf9-a4589f08e27b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f170ceef-fe2c-4890-baf9-a4589f08e27b" (UID: "f170ceef-fe2c-4890-baf9-a4589f08e27b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 14:48:40.726207 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:40.726106    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f170ceef-fe2c-4890-baf9-a4589f08e27b-istio-data" (OuterVolumeSpecName: "istio-data") pod "f170ceef-fe2c-4890-baf9-a4589f08e27b" (UID: "f170ceef-fe2c-4890-baf9-a4589f08e27b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 14:48:40.726207 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:40.726179    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f170ceef-fe2c-4890-baf9-a4589f08e27b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f170ceef-fe2c-4890-baf9-a4589f08e27b" (UID: "f170ceef-fe2c-4890-baf9-a4589f08e27b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 14:48:40.727232 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:40.727206    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f170ceef-fe2c-4890-baf9-a4589f08e27b-istio-token" (OuterVolumeSpecName: "istio-token") pod "f170ceef-fe2c-4890-baf9-a4589f08e27b" (UID: "f170ceef-fe2c-4890-baf9-a4589f08e27b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 14:48:40.727336 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:40.727260    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f170ceef-fe2c-4890-baf9-a4589f08e27b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f170ceef-fe2c-4890-baf9-a4589f08e27b" (UID: "f170ceef-fe2c-4890-baf9-a4589f08e27b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 14:48:40.727336 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:40.727272    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f170ceef-fe2c-4890-baf9-a4589f08e27b-kube-api-access-v5k8d" (OuterVolumeSpecName: "kube-api-access-v5k8d") pod "f170ceef-fe2c-4890-baf9-a4589f08e27b" (UID: "f170ceef-fe2c-4890-baf9-a4589f08e27b"). InnerVolumeSpecName "kube-api-access-v5k8d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 14:48:40.727336 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:40.727286    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f170ceef-fe2c-4890-baf9-a4589f08e27b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f170ceef-fe2c-4890-baf9-a4589f08e27b" (UID: "f170ceef-fe2c-4890-baf9-a4589f08e27b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 06 14:48:40.826058 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:40.826020    2559 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f170ceef-fe2c-4890-baf9-a4589f08e27b-istio-data\") on node \"ip-10-0-133-112.ec2.internal\" DevicePath \"\""
Jul 06 14:48:40.826058 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:40.826052    2559 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f170ceef-fe2c-4890-baf9-a4589f08e27b-credential-socket\") on node \"ip-10-0-133-112.ec2.internal\" DevicePath \"\""
Jul 06 14:48:40.826058 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:40.826063    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v5k8d\" (UniqueName: \"kubernetes.io/projected/f170ceef-fe2c-4890-baf9-a4589f08e27b-kube-api-access-v5k8d\") on node \"ip-10-0-133-112.ec2.internal\" DevicePath \"\""
Jul 06 14:48:40.826309 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:40.826074    2559 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f170ceef-fe2c-4890-baf9-a4589f08e27b-workload-certs\") on node \"ip-10-0-133-112.ec2.internal\" DevicePath \"\""
Jul 06 14:48:40.826309 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:40.826082    2559 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f170ceef-fe2c-4890-baf9-a4589f08e27b-istio-token\") on node \"ip-10-0-133-112.ec2.internal\" DevicePath \"\""
Jul 06 14:48:40.826309 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:40.826093    2559 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f170ceef-fe2c-4890-baf9-a4589f08e27b-istio-podinfo\") on node \"ip-10-0-133-112.ec2.internal\" DevicePath \"\""
Jul 06 14:48:40.826309 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:40.826105    2559 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f170ceef-fe2c-4890-baf9-a4589f08e27b-istio-envoy\") on node \"ip-10-0-133-112.ec2.internal\" DevicePath \"\""
Jul 06 14:48:41.410838 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:41.410797    2559 generic.go:356] "Generic (PLEG): container finished" podID="f170ceef-fe2c-4890-baf9-a4589f08e27b" containerID="ed50ed4dee52ef802815cd39793b1031c35429f2a5eec51e9446bcc8f20d84fe" exitCode=0
Jul 06 14:48:41.411125 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:41.410866    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-4dff23b7-openshift-default-c8d66bb55-s5cl4"
Jul 06 14:48:41.411125 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:41.410869    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4dff23b7-openshift-default-c8d66bb55-s5cl4" event={"ID":"f170ceef-fe2c-4890-baf9-a4589f08e27b","Type":"ContainerDied","Data":"ed50ed4dee52ef802815cd39793b1031c35429f2a5eec51e9446bcc8f20d84fe"}
Jul 06 14:48:41.411125 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:41.411024    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4dff23b7-openshift-default-c8d66bb55-s5cl4" event={"ID":"f170ceef-fe2c-4890-baf9-a4589f08e27b","Type":"ContainerDied","Data":"72676ea783e2e7d0a93183c31c18628a2b07c99d05648613961353918833ac43"}
Jul 06 14:48:41.411125 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:41.411048    2559 scope.go:117] "RemoveContainer" containerID="ed50ed4dee52ef802815cd39793b1031c35429f2a5eec51e9446bcc8f20d84fe"
Jul 06 14:48:41.420617 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:41.420599    2559 scope.go:117] "RemoveContainer" containerID="13b054ab1d71b84029a30d9f5e8d07185a37dd5067d91fddd9a54af9f965a17c"
Jul 06 14:48:41.429503 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:41.429477    2559 scope.go:117] "RemoveContainer" containerID="ed50ed4dee52ef802815cd39793b1031c35429f2a5eec51e9446bcc8f20d84fe"
Jul 06 14:48:41.429814 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:48:41.429791    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ed50ed4dee52ef802815cd39793b1031c35429f2a5eec51e9446bcc8f20d84fe\": container with ID starting with ed50ed4dee52ef802815cd39793b1031c35429f2a5eec51e9446bcc8f20d84fe not found: ID does not exist" containerID="ed50ed4dee52ef802815cd39793b1031c35429f2a5eec51e9446bcc8f20d84fe"
Jul 06 14:48:41.429882 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:41.429828    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ed50ed4dee52ef802815cd39793b1031c35429f2a5eec51e9446bcc8f20d84fe"} err="failed to get container status \"ed50ed4dee52ef802815cd39793b1031c35429f2a5eec51e9446bcc8f20d84fe\": rpc error: code = NotFound desc = could not find container \"ed50ed4dee52ef802815cd39793b1031c35429f2a5eec51e9446bcc8f20d84fe\": container with ID starting with ed50ed4dee52ef802815cd39793b1031c35429f2a5eec51e9446bcc8f20d84fe not found: ID does not exist"
Jul 06 14:48:41.429882 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:41.429855    2559 scope.go:117] "RemoveContainer" containerID="13b054ab1d71b84029a30d9f5e8d07185a37dd5067d91fddd9a54af9f965a17c"
Jul 06 14:48:41.430125 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:48:41.430105    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"13b054ab1d71b84029a30d9f5e8d07185a37dd5067d91fddd9a54af9f965a17c\": container with ID starting with 13b054ab1d71b84029a30d9f5e8d07185a37dd5067d91fddd9a54af9f965a17c not found: ID does not exist" containerID="13b054ab1d71b84029a30d9f5e8d07185a37dd5067d91fddd9a54af9f965a17c"
Jul 06 14:48:41.430186 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:41.430137    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"13b054ab1d71b84029a30d9f5e8d07185a37dd5067d91fddd9a54af9f965a17c"} err="failed to get container status \"13b054ab1d71b84029a30d9f5e8d07185a37dd5067d91fddd9a54af9f965a17c\": rpc error: code = NotFound desc = could not find container \"13b054ab1d71b84029a30d9f5e8d07185a37dd5067d91fddd9a54af9f965a17c\": container with ID starting with 13b054ab1d71b84029a30d9f5e8d07185a37dd5067d91fddd9a54af9f965a17c not found: ID does not exist"
Jul 06 14:48:41.434858 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:41.434830    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-4dff23b7-openshift-default-c8d66bb55-s5cl4"]
Jul 06 14:48:41.439648 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:41.439620    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-4dff23b7-openshift-default-c8d66bb55-s5cl4"]
Jul 06 14:48:42.227670 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:42.227635    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f170ceef-fe2c-4890-baf9-a4589f08e27b" path="/var/lib/kubelet/pods/f170ceef-fe2c-4890-baf9-a4589f08e27b/volumes"
Jul 06 14:48:54.991041 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:54.991000    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-ad48ac23-openshift-default-d896fdd95-gcrft"]
Jul 06 14:48:54.991499 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:54.991483    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f170ceef-fe2c-4890-baf9-a4589f08e27b" containerName="istio-proxy"
Jul 06 14:48:54.991549 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:54.991501    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="f170ceef-fe2c-4890-baf9-a4589f08e27b" containerName="istio-proxy"
Jul 06 14:48:54.991606 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:54.991595    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="f170ceef-fe2c-4890-baf9-a4589f08e27b" containerName="istio-proxy"
Jul 06 14:48:54.991652 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:54.991608    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="f170ceef-fe2c-4890-baf9-a4589f08e27b" containerName="istio-proxy"
Jul 06 14:48:54.996148 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:54.996132    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-ad48ac23-openshift-default-d896fdd95-gcrft"
Jul 06 14:48:54.999536 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:54.999509    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-ad48ac23-openshift-default-dockercfg-clz8k\""
Jul 06 14:48:55.009962 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:55.009934    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-ad48ac23-openshift-default-d896fdd95-gcrft"]
Jul 06 14:48:55.061684 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:55.061649    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7bpqx\" (UniqueName: \"kubernetes.io/projected/eb564511-8dea-4e9e-a79d-6e78198ed7bb-kube-api-access-7bpqx\") pod \"e2e-mt-ad48ac23-openshift-default-d896fdd95-gcrft\" (UID: \"eb564511-8dea-4e9e-a79d-6e78198ed7bb\") " pod="openshift-ingress/e2e-mt-ad48ac23-openshift-default-d896fdd95-gcrft"
Jul 06 14:48:55.061684 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:55.061693    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eb564511-8dea-4e9e-a79d-6e78198ed7bb-workload-certs\") pod \"e2e-mt-ad48ac23-openshift-default-d896fdd95-gcrft\" (UID: \"eb564511-8dea-4e9e-a79d-6e78198ed7bb\") " pod="openshift-ingress/e2e-mt-ad48ac23-openshift-default-d896fdd95-gcrft"
Jul 06 14:48:55.061952 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:55.061715    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eb564511-8dea-4e9e-a79d-6e78198ed7bb-istio-data\") pod \"e2e-mt-ad48ac23-openshift-default-d896fdd95-gcrft\" (UID: \"eb564511-8dea-4e9e-a79d-6e78198ed7bb\") " pod="openshift-ingress/e2e-mt-ad48ac23-openshift-default-d896fdd95-gcrft"
Jul 06 14:48:55.061952 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:55.061734    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb564511-8dea-4e9e-a79d-6e78198ed7bb-workload-socket\") pod \"e2e-mt-ad48ac23-openshift-default-d896fdd95-gcrft\" (UID: \"eb564511-8dea-4e9e-a79d-6e78198ed7bb\") " pod="openshift-ingress/e2e-mt-ad48ac23-openshift-default-d896fdd95-gcrft"
Jul 06 14:48:55.061952 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:55.061805    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb564511-8dea-4e9e-a79d-6e78198ed7bb-credential-socket\") pod \"e2e-mt-ad48ac23-openshift-default-d896fdd95-gcrft\" (UID: \"eb564511-8dea-4e9e-a79d-6e78198ed7bb\") " pod="openshift-ingress/e2e-mt-ad48ac23-openshift-default-d896fdd95-gcrft"
Jul 06 14:48:55.061952 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:55.061840    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eb564511-8dea-4e9e-a79d-6e78198ed7bb-istiod-ca-cert\") pod \"e2e-mt-ad48ac23-openshift-default-d896fdd95-gcrft\" (UID: \"eb564511-8dea-4e9e-a79d-6e78198ed7bb\") " pod="openshift-ingress/e2e-mt-ad48ac23-openshift-default-d896fdd95-gcrft"
Jul 06 14:48:55.061952 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:55.061887    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eb564511-8dea-4e9e-a79d-6e78198ed7bb-istio-envoy\") pod \"e2e-mt-ad48ac23-openshift-default-d896fdd95-gcrft\" (UID: \"eb564511-8dea-4e9e-a79d-6e78198ed7bb\") " pod="openshift-ingress/e2e-mt-ad48ac23-openshift-default-d896fdd95-gcrft"
Jul 06 14:48:55.062177 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:55.061963    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eb564511-8dea-4e9e-a79d-6e78198ed7bb-istio-podinfo\") pod \"e2e-mt-ad48ac23-openshift-default-d896fdd95-gcrft\" (UID: \"eb564511-8dea-4e9e-a79d-6e78198ed7bb\") " pod="openshift-ingress/e2e-mt-ad48ac23-openshift-default-d896fdd95-gcrft"
Jul 06 14:48:55.062177 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:55.062026    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eb564511-8dea-4e9e-a79d-6e78198ed7bb-istio-token\") pod \"e2e-mt-ad48ac23-openshift-default-d896fdd95-gcrft\" (UID: \"eb564511-8dea-4e9e-a79d-6e78198ed7bb\") " pod="openshift-ingress/e2e-mt-ad48ac23-openshift-default-d896fdd95-gcrft"
Jul 06 14:48:55.163461 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:55.163413    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eb564511-8dea-4e9e-a79d-6e78198ed7bb-workload-certs\") pod \"e2e-mt-ad48ac23-openshift-default-d896fdd95-gcrft\" (UID: \"eb564511-8dea-4e9e-a79d-6e78198ed7bb\") " pod="openshift-ingress/e2e-mt-ad48ac23-openshift-default-d896fdd95-gcrft"
Jul 06 14:48:55.163461 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:55.163466    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eb564511-8dea-4e9e-a79d-6e78198ed7bb-istio-data\") pod \"e2e-mt-ad48ac23-openshift-default-d896fdd95-gcrft\" (UID: \"eb564511-8dea-4e9e-a79d-6e78198ed7bb\") " pod="openshift-ingress/e2e-mt-ad48ac23-openshift-default-d896fdd95-gcrft"
Jul 06 14:48:55.163800 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:55.163529    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb564511-8dea-4e9e-a79d-6e78198ed7bb-workload-socket\") pod \"e2e-mt-ad48ac23-openshift-default-d896fdd95-gcrft\" (UID: \"eb564511-8dea-4e9e-a79d-6e78198ed7bb\") " pod="openshift-ingress/e2e-mt-ad48ac23-openshift-default-d896fdd95-gcrft"
Jul 06 14:48:55.163800 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:55.163629    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb564511-8dea-4e9e-a79d-6e78198ed7bb-credential-socket\") pod \"e2e-mt-ad48ac23-openshift-default-d896fdd95-gcrft\" (UID: \"eb564511-8dea-4e9e-a79d-6e78198ed7bb\") " pod="openshift-ingress/e2e-mt-ad48ac23-openshift-default-d896fdd95-gcrft"
Jul 06 14:48:55.163800 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:55.163660    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eb564511-8dea-4e9e-a79d-6e78198ed7bb-istiod-ca-cert\") pod \"e2e-mt-ad48ac23-openshift-default-d896fdd95-gcrft\" (UID: \"eb564511-8dea-4e9e-a79d-6e78198ed7bb\") " pod="openshift-ingress/e2e-mt-ad48ac23-openshift-default-d896fdd95-gcrft"
Jul 06 14:48:55.163800 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:55.163723    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eb564511-8dea-4e9e-a79d-6e78198ed7bb-istio-envoy\") pod \"e2e-mt-ad48ac23-openshift-default-d896fdd95-gcrft\" (UID: \"eb564511-8dea-4e9e-a79d-6e78198ed7bb\") " pod="openshift-ingress/e2e-mt-ad48ac23-openshift-default-d896fdd95-gcrft"
Jul 06 14:48:55.163800 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:55.163789    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eb564511-8dea-4e9e-a79d-6e78198ed7bb-istio-podinfo\") pod \"e2e-mt-ad48ac23-openshift-default-d896fdd95-gcrft\" (UID: \"eb564511-8dea-4e9e-a79d-6e78198ed7bb\") " pod="openshift-ingress/e2e-mt-ad48ac23-openshift-default-d896fdd95-gcrft"
Jul 06 14:48:55.164062 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:55.163851    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eb564511-8dea-4e9e-a79d-6e78198ed7bb-istio-token\") pod \"e2e-mt-ad48ac23-openshift-default-d896fdd95-gcrft\" (UID: \"eb564511-8dea-4e9e-a79d-6e78198ed7bb\") " pod="openshift-ingress/e2e-mt-ad48ac23-openshift-default-d896fdd95-gcrft"
Jul 06 14:48:55.164062 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:55.163876    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb564511-8dea-4e9e-a79d-6e78198ed7bb-workload-socket\") pod \"e2e-mt-ad48ac23-openshift-default-d896fdd95-gcrft\" (UID: \"eb564511-8dea-4e9e-a79d-6e78198ed7bb\") " pod="openshift-ingress/e2e-mt-ad48ac23-openshift-default-d896fdd95-gcrft"
Jul 06 14:48:55.164062 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:55.163927    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7bpqx\" (UniqueName: \"kubernetes.io/projected/eb564511-8dea-4e9e-a79d-6e78198ed7bb-kube-api-access-7bpqx\") pod \"e2e-mt-ad48ac23-openshift-default-d896fdd95-gcrft\" (UID: \"eb564511-8dea-4e9e-a79d-6e78198ed7bb\") " pod="openshift-ingress/e2e-mt-ad48ac23-openshift-default-d896fdd95-gcrft"
Jul 06 14:48:55.164062 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:55.163935    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eb564511-8dea-4e9e-a79d-6e78198ed7bb-workload-certs\") pod \"e2e-mt-ad48ac23-openshift-default-d896fdd95-gcrft\" (UID: \"eb564511-8dea-4e9e-a79d-6e78198ed7bb\") " pod="openshift-ingress/e2e-mt-ad48ac23-openshift-default-d896fdd95-gcrft"
Jul 06 14:48:55.164263 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:55.164067    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb564511-8dea-4e9e-a79d-6e78198ed7bb-credential-socket\") pod \"e2e-mt-ad48ac23-openshift-default-d896fdd95-gcrft\" (UID: \"eb564511-8dea-4e9e-a79d-6e78198ed7bb\") " pod="openshift-ingress/e2e-mt-ad48ac23-openshift-default-d896fdd95-gcrft"
Jul 06 14:48:55.164263 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:55.164164    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eb564511-8dea-4e9e-a79d-6e78198ed7bb-istio-data\") pod \"e2e-mt-ad48ac23-openshift-default-d896fdd95-gcrft\" (UID: \"eb564511-8dea-4e9e-a79d-6e78198ed7bb\") " pod="openshift-ingress/e2e-mt-ad48ac23-openshift-default-d896fdd95-gcrft"
Jul 06 14:48:55.164412 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:55.164392    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eb564511-8dea-4e9e-a79d-6e78198ed7bb-istiod-ca-cert\") pod \"e2e-mt-ad48ac23-openshift-default-d896fdd95-gcrft\" (UID: \"eb564511-8dea-4e9e-a79d-6e78198ed7bb\") " pod="openshift-ingress/e2e-mt-ad48ac23-openshift-default-d896fdd95-gcrft"
Jul 06 14:48:55.166159 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:55.166133    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eb564511-8dea-4e9e-a79d-6e78198ed7bb-istio-envoy\") pod \"e2e-mt-ad48ac23-openshift-default-d896fdd95-gcrft\" (UID: \"eb564511-8dea-4e9e-a79d-6e78198ed7bb\") " pod="openshift-ingress/e2e-mt-ad48ac23-openshift-default-d896fdd95-gcrft"
Jul 06 14:48:55.166453 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:55.166435    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eb564511-8dea-4e9e-a79d-6e78198ed7bb-istio-podinfo\") pod \"e2e-mt-ad48ac23-openshift-default-d896fdd95-gcrft\" (UID: \"eb564511-8dea-4e9e-a79d-6e78198ed7bb\") " pod="openshift-ingress/e2e-mt-ad48ac23-openshift-default-d896fdd95-gcrft"
Jul 06 14:48:55.172825 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:55.172797    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eb564511-8dea-4e9e-a79d-6e78198ed7bb-istio-token\") pod \"e2e-mt-ad48ac23-openshift-default-d896fdd95-gcrft\" (UID: \"eb564511-8dea-4e9e-a79d-6e78198ed7bb\") " pod="openshift-ingress/e2e-mt-ad48ac23-openshift-default-d896fdd95-gcrft"
Jul 06 14:48:55.172981 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:55.172962    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7bpqx\" (UniqueName: \"kubernetes.io/projected/eb564511-8dea-4e9e-a79d-6e78198ed7bb-kube-api-access-7bpqx\") pod \"e2e-mt-ad48ac23-openshift-default-d896fdd95-gcrft\" (UID: \"eb564511-8dea-4e9e-a79d-6e78198ed7bb\") " pod="openshift-ingress/e2e-mt-ad48ac23-openshift-default-d896fdd95-gcrft"
Jul 06 14:48:55.307833 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:55.307736    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-ad48ac23-openshift-default-d896fdd95-gcrft"
Jul 06 14:48:55.459009 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:55.458964    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-ad48ac23-openshift-default-d896fdd95-gcrft"]
Jul 06 14:48:55.462320 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:48:55.462290    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb564511_8dea_4e9e_a79d_6e78198ed7bb.slice/crio-41a0236404a74e9af201497cbf21b6d2c03f18b48c169eb38a112f3e5ebd3878 WatchSource:0}: Error finding container 41a0236404a74e9af201497cbf21b6d2c03f18b48c169eb38a112f3e5ebd3878: Status 404 returned error can't find the container with id 41a0236404a74e9af201497cbf21b6d2c03f18b48c169eb38a112f3e5ebd3878
Jul 06 14:48:55.464526 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:55.464488    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 06 14:48:55.464666 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:55.464617    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 06 14:48:55.464734 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:55.464672    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 06 14:48:55.483641 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:55.483607    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ad48ac23-openshift-default-d896fdd95-gcrft" event={"ID":"eb564511-8dea-4e9e-a79d-6e78198ed7bb","Type":"ContainerStarted","Data":"41a0236404a74e9af201497cbf21b6d2c03f18b48c169eb38a112f3e5ebd3878"}
Jul 06 14:48:56.490601 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:56.490539    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ad48ac23-openshift-default-d896fdd95-gcrft" event={"ID":"eb564511-8dea-4e9e-a79d-6e78198ed7bb","Type":"ContainerStarted","Data":"2e3821dbb44ba4275311c1d17b534e99edcb78a250b18faddfe6e13af6719ea8"}
Jul 06 14:48:56.513018 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:56.512955    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-ad48ac23-openshift-default-d896fdd95-gcrft" podStartSLOduration=2.5129370890000002 podStartE2EDuration="2.512937089s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:48:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:48:56.511154734 +0000 UTC m=+1758.818446854" watchObservedRunningTime="2026-07-06 14:48:56.512937089 +0000 UTC m=+1758.820229228"
Jul 06 14:48:57.308490 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:57.308452    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-ad48ac23-openshift-default-d896fdd95-gcrft"
Jul 06 14:48:57.313586 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:57.313539    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-ad48ac23-openshift-default-d896fdd95-gcrft"
Jul 06 14:48:57.495233 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:57.495199    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-ad48ac23-openshift-default-d896fdd95-gcrft"
Jul 06 14:48:57.496221 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:48:57.496203    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-ad48ac23-openshift-default-d896fdd95-gcrft"
Jul 06 14:49:01.877413 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:49:01.877365    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-mt-ad48ac23-6d49797b46-sxrtn"]
Jul 06 14:49:01.878001 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:49:01.877975    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f170ceef-fe2c-4890-baf9-a4589f08e27b" containerName="istio-proxy"
Jul 06 14:49:01.878001 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:49:01.878000    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="f170ceef-fe2c-4890-baf9-a4589f08e27b" containerName="istio-proxy"
Jul 06 14:49:01.881512 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:49:01.881490    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-ad48ac23-6d49797b46-sxrtn"
Jul 06 14:49:01.883843 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:49:01.883815    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-mt-ad48ac23-serving-cert\""
Jul 06 14:49:01.894666 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:49:01.894638    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-ad48ac23-6d49797b46-sxrtn"]
Jul 06 14:49:02.029043 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:49:02.028983    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0727b0b2-db59-46b1-a2b6-b50f950299df-maas-api-tls\") pod \"maas-api-e2e-mt-ad48ac23-6d49797b46-sxrtn\" (UID: \"0727b0b2-db59-46b1-a2b6-b50f950299df\") " pod="opendatahub/maas-api-e2e-mt-ad48ac23-6d49797b46-sxrtn"
Jul 06 14:49:02.029250 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:49:02.029108    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0727b0b2-db59-46b1-a2b6-b50f950299df-openshift-service-ca\") pod \"maas-api-e2e-mt-ad48ac23-6d49797b46-sxrtn\" (UID: \"0727b0b2-db59-46b1-a2b6-b50f950299df\") " pod="opendatahub/maas-api-e2e-mt-ad48ac23-6d49797b46-sxrtn"
Jul 06 14:49:02.029250 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:49:02.029174    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ltxp2\" (UniqueName: \"kubernetes.io/projected/0727b0b2-db59-46b1-a2b6-b50f950299df-kube-api-access-ltxp2\") pod \"maas-api-e2e-mt-ad48ac23-6d49797b46-sxrtn\" (UID: \"0727b0b2-db59-46b1-a2b6-b50f950299df\") " pod="opendatahub/maas-api-e2e-mt-ad48ac23-6d49797b46-sxrtn"
Jul 06 14:49:02.130652 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:49:02.130537    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0727b0b2-db59-46b1-a2b6-b50f950299df-maas-api-tls\") pod \"maas-api-e2e-mt-ad48ac23-6d49797b46-sxrtn\" (UID: \"0727b0b2-db59-46b1-a2b6-b50f950299df\") " pod="opendatahub/maas-api-e2e-mt-ad48ac23-6d49797b46-sxrtn"
Jul 06 14:49:02.130652 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:49:02.130619    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0727b0b2-db59-46b1-a2b6-b50f950299df-openshift-service-ca\") pod \"maas-api-e2e-mt-ad48ac23-6d49797b46-sxrtn\" (UID: \"0727b0b2-db59-46b1-a2b6-b50f950299df\") " pod="opendatahub/maas-api-e2e-mt-ad48ac23-6d49797b46-sxrtn"
Jul 06 14:49:02.130902 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:49:02.130661    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ltxp2\" (UniqueName: \"kubernetes.io/projected/0727b0b2-db59-46b1-a2b6-b50f950299df-kube-api-access-ltxp2\") pod \"maas-api-e2e-mt-ad48ac23-6d49797b46-sxrtn\" (UID: \"0727b0b2-db59-46b1-a2b6-b50f950299df\") " pod="opendatahub/maas-api-e2e-mt-ad48ac23-6d49797b46-sxrtn"
Jul 06 14:49:02.131265 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:49:02.131241    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0727b0b2-db59-46b1-a2b6-b50f950299df-openshift-service-ca\") pod \"maas-api-e2e-mt-ad48ac23-6d49797b46-sxrtn\" (UID: \"0727b0b2-db59-46b1-a2b6-b50f950299df\") " pod="opendatahub/maas-api-e2e-mt-ad48ac23-6d49797b46-sxrtn"
Jul 06 14:49:02.133143 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:49:02.133117    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0727b0b2-db59-46b1-a2b6-b50f950299df-maas-api-tls\") pod \"maas-api-e2e-mt-ad48ac23-6d49797b46-sxrtn\" (UID: \"0727b0b2-db59-46b1-a2b6-b50f950299df\") " pod="opendatahub/maas-api-e2e-mt-ad48ac23-6d49797b46-sxrtn"
Jul 06 14:49:02.144938 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:49:02.144910    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ltxp2\" (UniqueName: \"kubernetes.io/projected/0727b0b2-db59-46b1-a2b6-b50f950299df-kube-api-access-ltxp2\") pod \"maas-api-e2e-mt-ad48ac23-6d49797b46-sxrtn\" (UID: \"0727b0b2-db59-46b1-a2b6-b50f950299df\") " pod="opendatahub/maas-api-e2e-mt-ad48ac23-6d49797b46-sxrtn"
Jul 06 14:49:02.193778 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:49:02.193736    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-ad48ac23-6d49797b46-sxrtn"
Jul 06 14:49:02.356355 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:49:02.356303    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0727b0b2_db59_46b1_a2b6_b50f950299df.slice/crio-a51aa8f468fba031b1b98fa09126e66b5f7fea8a26bc85dde343193b032abe63 WatchSource:0}: Error finding container a51aa8f468fba031b1b98fa09126e66b5f7fea8a26bc85dde343193b032abe63: Status 404 returned error can't find the container with id a51aa8f468fba031b1b98fa09126e66b5f7fea8a26bc85dde343193b032abe63
Jul 06 14:49:02.358510 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:49:02.358413    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-ad48ac23-6d49797b46-sxrtn"]
Jul 06 14:49:02.521046 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:49:02.521008    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-ad48ac23-6d49797b46-sxrtn" event={"ID":"0727b0b2-db59-46b1-a2b6-b50f950299df","Type":"ContainerStarted","Data":"a51aa8f468fba031b1b98fa09126e66b5f7fea8a26bc85dde343193b032abe63"}
Jul 06 14:49:03.527703 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:49:03.527661    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-ad48ac23-6d49797b46-sxrtn" event={"ID":"0727b0b2-db59-46b1-a2b6-b50f950299df","Type":"ContainerStarted","Data":"4f2307af0f376237cb60f30e1afb629dcf14fb0759f994057568b810808fed0a"}
Jul 06 14:49:03.528148 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:49:03.527740    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-mt-ad48ac23-6d49797b46-sxrtn"
Jul 06 14:49:03.547900 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:49:03.547825    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-mt-ad48ac23-6d49797b46-sxrtn" podStartSLOduration=2.065819499 podStartE2EDuration="2.547802994s" totalImagesPullingTime="481.983495ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:49:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:49:03.547191359 +0000 UTC m=+1765.854483442" watchObservedRunningTime="2026-07-06 14:49:03.547802994 +0000 UTC m=+1765.855095092"
Jul 06 14:49:09.538974 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:49:09.538941    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-mt-ad48ac23-6d49797b46-sxrtn"
Jul 06 14:49:12.585911 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:49:12.584766    2559 patch_prober.go:28] interesting pod/e2e-mt-ad48ac23-openshift-default-d896fdd95-gcrft container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": read tcp 10.132.0.2:33384->10.132.0.51:15021: read: connection reset by peer" start-of-body=
Jul 06 14:49:12.585911 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:49:12.584850    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-ad48ac23-openshift-default-d896fdd95-gcrft" podUID="eb564511-8dea-4e9e-a79d-6e78198ed7bb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": read tcp 10.132.0.2:33384->10.132.0.51:15021: read: connection reset by peer"
Jul 06 14:49:13.579270 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:49:13.579227    2559 generic.go:356] "Generic (PLEG): container finished" podID="eb564511-8dea-4e9e-a79d-6e78198ed7bb" containerID="2e3821dbb44ba4275311c1d17b534e99edcb78a250b18faddfe6e13af6719ea8" exitCode=0
Jul 06 14:49:13.579473 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:49:13.579299    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ad48ac23-openshift-default-d896fdd95-gcrft" event={"ID":"eb564511-8dea-4e9e-a79d-6e78198ed7bb","Type":"ContainerDied","Data":"2e3821dbb44ba4275311c1d17b534e99edcb78a250b18faddfe6e13af6719ea8"}
Jul 06 14:49:13.579743 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:49:13.579725    2559 scope.go:117] "RemoveContainer" containerID="2e3821dbb44ba4275311c1d17b534e99edcb78a250b18faddfe6e13af6719ea8"
Jul 06 14:49:13.580726 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:49:13.580700    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 06 14:49:13.580794 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:49:13.580770    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 06 14:49:13.580833 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:49:13.580801    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 06 14:49:14.589788 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:49:14.589734    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ad48ac23-openshift-default-d896fdd95-gcrft" event={"ID":"eb564511-8dea-4e9e-a79d-6e78198ed7bb","Type":"ContainerStarted","Data":"e517fc76545d1f80265dfcb77efe90f33d4523e653ea2b27f7544c681af0fea7"}
Jul 06 14:49:15.308651 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:49:15.308609    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-ad48ac23-openshift-default-d896fdd95-gcrft"
Jul 06 14:49:15.308979 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:49:15.308949    2559 patch_prober.go:28] interesting pod/e2e-mt-ad48ac23-openshift-default-d896fdd95-gcrft container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body=
Jul 06 14:49:15.309051 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:49:15.309027    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-ad48ac23-openshift-default-d896fdd95-gcrft" podUID="eb564511-8dea-4e9e-a79d-6e78198ed7bb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused"
Jul 06 14:49:16.308987 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:49:16.308950    2559 patch_prober.go:28] interesting pod/e2e-mt-ad48ac23-openshift-default-d896fdd95-gcrft container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body=
Jul 06 14:49:16.309404 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:49:16.309013    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-ad48ac23-openshift-default-d896fdd95-gcrft" podUID="eb564511-8dea-4e9e-a79d-6e78198ed7bb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused"
Jul 06 14:49:17.308787 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:49:17.308744    2559 patch_prober.go:28] interesting pod/e2e-mt-ad48ac23-openshift-default-d896fdd95-gcrft container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body=
Jul 06 14:49:17.308980 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:49:17.308808    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-ad48ac23-openshift-default-d896fdd95-gcrft" podUID="eb564511-8dea-4e9e-a79d-6e78198ed7bb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused"
Jul 06 14:49:18.309219 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:49:18.309179    2559 patch_prober.go:28] interesting pod/e2e-mt-ad48ac23-openshift-default-d896fdd95-gcrft container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body=
Jul 06 14:49:18.309659 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:49:18.309254    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-ad48ac23-openshift-default-d896fdd95-gcrft" podUID="eb564511-8dea-4e9e-a79d-6e78198ed7bb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused"
Jul 06 14:49:19.311774 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:49:19.311738    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-ad48ac23-openshift-default-d896fdd95-gcrft"
Jul 06 14:49:19.312171 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:49:19.312142    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-ad48ac23-openshift-default-d896fdd95-gcrft"
Jul 06 14:49:19.312903 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:49:19.312882    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-ad48ac23-openshift-default-d896fdd95-gcrft"
Jul 06 14:49:38.258288 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:49:38.258253    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-g5w5m_e35b0247-7c4e-4bd1-bc28-c084dc40ffa1/console-operator/1.log"
Jul 06 14:49:38.260716 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:49:38.260687    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-g5w5m_e35b0247-7c4e-4bd1-bc28-c084dc40ffa1/console-operator/1.log"
Jul 06 14:49:50.067061 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:49:50.067024    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-ad48ac23-6d49797b46-sxrtn"]
Jul 06 14:49:50.067523 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:49:50.067241    2559 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-mt-ad48ac23-6d49797b46-sxrtn" podUID="0727b0b2-db59-46b1-a2b6-b50f950299df" containerName="maas-api" containerID="cri-o://4f2307af0f376237cb60f30e1afb629dcf14fb0759f994057568b810808fed0a" gracePeriod=30
Jul 06 14:49:50.332546 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:49:50.332476    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-ad48ac23-6d49797b46-sxrtn"
Jul 06 14:49:50.413292 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:49:50.413255    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0727b0b2-db59-46b1-a2b6-b50f950299df-openshift-service-ca\") pod \"0727b0b2-db59-46b1-a2b6-b50f950299df\" (UID: \"0727b0b2-db59-46b1-a2b6-b50f950299df\") "
Jul 06 14:49:50.413474 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:49:50.413326    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ltxp2\" (UniqueName: \"kubernetes.io/projected/0727b0b2-db59-46b1-a2b6-b50f950299df-kube-api-access-ltxp2\") pod \"0727b0b2-db59-46b1-a2b6-b50f950299df\" (UID: \"0727b0b2-db59-46b1-a2b6-b50f950299df\") "
Jul 06 14:49:50.413474 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:49:50.413389    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0727b0b2-db59-46b1-a2b6-b50f950299df-maas-api-tls\") pod \"0727b0b2-db59-46b1-a2b6-b50f950299df\" (UID: \"0727b0b2-db59-46b1-a2b6-b50f950299df\") "
Jul 06 14:49:50.413707 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:49:50.413679    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0727b0b2-db59-46b1-a2b6-b50f950299df-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "0727b0b2-db59-46b1-a2b6-b50f950299df" (UID: "0727b0b2-db59-46b1-a2b6-b50f950299df"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 14:49:50.413788 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:49:50.413747    2559 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0727b0b2-db59-46b1-a2b6-b50f950299df-openshift-service-ca\") on node \"ip-10-0-133-112.ec2.internal\" DevicePath \"\""
Jul 06 14:49:50.415458 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:49:50.415434    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0727b0b2-db59-46b1-a2b6-b50f950299df-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "0727b0b2-db59-46b1-a2b6-b50f950299df" (UID: "0727b0b2-db59-46b1-a2b6-b50f950299df"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 14:49:50.415532 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:49:50.415517    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0727b0b2-db59-46b1-a2b6-b50f950299df-kube-api-access-ltxp2" (OuterVolumeSpecName: "kube-api-access-ltxp2") pod "0727b0b2-db59-46b1-a2b6-b50f950299df" (UID: "0727b0b2-db59-46b1-a2b6-b50f950299df"). InnerVolumeSpecName "kube-api-access-ltxp2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 14:49:50.514828 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:49:50.514789    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0727b0b2-db59-46b1-a2b6-b50f950299df-maas-api-tls\") on node \"ip-10-0-133-112.ec2.internal\" DevicePath \"\""
Jul 06 14:49:50.514828 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:49:50.514822    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ltxp2\" (UniqueName: \"kubernetes.io/projected/0727b0b2-db59-46b1-a2b6-b50f950299df-kube-api-access-ltxp2\") on node \"ip-10-0-133-112.ec2.internal\" DevicePath \"\""
Jul 06 14:49:50.779754 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:49:50.779718    2559 generic.go:356] "Generic (PLEG): container finished" podID="0727b0b2-db59-46b1-a2b6-b50f950299df" containerID="4f2307af0f376237cb60f30e1afb629dcf14fb0759f994057568b810808fed0a" exitCode=0
Jul 06 14:49:50.779979 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:49:50.779775    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-ad48ac23-6d49797b46-sxrtn"
Jul 06 14:49:50.779979 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:49:50.779788    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-ad48ac23-6d49797b46-sxrtn" event={"ID":"0727b0b2-db59-46b1-a2b6-b50f950299df","Type":"ContainerDied","Data":"4f2307af0f376237cb60f30e1afb629dcf14fb0759f994057568b810808fed0a"}
Jul 06 14:49:50.779979 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:49:50.779853    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-ad48ac23-6d49797b46-sxrtn" event={"ID":"0727b0b2-db59-46b1-a2b6-b50f950299df","Type":"ContainerDied","Data":"a51aa8f468fba031b1b98fa09126e66b5f7fea8a26bc85dde343193b032abe63"}
Jul 06 14:49:50.779979 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:49:50.779875    2559 scope.go:117] "RemoveContainer" containerID="4f2307af0f376237cb60f30e1afb629dcf14fb0759f994057568b810808fed0a"
Jul 06 14:49:50.789617 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:49:50.789556    2559 scope.go:117] "RemoveContainer" containerID="4f2307af0f376237cb60f30e1afb629dcf14fb0759f994057568b810808fed0a"
Jul 06 14:49:50.789908 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:49:50.789890    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4f2307af0f376237cb60f30e1afb629dcf14fb0759f994057568b810808fed0a\": container with ID starting with 4f2307af0f376237cb60f30e1afb629dcf14fb0759f994057568b810808fed0a not found: ID does not exist" containerID="4f2307af0f376237cb60f30e1afb629dcf14fb0759f994057568b810808fed0a"
Jul 06 14:49:50.789961 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:49:50.789917    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4f2307af0f376237cb60f30e1afb629dcf14fb0759f994057568b810808fed0a"} err="failed to get container status \"4f2307af0f376237cb60f30e1afb629dcf14fb0759f994057568b810808fed0a\": rpc error: code = NotFound desc = could not find container \"4f2307af0f376237cb60f30e1afb629dcf14fb0759f994057568b810808fed0a\": container with ID starting with 4f2307af0f376237cb60f30e1afb629dcf14fb0759f994057568b810808fed0a not found: ID does not exist"
Jul 06 14:49:50.823082 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:49:50.823042    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-ad48ac23-6d49797b46-sxrtn"]
Jul 06 14:49:50.830085 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:49:50.830052    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-mt-ad48ac23-6d49797b46-sxrtn"]
Jul 06 14:49:52.226765 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:49:52.226730    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0727b0b2-db59-46b1-a2b6-b50f950299df" path="/var/lib/kubelet/pods/0727b0b2-db59-46b1-a2b6-b50f950299df/volumes"
Jul 06 14:50:47.791532 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:50:47.791493    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-ad48ac23-openshift-default-d896fdd95-gcrft"]
Jul 06 14:50:47.792047 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:50:47.791722    2559 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-ad48ac23-openshift-default-d896fdd95-gcrft" podUID="eb564511-8dea-4e9e-a79d-6e78198ed7bb" containerName="istio-proxy" containerID="cri-o://e517fc76545d1f80265dfcb77efe90f33d4523e653ea2b27f7544c681af0fea7" gracePeriod=30
Jul 06 14:50:49.314546 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:50:49.314506    2559 patch_prober.go:28] interesting pod/e2e-mt-ad48ac23-openshift-default-d896fdd95-gcrft container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 06 14:50:49.314954 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:50:49.314601    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-ad48ac23-openshift-default-d896fdd95-gcrft" podUID="eb564511-8dea-4e9e-a79d-6e78198ed7bb" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 06 14:50:53.052809 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:50:53.052777    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-ad48ac23-openshift-default-d896fdd95-gcrft"
Jul 06 14:50:53.090353 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:50:53.090262    2559 generic.go:356] "Generic (PLEG): container finished" podID="eb564511-8dea-4e9e-a79d-6e78198ed7bb" containerID="e517fc76545d1f80265dfcb77efe90f33d4523e653ea2b27f7544c681af0fea7" exitCode=0
Jul 06 14:50:53.090353 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:50:53.090320    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-ad48ac23-openshift-default-d896fdd95-gcrft"
Jul 06 14:50:53.090606 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:50:53.090351    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ad48ac23-openshift-default-d896fdd95-gcrft" event={"ID":"eb564511-8dea-4e9e-a79d-6e78198ed7bb","Type":"ContainerDied","Data":"e517fc76545d1f80265dfcb77efe90f33d4523e653ea2b27f7544c681af0fea7"}
Jul 06 14:50:53.090606 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:50:53.090412    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ad48ac23-openshift-default-d896fdd95-gcrft" event={"ID":"eb564511-8dea-4e9e-a79d-6e78198ed7bb","Type":"ContainerDied","Data":"41a0236404a74e9af201497cbf21b6d2c03f18b48c169eb38a112f3e5ebd3878"}
Jul 06 14:50:53.090606 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:50:53.090442    2559 scope.go:117] "RemoveContainer" containerID="e517fc76545d1f80265dfcb77efe90f33d4523e653ea2b27f7544c681af0fea7"
Jul 06 14:50:53.099637 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:50:53.099609    2559 scope.go:117] "RemoveContainer" containerID="2e3821dbb44ba4275311c1d17b534e99edcb78a250b18faddfe6e13af6719ea8"
Jul 06 14:50:53.113725 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:50:53.113652    2559 scope.go:117] "RemoveContainer" containerID="e517fc76545d1f80265dfcb77efe90f33d4523e653ea2b27f7544c681af0fea7"
Jul 06 14:50:53.114065 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:50:53.114037    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e517fc76545d1f80265dfcb77efe90f33d4523e653ea2b27f7544c681af0fea7\": container with ID starting with e517fc76545d1f80265dfcb77efe90f33d4523e653ea2b27f7544c681af0fea7 not found: ID does not exist" containerID="e517fc76545d1f80265dfcb77efe90f33d4523e653ea2b27f7544c681af0fea7"
Jul 06 14:50:53.114181 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:50:53.114072    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e517fc76545d1f80265dfcb77efe90f33d4523e653ea2b27f7544c681af0fea7"} err="failed to get container status \"e517fc76545d1f80265dfcb77efe90f33d4523e653ea2b27f7544c681af0fea7\": rpc error: code = NotFound desc = could not find container \"e517fc76545d1f80265dfcb77efe90f33d4523e653ea2b27f7544c681af0fea7\": container with ID starting with e517fc76545d1f80265dfcb77efe90f33d4523e653ea2b27f7544c681af0fea7 not found: ID does not exist"
Jul 06 14:50:53.114181 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:50:53.114091    2559 scope.go:117] "RemoveContainer" containerID="2e3821dbb44ba4275311c1d17b534e99edcb78a250b18faddfe6e13af6719ea8"
Jul 06 14:50:53.114430 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:50:53.114404    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2e3821dbb44ba4275311c1d17b534e99edcb78a250b18faddfe6e13af6719ea8\": container with ID starting with 2e3821dbb44ba4275311c1d17b534e99edcb78a250b18faddfe6e13af6719ea8 not found: ID does not exist" containerID="2e3821dbb44ba4275311c1d17b534e99edcb78a250b18faddfe6e13af6719ea8"
Jul 06 14:50:53.114499 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:50:53.114439    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2e3821dbb44ba4275311c1d17b534e99edcb78a250b18faddfe6e13af6719ea8"} err="failed to get container status \"2e3821dbb44ba4275311c1d17b534e99edcb78a250b18faddfe6e13af6719ea8\": rpc error: code = NotFound desc = could not find container \"2e3821dbb44ba4275311c1d17b534e99edcb78a250b18faddfe6e13af6719ea8\": container with ID starting with 2e3821dbb44ba4275311c1d17b534e99edcb78a250b18faddfe6e13af6719ea8 not found: ID does not exist"
Jul 06 14:50:53.224723 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:50:53.224689    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eb564511-8dea-4e9e-a79d-6e78198ed7bb-istio-envoy\") pod \"eb564511-8dea-4e9e-a79d-6e78198ed7bb\" (UID: \"eb564511-8dea-4e9e-a79d-6e78198ed7bb\") "
Jul 06 14:50:53.224926 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:50:53.224733    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eb564511-8dea-4e9e-a79d-6e78198ed7bb-istio-podinfo\") pod \"eb564511-8dea-4e9e-a79d-6e78198ed7bb\" (UID: \"eb564511-8dea-4e9e-a79d-6e78198ed7bb\") "
Jul 06 14:50:53.224926 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:50:53.224782    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7bpqx\" (UniqueName: \"kubernetes.io/projected/eb564511-8dea-4e9e-a79d-6e78198ed7bb-kube-api-access-7bpqx\") pod \"eb564511-8dea-4e9e-a79d-6e78198ed7bb\" (UID: \"eb564511-8dea-4e9e-a79d-6e78198ed7bb\") "
Jul 06 14:50:53.224926 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:50:53.224820    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb564511-8dea-4e9e-a79d-6e78198ed7bb-workload-socket\") pod \"eb564511-8dea-4e9e-a79d-6e78198ed7bb\" (UID: \"eb564511-8dea-4e9e-a79d-6e78198ed7bb\") "
Jul 06 14:50:53.225099 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:50:53.225013    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eb564511-8dea-4e9e-a79d-6e78198ed7bb-istio-token\") pod \"eb564511-8dea-4e9e-a79d-6e78198ed7bb\" (UID: \"eb564511-8dea-4e9e-a79d-6e78198ed7bb\") "
Jul 06 14:50:53.225099 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:50:53.225079    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb564511-8dea-4e9e-a79d-6e78198ed7bb-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "eb564511-8dea-4e9e-a79d-6e78198ed7bb" (UID: "eb564511-8dea-4e9e-a79d-6e78198ed7bb"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 14:50:53.225210 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:50:53.225109    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eb564511-8dea-4e9e-a79d-6e78198ed7bb-istio-data\") pod \"eb564511-8dea-4e9e-a79d-6e78198ed7bb\" (UID: \"eb564511-8dea-4e9e-a79d-6e78198ed7bb\") "
Jul 06 14:50:53.225210 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:50:53.225159    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eb564511-8dea-4e9e-a79d-6e78198ed7bb-workload-certs\") pod \"eb564511-8dea-4e9e-a79d-6e78198ed7bb\" (UID: \"eb564511-8dea-4e9e-a79d-6e78198ed7bb\") "
Jul 06 14:50:53.225210 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:50:53.225189    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb564511-8dea-4e9e-a79d-6e78198ed7bb-credential-socket\") pod \"eb564511-8dea-4e9e-a79d-6e78198ed7bb\" (UID: \"eb564511-8dea-4e9e-a79d-6e78198ed7bb\") "
Jul 06 14:50:53.225375 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:50:53.225217    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eb564511-8dea-4e9e-a79d-6e78198ed7bb-istiod-ca-cert\") pod \"eb564511-8dea-4e9e-a79d-6e78198ed7bb\" (UID: \"eb564511-8dea-4e9e-a79d-6e78198ed7bb\") "
Jul 06 14:50:53.225430 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:50:53.225399    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb564511-8dea-4e9e-a79d-6e78198ed7bb-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "eb564511-8dea-4e9e-a79d-6e78198ed7bb" (UID: "eb564511-8dea-4e9e-a79d-6e78198ed7bb"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 14:50:53.225494 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:50:53.225422    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb564511-8dea-4e9e-a79d-6e78198ed7bb-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "eb564511-8dea-4e9e-a79d-6e78198ed7bb" (UID: "eb564511-8dea-4e9e-a79d-6e78198ed7bb"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 14:50:53.225649 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:50:53.225629    2559 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb564511-8dea-4e9e-a79d-6e78198ed7bb-workload-socket\") on node \"ip-10-0-133-112.ec2.internal\" DevicePath \"\""
Jul 06 14:50:53.225735 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:50:53.225653    2559 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eb564511-8dea-4e9e-a79d-6e78198ed7bb-workload-certs\") on node \"ip-10-0-133-112.ec2.internal\" DevicePath \"\""
Jul 06 14:50:53.225735 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:50:53.225668    2559 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb564511-8dea-4e9e-a79d-6e78198ed7bb-credential-socket\") on node \"ip-10-0-133-112.ec2.internal\" DevicePath \"\""
Jul 06 14:50:53.225735 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:50:53.225697    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eb564511-8dea-4e9e-a79d-6e78198ed7bb-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "eb564511-8dea-4e9e-a79d-6e78198ed7bb" (UID: "eb564511-8dea-4e9e-a79d-6e78198ed7bb"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 14:50:53.226419 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:50:53.226229    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb564511-8dea-4e9e-a79d-6e78198ed7bb-istio-data" (OuterVolumeSpecName: "istio-data") pod "eb564511-8dea-4e9e-a79d-6e78198ed7bb" (UID: "eb564511-8dea-4e9e-a79d-6e78198ed7bb"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 14:50:53.227589 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:50:53.227523    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eb564511-8dea-4e9e-a79d-6e78198ed7bb-kube-api-access-7bpqx" (OuterVolumeSpecName: "kube-api-access-7bpqx") pod "eb564511-8dea-4e9e-a79d-6e78198ed7bb" (UID: "eb564511-8dea-4e9e-a79d-6e78198ed7bb"). InnerVolumeSpecName "kube-api-access-7bpqx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 14:50:53.227771 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:50:53.227746    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eb564511-8dea-4e9e-a79d-6e78198ed7bb-istio-token" (OuterVolumeSpecName: "istio-token") pod "eb564511-8dea-4e9e-a79d-6e78198ed7bb" (UID: "eb564511-8dea-4e9e-a79d-6e78198ed7bb"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 14:50:53.228149 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:50:53.228128    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb564511-8dea-4e9e-a79d-6e78198ed7bb-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "eb564511-8dea-4e9e-a79d-6e78198ed7bb" (UID: "eb564511-8dea-4e9e-a79d-6e78198ed7bb"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 14:50:53.228466 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:50:53.228450    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/eb564511-8dea-4e9e-a79d-6e78198ed7bb-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "eb564511-8dea-4e9e-a79d-6e78198ed7bb" (UID: "eb564511-8dea-4e9e-a79d-6e78198ed7bb"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 06 14:50:53.326637 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:50:53.326599    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7bpqx\" (UniqueName: \"kubernetes.io/projected/eb564511-8dea-4e9e-a79d-6e78198ed7bb-kube-api-access-7bpqx\") on node \"ip-10-0-133-112.ec2.internal\" DevicePath \"\""
Jul 06 14:50:53.326637 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:50:53.326636    2559 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eb564511-8dea-4e9e-a79d-6e78198ed7bb-istio-token\") on node \"ip-10-0-133-112.ec2.internal\" DevicePath \"\""
Jul 06 14:50:53.326637 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:50:53.326646    2559 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eb564511-8dea-4e9e-a79d-6e78198ed7bb-istio-data\") on node \"ip-10-0-133-112.ec2.internal\" DevicePath \"\""
Jul 06 14:50:53.326876 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:50:53.326654    2559 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eb564511-8dea-4e9e-a79d-6e78198ed7bb-istiod-ca-cert\") on node \"ip-10-0-133-112.ec2.internal\" DevicePath \"\""
Jul 06 14:50:53.326876 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:50:53.326663    2559 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eb564511-8dea-4e9e-a79d-6e78198ed7bb-istio-envoy\") on node \"ip-10-0-133-112.ec2.internal\" DevicePath \"\""
Jul 06 14:50:53.326876 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:50:53.326670    2559 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eb564511-8dea-4e9e-a79d-6e78198ed7bb-istio-podinfo\") on node \"ip-10-0-133-112.ec2.internal\" DevicePath \"\""
Jul 06 14:50:53.421066 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:50:53.421024    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-ad48ac23-openshift-default-d896fdd95-gcrft"]
Jul 06 14:50:53.426136 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:50:53.426102    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-ad48ac23-openshift-default-d896fdd95-gcrft"]
Jul 06 14:50:54.228431 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:50:54.228394    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eb564511-8dea-4e9e-a79d-6e78198ed7bb" path="/var/lib/kubelet/pods/eb564511-8dea-4e9e-a79d-6e78198ed7bb/volumes"
Jul 06 14:51:45.893234 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:51:45.893203    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-3f9274-openshift-default-56c654ff75-9xq2r"]
Jul 06 14:51:45.893734 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:51:45.893603    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eb564511-8dea-4e9e-a79d-6e78198ed7bb" containerName="istio-proxy"
Jul 06 14:51:45.893734 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:51:45.893615    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="eb564511-8dea-4e9e-a79d-6e78198ed7bb" containerName="istio-proxy"
Jul 06 14:51:45.893734 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:51:45.893632    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eb564511-8dea-4e9e-a79d-6e78198ed7bb" containerName="istio-proxy"
Jul 06 14:51:45.893734 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:51:45.893638    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="eb564511-8dea-4e9e-a79d-6e78198ed7bb" containerName="istio-proxy"
Jul 06 14:51:45.893734 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:51:45.893644    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0727b0b2-db59-46b1-a2b6-b50f950299df" containerName="maas-api"
Jul 06 14:51:45.893734 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:51:45.893651    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="0727b0b2-db59-46b1-a2b6-b50f950299df" containerName="maas-api"
Jul 06 14:51:45.893734 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:51:45.893704    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="eb564511-8dea-4e9e-a79d-6e78198ed7bb" containerName="istio-proxy"
Jul 06 14:51:45.893734 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:51:45.893712    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="0727b0b2-db59-46b1-a2b6-b50f950299df" containerName="maas-api"
Jul 06 14:51:45.896867 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:51:45.896850    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-3f9274-openshift-default-56c654ff75-9xq2r"
Jul 06 14:51:45.906800 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:51:45.906783    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-3f9274-openshift-default-dockercfg-xr7nx\""
Jul 06 14:51:45.988873 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:51:45.988834    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-3f9274-openshift-default-56c654ff75-9xq2r"]
Jul 06 14:51:45.996950 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:51:45.996924    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/504db21a-2c51-47ec-94d4-c3921ba0831e-credential-socket\") pod \"e2e-inf-a-3f9274-openshift-default-56c654ff75-9xq2r\" (UID: \"504db21a-2c51-47ec-94d4-c3921ba0831e\") " pod="openshift-ingress/e2e-inf-a-3f9274-openshift-default-56c654ff75-9xq2r"
Jul 06 14:51:45.997093 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:51:45.996974    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/504db21a-2c51-47ec-94d4-c3921ba0831e-istio-data\") pod \"e2e-inf-a-3f9274-openshift-default-56c654ff75-9xq2r\" (UID: \"504db21a-2c51-47ec-94d4-c3921ba0831e\") " pod="openshift-ingress/e2e-inf-a-3f9274-openshift-default-56c654ff75-9xq2r"
Jul 06 14:51:45.997093 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:51:45.996997    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/504db21a-2c51-47ec-94d4-c3921ba0831e-istiod-ca-cert\") pod \"e2e-inf-a-3f9274-openshift-default-56c654ff75-9xq2r\" (UID: \"504db21a-2c51-47ec-94d4-c3921ba0831e\") " pod="openshift-ingress/e2e-inf-a-3f9274-openshift-default-56c654ff75-9xq2r"
Jul 06 14:51:45.997093 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:51:45.997064    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/504db21a-2c51-47ec-94d4-c3921ba0831e-workload-certs\") pod \"e2e-inf-a-3f9274-openshift-default-56c654ff75-9xq2r\" (UID: \"504db21a-2c51-47ec-94d4-c3921ba0831e\") " pod="openshift-ingress/e2e-inf-a-3f9274-openshift-default-56c654ff75-9xq2r"
Jul 06 14:51:45.997093 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:51:45.997088    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/504db21a-2c51-47ec-94d4-c3921ba0831e-workload-socket\") pod \"e2e-inf-a-3f9274-openshift-default-56c654ff75-9xq2r\" (UID: \"504db21a-2c51-47ec-94d4-c3921ba0831e\") " pod="openshift-ingress/e2e-inf-a-3f9274-openshift-default-56c654ff75-9xq2r"
Jul 06 14:51:45.997255 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:51:45.997124    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/504db21a-2c51-47ec-94d4-c3921ba0831e-istio-envoy\") pod \"e2e-inf-a-3f9274-openshift-default-56c654ff75-9xq2r\" (UID: \"504db21a-2c51-47ec-94d4-c3921ba0831e\") " pod="openshift-ingress/e2e-inf-a-3f9274-openshift-default-56c654ff75-9xq2r"
Jul 06 14:51:45.997255 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:51:45.997154    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/504db21a-2c51-47ec-94d4-c3921ba0831e-istio-token\") pod \"e2e-inf-a-3f9274-openshift-default-56c654ff75-9xq2r\" (UID: \"504db21a-2c51-47ec-94d4-c3921ba0831e\") " pod="openshift-ingress/e2e-inf-a-3f9274-openshift-default-56c654ff75-9xq2r"
Jul 06 14:51:45.997255 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:51:45.997171    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f57jj\" (UniqueName: \"kubernetes.io/projected/504db21a-2c51-47ec-94d4-c3921ba0831e-kube-api-access-f57jj\") pod \"e2e-inf-a-3f9274-openshift-default-56c654ff75-9xq2r\" (UID: \"504db21a-2c51-47ec-94d4-c3921ba0831e\") " pod="openshift-ingress/e2e-inf-a-3f9274-openshift-default-56c654ff75-9xq2r"
Jul 06 14:51:45.997255 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:51:45.997202    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/504db21a-2c51-47ec-94d4-c3921ba0831e-istio-podinfo\") pod \"e2e-inf-a-3f9274-openshift-default-56c654ff75-9xq2r\" (UID: \"504db21a-2c51-47ec-94d4-c3921ba0831e\") " pod="openshift-ingress/e2e-inf-a-3f9274-openshift-default-56c654ff75-9xq2r"
Jul 06 14:51:46.098107 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:51:46.098070    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/504db21a-2c51-47ec-94d4-c3921ba0831e-istio-data\") pod \"e2e-inf-a-3f9274-openshift-default-56c654ff75-9xq2r\" (UID: \"504db21a-2c51-47ec-94d4-c3921ba0831e\") " pod="openshift-ingress/e2e-inf-a-3f9274-openshift-default-56c654ff75-9xq2r"
Jul 06 14:51:46.098107 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:51:46.098110    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/504db21a-2c51-47ec-94d4-c3921ba0831e-istiod-ca-cert\") pod \"e2e-inf-a-3f9274-openshift-default-56c654ff75-9xq2r\" (UID: \"504db21a-2c51-47ec-94d4-c3921ba0831e\") " pod="openshift-ingress/e2e-inf-a-3f9274-openshift-default-56c654ff75-9xq2r"
Jul 06 14:51:46.098321 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:51:46.098234    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/504db21a-2c51-47ec-94d4-c3921ba0831e-workload-certs\") pod \"e2e-inf-a-3f9274-openshift-default-56c654ff75-9xq2r\" (UID: \"504db21a-2c51-47ec-94d4-c3921ba0831e\") " pod="openshift-ingress/e2e-inf-a-3f9274-openshift-default-56c654ff75-9xq2r"
Jul 06 14:51:46.098321 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:51:46.098270    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/504db21a-2c51-47ec-94d4-c3921ba0831e-workload-socket\") pod \"e2e-inf-a-3f9274-openshift-default-56c654ff75-9xq2r\" (UID: \"504db21a-2c51-47ec-94d4-c3921ba0831e\") " pod="openshift-ingress/e2e-inf-a-3f9274-openshift-default-56c654ff75-9xq2r"
Jul 06 14:51:46.098394 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:51:46.098329    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/504db21a-2c51-47ec-94d4-c3921ba0831e-istio-envoy\") pod \"e2e-inf-a-3f9274-openshift-default-56c654ff75-9xq2r\" (UID: \"504db21a-2c51-47ec-94d4-c3921ba0831e\") " pod="openshift-ingress/e2e-inf-a-3f9274-openshift-default-56c654ff75-9xq2r"
Jul 06 14:51:46.098394 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:51:46.098373    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/504db21a-2c51-47ec-94d4-c3921ba0831e-istio-token\") pod \"e2e-inf-a-3f9274-openshift-default-56c654ff75-9xq2r\" (UID: \"504db21a-2c51-47ec-94d4-c3921ba0831e\") " pod="openshift-ingress/e2e-inf-a-3f9274-openshift-default-56c654ff75-9xq2r"
Jul 06 14:51:46.098493 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:51:46.098400    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f57jj\" (UniqueName: \"kubernetes.io/projected/504db21a-2c51-47ec-94d4-c3921ba0831e-kube-api-access-f57jj\") pod \"e2e-inf-a-3f9274-openshift-default-56c654ff75-9xq2r\" (UID: \"504db21a-2c51-47ec-94d4-c3921ba0831e\") " pod="openshift-ingress/e2e-inf-a-3f9274-openshift-default-56c654ff75-9xq2r"
Jul 06 14:51:46.098493 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:51:46.098448    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/504db21a-2c51-47ec-94d4-c3921ba0831e-istio-podinfo\") pod \"e2e-inf-a-3f9274-openshift-default-56c654ff75-9xq2r\" (UID: \"504db21a-2c51-47ec-94d4-c3921ba0831e\") " pod="openshift-ingress/e2e-inf-a-3f9274-openshift-default-56c654ff75-9xq2r"
Jul 06 14:51:46.098493 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:51:46.098459    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/504db21a-2c51-47ec-94d4-c3921ba0831e-istio-data\") pod \"e2e-inf-a-3f9274-openshift-default-56c654ff75-9xq2r\" (UID: \"504db21a-2c51-47ec-94d4-c3921ba0831e\") " pod="openshift-ingress/e2e-inf-a-3f9274-openshift-default-56c654ff75-9xq2r"
Jul 06 14:51:46.098493 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:51:46.098478    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/504db21a-2c51-47ec-94d4-c3921ba0831e-credential-socket\") pod \"e2e-inf-a-3f9274-openshift-default-56c654ff75-9xq2r\" (UID: \"504db21a-2c51-47ec-94d4-c3921ba0831e\") " pod="openshift-ingress/e2e-inf-a-3f9274-openshift-default-56c654ff75-9xq2r"
Jul 06 14:51:46.098720 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:51:46.098593    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/504db21a-2c51-47ec-94d4-c3921ba0831e-workload-certs\") pod \"e2e-inf-a-3f9274-openshift-default-56c654ff75-9xq2r\" (UID: \"504db21a-2c51-47ec-94d4-c3921ba0831e\") " pod="openshift-ingress/e2e-inf-a-3f9274-openshift-default-56c654ff75-9xq2r"
Jul 06 14:51:46.098720 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:51:46.098670    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/504db21a-2c51-47ec-94d4-c3921ba0831e-workload-socket\") pod \"e2e-inf-a-3f9274-openshift-default-56c654ff75-9xq2r\" (UID: \"504db21a-2c51-47ec-94d4-c3921ba0831e\") " pod="openshift-ingress/e2e-inf-a-3f9274-openshift-default-56c654ff75-9xq2r"
Jul 06 14:51:46.098827 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:51:46.098805    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/504db21a-2c51-47ec-94d4-c3921ba0831e-credential-socket\") pod \"e2e-inf-a-3f9274-openshift-default-56c654ff75-9xq2r\" (UID: \"504db21a-2c51-47ec-94d4-c3921ba0831e\") " pod="openshift-ingress/e2e-inf-a-3f9274-openshift-default-56c654ff75-9xq2r"
Jul 06 14:51:46.098884 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:51:46.098868    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/504db21a-2c51-47ec-94d4-c3921ba0831e-istiod-ca-cert\") pod \"e2e-inf-a-3f9274-openshift-default-56c654ff75-9xq2r\" (UID: \"504db21a-2c51-47ec-94d4-c3921ba0831e\") " pod="openshift-ingress/e2e-inf-a-3f9274-openshift-default-56c654ff75-9xq2r"
Jul 06 14:51:46.100552 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:51:46.100524    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/504db21a-2c51-47ec-94d4-c3921ba0831e-istio-envoy\") pod \"e2e-inf-a-3f9274-openshift-default-56c654ff75-9xq2r\" (UID: \"504db21a-2c51-47ec-94d4-c3921ba0831e\") " pod="openshift-ingress/e2e-inf-a-3f9274-openshift-default-56c654ff75-9xq2r"
Jul 06 14:51:46.100771 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:51:46.100755    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/504db21a-2c51-47ec-94d4-c3921ba0831e-istio-podinfo\") pod \"e2e-inf-a-3f9274-openshift-default-56c654ff75-9xq2r\" (UID: \"504db21a-2c51-47ec-94d4-c3921ba0831e\") " pod="openshift-ingress/e2e-inf-a-3f9274-openshift-default-56c654ff75-9xq2r"
Jul 06 14:51:46.124169 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:51:46.124137    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/504db21a-2c51-47ec-94d4-c3921ba0831e-istio-token\") pod \"e2e-inf-a-3f9274-openshift-default-56c654ff75-9xq2r\" (UID: \"504db21a-2c51-47ec-94d4-c3921ba0831e\") " pod="openshift-ingress/e2e-inf-a-3f9274-openshift-default-56c654ff75-9xq2r"
Jul 06 14:51:46.124364 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:51:46.124339    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f57jj\" (UniqueName: \"kubernetes.io/projected/504db21a-2c51-47ec-94d4-c3921ba0831e-kube-api-access-f57jj\") pod \"e2e-inf-a-3f9274-openshift-default-56c654ff75-9xq2r\" (UID: \"504db21a-2c51-47ec-94d4-c3921ba0831e\") " pod="openshift-ingress/e2e-inf-a-3f9274-openshift-default-56c654ff75-9xq2r"
Jul 06 14:51:46.207362 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:51:46.207335    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-3f9274-openshift-default-56c654ff75-9xq2r"
Jul 06 14:51:46.366985 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:51:46.366935    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-3f9274-openshift-default-56c654ff75-9xq2r"]
Jul 06 14:51:46.371639 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:51:46.371614    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod504db21a_2c51_47ec_94d4_c3921ba0831e.slice/crio-6bb7fab634e96f262a103a043d2f85a9f997752c08a0ece7e445293c3ee54a3f WatchSource:0}: Error finding container 6bb7fab634e96f262a103a043d2f85a9f997752c08a0ece7e445293c3ee54a3f: Status 404 returned error can't find the container with id 6bb7fab634e96f262a103a043d2f85a9f997752c08a0ece7e445293c3ee54a3f
Jul 06 14:51:46.373621 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:51:46.373584    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 06 14:51:46.373693 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:51:46.373656    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 06 14:51:46.373693 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:51:46.373689    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 06 14:51:47.361864 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:51:47.361827    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-3f9274-openshift-default-56c654ff75-9xq2r" event={"ID":"504db21a-2c51-47ec-94d4-c3921ba0831e","Type":"ContainerStarted","Data":"c7e5f428956d0a92df2776b590d8291132fbd7e11cc0a0d50115fc2efe8d2798"}
Jul 06 14:51:47.362306 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:51:47.361877    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-3f9274-openshift-default-56c654ff75-9xq2r" event={"ID":"504db21a-2c51-47ec-94d4-c3921ba0831e","Type":"ContainerStarted","Data":"6bb7fab634e96f262a103a043d2f85a9f997752c08a0ece7e445293c3ee54a3f"}
Jul 06 14:51:48.207810 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:51:48.207770    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-3f9274-openshift-default-56c654ff75-9xq2r"
Jul 06 14:51:48.212599 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:51:48.212552    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-3f9274-openshift-default-56c654ff75-9xq2r"
Jul 06 14:51:48.248192 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:51:48.248131    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-3f9274-openshift-default-56c654ff75-9xq2r" podStartSLOduration=3.248110868 podStartE2EDuration="3.248110868s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:51:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:51:47.395830773 +0000 UTC m=+1929.703122855" watchObservedRunningTime="2026-07-06 14:51:48.248110868 +0000 UTC m=+1930.555402948"
Jul 06 14:51:48.367208 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:51:48.367177    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-3f9274-openshift-default-56c654ff75-9xq2r"
Jul 06 14:51:48.368209 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:51:48.368190    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-3f9274-openshift-default-56c654ff75-9xq2r"
Jul 06 14:51:49.781271 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:51:49.781240    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-inf-a-3f9274-5b877cd759-ds29f"]
Jul 06 14:51:49.781714 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:51:49.781700    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="eb564511-8dea-4e9e-a79d-6e78198ed7bb" containerName="istio-proxy"
Jul 06 14:51:49.784673 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:51:49.784645    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-3f9274-5b877cd759-ds29f"
Jul 06 14:51:49.790132 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:51:49.790109    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-inf-a-3f9274-serving-cert\""
Jul 06 14:51:49.804292 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:51:49.804269    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-3f9274-5b877cd759-ds29f"]
Jul 06 14:51:49.936103 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:51:49.936056    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a921fb9a-a0e1-42a4-bfd3-f0d50bfd7609-maas-api-tls\") pod \"maas-api-e2e-inf-a-3f9274-5b877cd759-ds29f\" (UID: \"a921fb9a-a0e1-42a4-bfd3-f0d50bfd7609\") " pod="opendatahub/maas-api-e2e-inf-a-3f9274-5b877cd759-ds29f"
Jul 06 14:51:49.936312 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:51:49.936197    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8b2k6\" (UniqueName: \"kubernetes.io/projected/a921fb9a-a0e1-42a4-bfd3-f0d50bfd7609-kube-api-access-8b2k6\") pod \"maas-api-e2e-inf-a-3f9274-5b877cd759-ds29f\" (UID: \"a921fb9a-a0e1-42a4-bfd3-f0d50bfd7609\") " pod="opendatahub/maas-api-e2e-inf-a-3f9274-5b877cd759-ds29f"
Jul 06 14:51:49.936312 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:51:49.936256    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a921fb9a-a0e1-42a4-bfd3-f0d50bfd7609-openshift-service-ca\") pod \"maas-api-e2e-inf-a-3f9274-5b877cd759-ds29f\" (UID: \"a921fb9a-a0e1-42a4-bfd3-f0d50bfd7609\") " pod="opendatahub/maas-api-e2e-inf-a-3f9274-5b877cd759-ds29f"
Jul 06 14:51:50.037364 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:51:50.037272    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8b2k6\" (UniqueName: \"kubernetes.io/projected/a921fb9a-a0e1-42a4-bfd3-f0d50bfd7609-kube-api-access-8b2k6\") pod \"maas-api-e2e-inf-a-3f9274-5b877cd759-ds29f\" (UID: \"a921fb9a-a0e1-42a4-bfd3-f0d50bfd7609\") " pod="opendatahub/maas-api-e2e-inf-a-3f9274-5b877cd759-ds29f"
Jul 06 14:51:50.037364 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:51:50.037316    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a921fb9a-a0e1-42a4-bfd3-f0d50bfd7609-openshift-service-ca\") pod \"maas-api-e2e-inf-a-3f9274-5b877cd759-ds29f\" (UID: \"a921fb9a-a0e1-42a4-bfd3-f0d50bfd7609\") " pod="opendatahub/maas-api-e2e-inf-a-3f9274-5b877cd759-ds29f"
Jul 06 14:51:50.037364 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:51:50.037343    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a921fb9a-a0e1-42a4-bfd3-f0d50bfd7609-maas-api-tls\") pod \"maas-api-e2e-inf-a-3f9274-5b877cd759-ds29f\" (UID: \"a921fb9a-a0e1-42a4-bfd3-f0d50bfd7609\") " pod="opendatahub/maas-api-e2e-inf-a-3f9274-5b877cd759-ds29f"
Jul 06 14:51:50.037693 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:51:50.037429    2559 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-inf-a-3f9274-serving-cert: secret "maas-api-e2e-inf-a-3f9274-serving-cert" not found
Jul 06 14:51:50.037693 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:51:50.037481    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a921fb9a-a0e1-42a4-bfd3-f0d50bfd7609-maas-api-tls podName:a921fb9a-a0e1-42a4-bfd3-f0d50bfd7609 nodeName:}" failed. No retries permitted until 2026-07-06 14:51:50.537466598 +0000 UTC m=+1932.844758657 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/a921fb9a-a0e1-42a4-bfd3-f0d50bfd7609-maas-api-tls") pod "maas-api-e2e-inf-a-3f9274-5b877cd759-ds29f" (UID: "a921fb9a-a0e1-42a4-bfd3-f0d50bfd7609") : secret "maas-api-e2e-inf-a-3f9274-serving-cert" not found
Jul 06 14:51:50.037935 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:51:50.037916    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a921fb9a-a0e1-42a4-bfd3-f0d50bfd7609-openshift-service-ca\") pod \"maas-api-e2e-inf-a-3f9274-5b877cd759-ds29f\" (UID: \"a921fb9a-a0e1-42a4-bfd3-f0d50bfd7609\") " pod="opendatahub/maas-api-e2e-inf-a-3f9274-5b877cd759-ds29f"
Jul 06 14:51:50.079464 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:51:50.079436    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8b2k6\" (UniqueName: \"kubernetes.io/projected/a921fb9a-a0e1-42a4-bfd3-f0d50bfd7609-kube-api-access-8b2k6\") pod \"maas-api-e2e-inf-a-3f9274-5b877cd759-ds29f\" (UID: \"a921fb9a-a0e1-42a4-bfd3-f0d50bfd7609\") " pod="opendatahub/maas-api-e2e-inf-a-3f9274-5b877cd759-ds29f"
Jul 06 14:51:50.542988 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:51:50.542938    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a921fb9a-a0e1-42a4-bfd3-f0d50bfd7609-maas-api-tls\") pod \"maas-api-e2e-inf-a-3f9274-5b877cd759-ds29f\" (UID: \"a921fb9a-a0e1-42a4-bfd3-f0d50bfd7609\") " pod="opendatahub/maas-api-e2e-inf-a-3f9274-5b877cd759-ds29f"
Jul 06 14:51:50.545358 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:51:50.545336    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a921fb9a-a0e1-42a4-bfd3-f0d50bfd7609-maas-api-tls\") pod \"maas-api-e2e-inf-a-3f9274-5b877cd759-ds29f\" (UID: \"a921fb9a-a0e1-42a4-bfd3-f0d50bfd7609\") " pod="opendatahub/maas-api-e2e-inf-a-3f9274-5b877cd759-ds29f"
Jul 06 14:51:50.696222 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:51:50.696182    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-3f9274-5b877cd759-ds29f"
Jul 06 14:51:50.852818 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:51:50.852794    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-3f9274-5b877cd759-ds29f"]
Jul 06 14:51:50.854489 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:51:50.854459    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda921fb9a_a0e1_42a4_bfd3_f0d50bfd7609.slice/crio-16b6674bc35fe54d8da04db28d5b34a91c2c8a8f9c7d7e9e51af33de8bd3fa4b WatchSource:0}: Error finding container 16b6674bc35fe54d8da04db28d5b34a91c2c8a8f9c7d7e9e51af33de8bd3fa4b: Status 404 returned error can't find the container with id 16b6674bc35fe54d8da04db28d5b34a91c2c8a8f9c7d7e9e51af33de8bd3fa4b
Jul 06 14:51:50.855831 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:51:50.855813    2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 06 14:51:51.383485 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:51:51.383442    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-3f9274-5b877cd759-ds29f" event={"ID":"a921fb9a-a0e1-42a4-bfd3-f0d50bfd7609","Type":"ContainerStarted","Data":"b8244489e2a9233a914800455cbaef5413482a431a4536a72133db7696e1c708"}
Jul 06 14:51:51.383703 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:51:51.383511    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-3f9274-5b877cd759-ds29f" event={"ID":"a921fb9a-a0e1-42a4-bfd3-f0d50bfd7609","Type":"ContainerStarted","Data":"16b6674bc35fe54d8da04db28d5b34a91c2c8a8f9c7d7e9e51af33de8bd3fa4b"}
Jul 06 14:51:51.383703 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:51:51.383537    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-inf-a-3f9274-5b877cd759-ds29f"
Jul 06 14:51:51.422132 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:51:51.422082    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-inf-a-3f9274-5b877cd759-ds29f" podStartSLOduration=2.020350937 podStartE2EDuration="2.422065446s" totalImagesPullingTime="401.714509ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:51:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:51:51.42080579 +0000 UTC m=+1933.728097871" watchObservedRunningTime="2026-07-06 14:51:51.422065446 +0000 UTC m=+1933.729357529"
Jul 06 14:51:57.402871 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:51:57.402837    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-inf-a-3f9274-5b877cd759-ds29f"
Jul 06 14:51:58.212624 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:51:58.212589    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-inf-b-421cfb-dd68bd559-8db7p"]
Jul 06 14:51:58.216359 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:51:58.216337    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-421cfb-dd68bd559-8db7p"
Jul 06 14:51:58.220446 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:51:58.220420    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-inf-b-421cfb-serving-cert\""
Jul 06 14:51:58.229829 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:51:58.229798    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-421cfb-dd68bd559-8db7p"]
Jul 06 14:51:58.316345 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:51:58.316313    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5bb9f763-587f-4123-a2cc-255d1934dead-openshift-service-ca\") pod \"maas-api-e2e-inf-b-421cfb-dd68bd559-8db7p\" (UID: \"5bb9f763-587f-4123-a2cc-255d1934dead\") " pod="opendatahub/maas-api-e2e-inf-b-421cfb-dd68bd559-8db7p"
Jul 06 14:51:58.316518 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:51:58.316376    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5bb9f763-587f-4123-a2cc-255d1934dead-maas-api-tls\") pod \"maas-api-e2e-inf-b-421cfb-dd68bd559-8db7p\" (UID: \"5bb9f763-587f-4123-a2cc-255d1934dead\") " pod="opendatahub/maas-api-e2e-inf-b-421cfb-dd68bd559-8db7p"
Jul 06 14:51:58.316518 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:51:58.316427    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kxk96\" (UniqueName: \"kubernetes.io/projected/5bb9f763-587f-4123-a2cc-255d1934dead-kube-api-access-kxk96\") pod \"maas-api-e2e-inf-b-421cfb-dd68bd559-8db7p\" (UID: \"5bb9f763-587f-4123-a2cc-255d1934dead\") " pod="opendatahub/maas-api-e2e-inf-b-421cfb-dd68bd559-8db7p"
Jul 06 14:51:58.417059 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:51:58.417023    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5bb9f763-587f-4123-a2cc-255d1934dead-openshift-service-ca\") pod \"maas-api-e2e-inf-b-421cfb-dd68bd559-8db7p\" (UID: \"5bb9f763-587f-4123-a2cc-255d1934dead\") " pod="opendatahub/maas-api-e2e-inf-b-421cfb-dd68bd559-8db7p"
Jul 06 14:51:58.417549 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:51:58.417078    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5bb9f763-587f-4123-a2cc-255d1934dead-maas-api-tls\") pod \"maas-api-e2e-inf-b-421cfb-dd68bd559-8db7p\" (UID: \"5bb9f763-587f-4123-a2cc-255d1934dead\") " pod="opendatahub/maas-api-e2e-inf-b-421cfb-dd68bd559-8db7p"
Jul 06 14:51:58.417549 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:51:58.417144    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kxk96\" (UniqueName: \"kubernetes.io/projected/5bb9f763-587f-4123-a2cc-255d1934dead-kube-api-access-kxk96\") pod \"maas-api-e2e-inf-b-421cfb-dd68bd559-8db7p\" (UID: \"5bb9f763-587f-4123-a2cc-255d1934dead\") " pod="opendatahub/maas-api-e2e-inf-b-421cfb-dd68bd559-8db7p"
Jul 06 14:51:58.417737 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:51:58.417715    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5bb9f763-587f-4123-a2cc-255d1934dead-openshift-service-ca\") pod \"maas-api-e2e-inf-b-421cfb-dd68bd559-8db7p\" (UID: \"5bb9f763-587f-4123-a2cc-255d1934dead\") " pod="opendatahub/maas-api-e2e-inf-b-421cfb-dd68bd559-8db7p"
Jul 06 14:51:58.419668 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:51:58.419647    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5bb9f763-587f-4123-a2cc-255d1934dead-maas-api-tls\") pod \"maas-api-e2e-inf-b-421cfb-dd68bd559-8db7p\" (UID: \"5bb9f763-587f-4123-a2cc-255d1934dead\") " pod="opendatahub/maas-api-e2e-inf-b-421cfb-dd68bd559-8db7p"
Jul 06 14:51:58.427635 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:51:58.427615    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kxk96\" (UniqueName: \"kubernetes.io/projected/5bb9f763-587f-4123-a2cc-255d1934dead-kube-api-access-kxk96\") pod \"maas-api-e2e-inf-b-421cfb-dd68bd559-8db7p\" (UID: \"5bb9f763-587f-4123-a2cc-255d1934dead\") " pod="opendatahub/maas-api-e2e-inf-b-421cfb-dd68bd559-8db7p"
Jul 06 14:51:58.528560 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:51:58.528461    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-421cfb-dd68bd559-8db7p"
Jul 06 14:51:58.901595 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:51:58.901456    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-421cfb-dd68bd559-8db7p"]
Jul 06 14:51:58.908310 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:51:58.907196    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5bb9f763_587f_4123_a2cc_255d1934dead.slice/crio-d3f8725fed2aeb6fcbd764ff9b9951af5ea4fce1aa40a8c19657197d84e9e63c WatchSource:0}: Error finding container d3f8725fed2aeb6fcbd764ff9b9951af5ea4fce1aa40a8c19657197d84e9e63c: Status 404 returned error can't find the container with id d3f8725fed2aeb6fcbd764ff9b9951af5ea4fce1aa40a8c19657197d84e9e63c
Jul 06 14:51:59.423563 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:51:59.423524    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-421cfb-dd68bd559-8db7p" event={"ID":"5bb9f763-587f-4123-a2cc-255d1934dead","Type":"ContainerStarted","Data":"e8fc1d4b2dba08f0baeb6df1c1c30bf5484f7b125c69161a2a068cf339ddf2ae"}
Jul 06 14:51:59.423982 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:51:59.423608    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-421cfb-dd68bd559-8db7p" event={"ID":"5bb9f763-587f-4123-a2cc-255d1934dead","Type":"ContainerStarted","Data":"d3f8725fed2aeb6fcbd764ff9b9951af5ea4fce1aa40a8c19657197d84e9e63c"}
Jul 06 14:51:59.423982 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:51:59.423716    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-inf-b-421cfb-dd68bd559-8db7p"
Jul 06 14:51:59.448870 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:51:59.448817    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-inf-b-421cfb-dd68bd559-8db7p" podStartSLOduration=1.091524565 podStartE2EDuration="1.448801126s" totalImagesPullingTime="357.276561ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:51:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:51:59.446604194 +0000 UTC m=+1941.753896273" watchObservedRunningTime="2026-07-06 14:51:59.448801126 +0000 UTC m=+1941.756093249"
Jul 06 14:52:03.368550 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:52:03.368512    2559 patch_prober.go:28] interesting pod/e2e-inf-a-3f9274-openshift-default-56c654ff75-9xq2r container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body=
Jul 06 14:52:03.369055 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:52:03.368597    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-3f9274-openshift-default-56c654ff75-9xq2r" podUID="504db21a-2c51-47ec-94d4-c3921ba0831e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused"
Jul 06 14:52:03.445444 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:52:03.445362    2559 generic.go:356] "Generic (PLEG): container finished" podID="504db21a-2c51-47ec-94d4-c3921ba0831e" containerID="c7e5f428956d0a92df2776b590d8291132fbd7e11cc0a0d50115fc2efe8d2798" exitCode=0
Jul 06 14:52:03.445444 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:52:03.445425    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-3f9274-openshift-default-56c654ff75-9xq2r" event={"ID":"504db21a-2c51-47ec-94d4-c3921ba0831e","Type":"ContainerDied","Data":"c7e5f428956d0a92df2776b590d8291132fbd7e11cc0a0d50115fc2efe8d2798"}
Jul 06 14:52:03.445832 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:52:03.445820    2559 scope.go:117] "RemoveContainer" containerID="c7e5f428956d0a92df2776b590d8291132fbd7e11cc0a0d50115fc2efe8d2798"
Jul 06 14:52:03.446861 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:52:03.446832    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 06 14:52:03.446915 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:52:03.446903    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 06 14:52:03.446950 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:52:03.446938    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 06 14:52:04.451606 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:52:04.451560    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-3f9274-openshift-default-56c654ff75-9xq2r" event={"ID":"504db21a-2c51-47ec-94d4-c3921ba0831e","Type":"ContainerStarted","Data":"7919c496884c1cfb1a0d5e283a01e7bd28991b96442c803da4dd5da9791088f6"}
Jul 06 14:52:05.208001 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:52:05.207954    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-3f9274-openshift-default-56c654ff75-9xq2r"
Jul 06 14:52:05.208289 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:52:05.208263    2559 patch_prober.go:28] interesting pod/e2e-inf-a-3f9274-openshift-default-56c654ff75-9xq2r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body=
Jul 06 14:52:05.208342 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:52:05.208322    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-3f9274-openshift-default-56c654ff75-9xq2r" podUID="504db21a-2c51-47ec-94d4-c3921ba0831e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused"
Jul 06 14:52:05.435290 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:52:05.435263    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-inf-b-421cfb-dd68bd559-8db7p"
Jul 06 14:52:06.208450 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:52:06.208410    2559 patch_prober.go:28] interesting pod/e2e-inf-a-3f9274-openshift-default-56c654ff75-9xq2r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body=
Jul 06 14:52:06.208907 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:52:06.208466    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-3f9274-openshift-default-56c654ff75-9xq2r" podUID="504db21a-2c51-47ec-94d4-c3921ba0831e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused"
Jul 06 14:52:07.207798 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:52:07.207758    2559 patch_prober.go:28] interesting pod/e2e-inf-a-3f9274-openshift-default-56c654ff75-9xq2r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body=
Jul 06 14:52:07.207988 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:52:07.207812    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-3f9274-openshift-default-56c654ff75-9xq2r" podUID="504db21a-2c51-47ec-94d4-c3921ba0831e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused"
Jul 06 14:52:08.207946 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:52:08.207900    2559 patch_prober.go:28] interesting pod/e2e-inf-a-3f9274-openshift-default-56c654ff75-9xq2r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body=
Jul 06 14:52:08.208339 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:52:08.207974    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-3f9274-openshift-default-56c654ff75-9xq2r" podUID="504db21a-2c51-47ec-94d4-c3921ba0831e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused"
Jul 06 14:52:09.211781 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:52:09.211750    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-3f9274-openshift-default-56c654ff75-9xq2r"
Jul 06 14:52:09.212224 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:52:09.212008    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-3f9274-openshift-default-56c654ff75-9xq2r"
Jul 06 14:52:09.212602 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:52:09.212586    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-3f9274-openshift-default-56c654ff75-9xq2r"
Jul 06 14:53:03.117489 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:53:03.117447    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-3f9274-5b877cd759-ds29f"]
Jul 06 14:53:03.118132 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:53:03.117801    2559 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-inf-a-3f9274-5b877cd759-ds29f" podUID="a921fb9a-a0e1-42a4-bfd3-f0d50bfd7609" containerName="maas-api" containerID="cri-o://b8244489e2a9233a914800455cbaef5413482a431a4536a72133db7696e1c708" gracePeriod=30
Jul 06 14:53:03.362302 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:53:03.362273    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-3f9274-5b877cd759-ds29f"
Jul 06 14:53:03.556823 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:53:03.556761    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a921fb9a-a0e1-42a4-bfd3-f0d50bfd7609-openshift-service-ca\") pod \"a921fb9a-a0e1-42a4-bfd3-f0d50bfd7609\" (UID: \"a921fb9a-a0e1-42a4-bfd3-f0d50bfd7609\") "
Jul 06 14:53:03.557019 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:53:03.556886    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8b2k6\" (UniqueName: \"kubernetes.io/projected/a921fb9a-a0e1-42a4-bfd3-f0d50bfd7609-kube-api-access-8b2k6\") pod \"a921fb9a-a0e1-42a4-bfd3-f0d50bfd7609\" (UID: \"a921fb9a-a0e1-42a4-bfd3-f0d50bfd7609\") "
Jul 06 14:53:03.557019 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:53:03.556940    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a921fb9a-a0e1-42a4-bfd3-f0d50bfd7609-maas-api-tls\") pod \"a921fb9a-a0e1-42a4-bfd3-f0d50bfd7609\" (UID: \"a921fb9a-a0e1-42a4-bfd3-f0d50bfd7609\") "
Jul 06 14:53:03.557220 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:53:03.557181    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a921fb9a-a0e1-42a4-bfd3-f0d50bfd7609-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a921fb9a-a0e1-42a4-bfd3-f0d50bfd7609" (UID: "a921fb9a-a0e1-42a4-bfd3-f0d50bfd7609"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 14:53:03.559171 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:53:03.559143    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a921fb9a-a0e1-42a4-bfd3-f0d50bfd7609-kube-api-access-8b2k6" (OuterVolumeSpecName: "kube-api-access-8b2k6") pod "a921fb9a-a0e1-42a4-bfd3-f0d50bfd7609" (UID: "a921fb9a-a0e1-42a4-bfd3-f0d50bfd7609"). InnerVolumeSpecName "kube-api-access-8b2k6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 14:53:03.559171 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:53:03.559162    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a921fb9a-a0e1-42a4-bfd3-f0d50bfd7609-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a921fb9a-a0e1-42a4-bfd3-f0d50bfd7609" (UID: "a921fb9a-a0e1-42a4-bfd3-f0d50bfd7609"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 14:53:03.658445 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:53:03.658406    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8b2k6\" (UniqueName: \"kubernetes.io/projected/a921fb9a-a0e1-42a4-bfd3-f0d50bfd7609-kube-api-access-8b2k6\") on node \"ip-10-0-133-112.ec2.internal\" DevicePath \"\""
Jul 06 14:53:03.658445 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:53:03.658438    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a921fb9a-a0e1-42a4-bfd3-f0d50bfd7609-maas-api-tls\") on node \"ip-10-0-133-112.ec2.internal\" DevicePath \"\""
Jul 06 14:53:03.658445 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:53:03.658448    2559 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a921fb9a-a0e1-42a4-bfd3-f0d50bfd7609-openshift-service-ca\") on node \"ip-10-0-133-112.ec2.internal\" DevicePath \"\""
Jul 06 14:53:03.744734 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:53:03.744691    2559 generic.go:356] "Generic (PLEG): container finished" podID="a921fb9a-a0e1-42a4-bfd3-f0d50bfd7609" containerID="b8244489e2a9233a914800455cbaef5413482a431a4536a72133db7696e1c708" exitCode=0
Jul 06 14:53:03.744917 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:53:03.744750    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-3f9274-5b877cd759-ds29f"
Jul 06 14:53:03.744917 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:53:03.744766    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-3f9274-5b877cd759-ds29f" event={"ID":"a921fb9a-a0e1-42a4-bfd3-f0d50bfd7609","Type":"ContainerDied","Data":"b8244489e2a9233a914800455cbaef5413482a431a4536a72133db7696e1c708"}
Jul 06 14:53:03.744917 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:53:03.744816    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-3f9274-5b877cd759-ds29f" event={"ID":"a921fb9a-a0e1-42a4-bfd3-f0d50bfd7609","Type":"ContainerDied","Data":"16b6674bc35fe54d8da04db28d5b34a91c2c8a8f9c7d7e9e51af33de8bd3fa4b"}
Jul 06 14:53:03.744917 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:53:03.744845    2559 scope.go:117] "RemoveContainer" containerID="b8244489e2a9233a914800455cbaef5413482a431a4536a72133db7696e1c708"
Jul 06 14:53:03.757183 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:53:03.757162    2559 scope.go:117] "RemoveContainer" containerID="b8244489e2a9233a914800455cbaef5413482a431a4536a72133db7696e1c708"
Jul 06 14:53:03.757449 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:53:03.757428    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b8244489e2a9233a914800455cbaef5413482a431a4536a72133db7696e1c708\": container with ID starting with b8244489e2a9233a914800455cbaef5413482a431a4536a72133db7696e1c708 not found: ID does not exist" containerID="b8244489e2a9233a914800455cbaef5413482a431a4536a72133db7696e1c708"
Jul 06 14:53:03.757531 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:53:03.757464    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b8244489e2a9233a914800455cbaef5413482a431a4536a72133db7696e1c708"} err="failed to get container status \"b8244489e2a9233a914800455cbaef5413482a431a4536a72133db7696e1c708\": rpc error: code = NotFound desc = could not find container \"b8244489e2a9233a914800455cbaef5413482a431a4536a72133db7696e1c708\": container with ID starting with b8244489e2a9233a914800455cbaef5413482a431a4536a72133db7696e1c708 not found: ID does not exist"
Jul 06 14:53:03.769691 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:53:03.769661    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-3f9274-5b877cd759-ds29f"]
Jul 06 14:53:03.773810 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:53:03.773789    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-3f9274-5b877cd759-ds29f"]
Jul 06 14:53:04.226908 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:53:04.226880    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a921fb9a-a0e1-42a4-bfd3-f0d50bfd7609" path="/var/lib/kubelet/pods/a921fb9a-a0e1-42a4-bfd3-f0d50bfd7609/volumes"
Jul 06 14:53:45.349723 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:53:45.349684    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-3f9274-openshift-default-56c654ff75-9xq2r"]
Jul 06 14:53:45.350263 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:53:45.349961    2559 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-3f9274-openshift-default-56c654ff75-9xq2r" podUID="504db21a-2c51-47ec-94d4-c3921ba0831e" containerName="istio-proxy" containerID="cri-o://7919c496884c1cfb1a0d5e283a01e7bd28991b96442c803da4dd5da9791088f6" gracePeriod=30
Jul 06 14:53:46.605631 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:53:46.605551    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-421cfb-dd68bd559-8db7p"]
Jul 06 14:53:46.606473 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:53:46.606436    2559 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-inf-b-421cfb-dd68bd559-8db7p" podUID="5bb9f763-587f-4123-a2cc-255d1934dead" containerName="maas-api" containerID="cri-o://e8fc1d4b2dba08f0baeb6df1c1c30bf5484f7b125c69161a2a068cf339ddf2ae" gracePeriod=30
Jul 06 14:53:46.865938 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:53:46.865873    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-421cfb-dd68bd559-8db7p"
Jul 06 14:53:46.952701 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:53:46.952665    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kxk96\" (UniqueName: \"kubernetes.io/projected/5bb9f763-587f-4123-a2cc-255d1934dead-kube-api-access-kxk96\") pod \"5bb9f763-587f-4123-a2cc-255d1934dead\" (UID: \"5bb9f763-587f-4123-a2cc-255d1934dead\") "
Jul 06 14:53:46.952891 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:53:46.952712    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5bb9f763-587f-4123-a2cc-255d1934dead-openshift-service-ca\") pod \"5bb9f763-587f-4123-a2cc-255d1934dead\" (UID: \"5bb9f763-587f-4123-a2cc-255d1934dead\") "
Jul 06 14:53:46.952891 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:53:46.952737    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5bb9f763-587f-4123-a2cc-255d1934dead-maas-api-tls\") pod \"5bb9f763-587f-4123-a2cc-255d1934dead\" (UID: \"5bb9f763-587f-4123-a2cc-255d1934dead\") "
Jul 06 14:53:46.953136 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:53:46.953104    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5bb9f763-587f-4123-a2cc-255d1934dead-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "5bb9f763-587f-4123-a2cc-255d1934dead" (UID: "5bb9f763-587f-4123-a2cc-255d1934dead"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 14:53:46.954883 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:53:46.954858    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5bb9f763-587f-4123-a2cc-255d1934dead-kube-api-access-kxk96" (OuterVolumeSpecName: "kube-api-access-kxk96") pod "5bb9f763-587f-4123-a2cc-255d1934dead" (UID: "5bb9f763-587f-4123-a2cc-255d1934dead"). InnerVolumeSpecName "kube-api-access-kxk96". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 14:53:46.954883 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:53:46.954874    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5bb9f763-587f-4123-a2cc-255d1934dead-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "5bb9f763-587f-4123-a2cc-255d1934dead" (UID: "5bb9f763-587f-4123-a2cc-255d1934dead"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 14:53:46.956771 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:53:46.956747    2559 generic.go:356] "Generic (PLEG): container finished" podID="5bb9f763-587f-4123-a2cc-255d1934dead" containerID="e8fc1d4b2dba08f0baeb6df1c1c30bf5484f7b125c69161a2a068cf339ddf2ae" exitCode=0
Jul 06 14:53:46.956877 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:53:46.956801    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-421cfb-dd68bd559-8db7p"
Jul 06 14:53:46.956877 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:53:46.956829    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-421cfb-dd68bd559-8db7p" event={"ID":"5bb9f763-587f-4123-a2cc-255d1934dead","Type":"ContainerDied","Data":"e8fc1d4b2dba08f0baeb6df1c1c30bf5484f7b125c69161a2a068cf339ddf2ae"}
Jul 06 14:53:46.956955 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:53:46.956884    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-421cfb-dd68bd559-8db7p" event={"ID":"5bb9f763-587f-4123-a2cc-255d1934dead","Type":"ContainerDied","Data":"d3f8725fed2aeb6fcbd764ff9b9951af5ea4fce1aa40a8c19657197d84e9e63c"}
Jul 06 14:53:46.956955 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:53:46.956910    2559 scope.go:117] "RemoveContainer" containerID="e8fc1d4b2dba08f0baeb6df1c1c30bf5484f7b125c69161a2a068cf339ddf2ae"
Jul 06 14:53:46.977784 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:53:46.977766    2559 scope.go:117] "RemoveContainer" containerID="e8fc1d4b2dba08f0baeb6df1c1c30bf5484f7b125c69161a2a068cf339ddf2ae"
Jul 06 14:53:46.978086 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:53:46.978064    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e8fc1d4b2dba08f0baeb6df1c1c30bf5484f7b125c69161a2a068cf339ddf2ae\": container with ID starting with e8fc1d4b2dba08f0baeb6df1c1c30bf5484f7b125c69161a2a068cf339ddf2ae not found: ID does not exist" containerID="e8fc1d4b2dba08f0baeb6df1c1c30bf5484f7b125c69161a2a068cf339ddf2ae"
Jul 06 14:53:46.978161 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:53:46.978095    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e8fc1d4b2dba08f0baeb6df1c1c30bf5484f7b125c69161a2a068cf339ddf2ae"} err="failed to get container status \"e8fc1d4b2dba08f0baeb6df1c1c30bf5484f7b125c69161a2a068cf339ddf2ae\": rpc error: code = NotFound desc = could not find container \"e8fc1d4b2dba08f0baeb6df1c1c30bf5484f7b125c69161a2a068cf339ddf2ae\": container with ID starting with e8fc1d4b2dba08f0baeb6df1c1c30bf5484f7b125c69161a2a068cf339ddf2ae not found: ID does not exist"
Jul 06 14:53:47.000127 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:53:47.000090    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-421cfb-dd68bd559-8db7p"]
Jul 06 14:53:47.011819 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:53:47.011786    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-421cfb-dd68bd559-8db7p"]
Jul 06 14:53:47.054192 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:53:47.054158    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kxk96\" (UniqueName: \"kubernetes.io/projected/5bb9f763-587f-4123-a2cc-255d1934dead-kube-api-access-kxk96\") on node \"ip-10-0-133-112.ec2.internal\" DevicePath \"\""
Jul 06 14:53:47.054192 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:53:47.054188    2559 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5bb9f763-587f-4123-a2cc-255d1934dead-openshift-service-ca\") on node \"ip-10-0-133-112.ec2.internal\" DevicePath \"\""
Jul 06 14:53:47.054192 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:53:47.054198    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5bb9f763-587f-4123-a2cc-255d1934dead-maas-api-tls\") on node \"ip-10-0-133-112.ec2.internal\" DevicePath \"\""
Jul 06 14:53:48.227298 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:53:48.227263    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5bb9f763-587f-4123-a2cc-255d1934dead" path="/var/lib/kubelet/pods/5bb9f763-587f-4123-a2cc-255d1934dead/volumes"
Jul 06 14:53:50.613426 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:53:50.613355    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-3f9274-openshift-default-56c654ff75-9xq2r"
Jul 06 14:53:50.687005 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:53:50.686971    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/504db21a-2c51-47ec-94d4-c3921ba0831e-istio-podinfo\") pod \"504db21a-2c51-47ec-94d4-c3921ba0831e\" (UID: \"504db21a-2c51-47ec-94d4-c3921ba0831e\") "
Jul 06 14:53:50.687204 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:53:50.687016    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/504db21a-2c51-47ec-94d4-c3921ba0831e-credential-socket\") pod \"504db21a-2c51-47ec-94d4-c3921ba0831e\" (UID: \"504db21a-2c51-47ec-94d4-c3921ba0831e\") "
Jul 06 14:53:50.687204 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:53:50.687039    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/504db21a-2c51-47ec-94d4-c3921ba0831e-istio-envoy\") pod \"504db21a-2c51-47ec-94d4-c3921ba0831e\" (UID: \"504db21a-2c51-47ec-94d4-c3921ba0831e\") "
Jul 06 14:53:50.687204 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:53:50.687064    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/504db21a-2c51-47ec-94d4-c3921ba0831e-workload-certs\") pod \"504db21a-2c51-47ec-94d4-c3921ba0831e\" (UID: \"504db21a-2c51-47ec-94d4-c3921ba0831e\") "
Jul 06 14:53:50.687204 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:53:50.687108    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/504db21a-2c51-47ec-94d4-c3921ba0831e-istiod-ca-cert\") pod \"504db21a-2c51-47ec-94d4-c3921ba0831e\" (UID: \"504db21a-2c51-47ec-94d4-c3921ba0831e\") "
Jul 06 14:53:50.687204 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:53:50.687164    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/504db21a-2c51-47ec-94d4-c3921ba0831e-workload-socket\") pod \"504db21a-2c51-47ec-94d4-c3921ba0831e\" (UID: \"504db21a-2c51-47ec-94d4-c3921ba0831e\") "
Jul 06 14:53:50.687204 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:53:50.687195    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f57jj\" (UniqueName: \"kubernetes.io/projected/504db21a-2c51-47ec-94d4-c3921ba0831e-kube-api-access-f57jj\") pod \"504db21a-2c51-47ec-94d4-c3921ba0831e\" (UID: \"504db21a-2c51-47ec-94d4-c3921ba0831e\") "
Jul 06 14:53:50.687516 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:53:50.687231    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/504db21a-2c51-47ec-94d4-c3921ba0831e-istio-data\") pod \"504db21a-2c51-47ec-94d4-c3921ba0831e\" (UID: \"504db21a-2c51-47ec-94d4-c3921ba0831e\") "
Jul 06 14:53:50.687516 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:53:50.687292    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/504db21a-2c51-47ec-94d4-c3921ba0831e-istio-token\") pod \"504db21a-2c51-47ec-94d4-c3921ba0831e\" (UID: \"504db21a-2c51-47ec-94d4-c3921ba0831e\") "
Jul 06 14:53:50.687516 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:53:50.687382    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/504db21a-2c51-47ec-94d4-c3921ba0831e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "504db21a-2c51-47ec-94d4-c3921ba0831e" (UID: "504db21a-2c51-47ec-94d4-c3921ba0831e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 14:53:50.687516 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:53:50.687439    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/504db21a-2c51-47ec-94d4-c3921ba0831e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "504db21a-2c51-47ec-94d4-c3921ba0831e" (UID: "504db21a-2c51-47ec-94d4-c3921ba0831e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 14:53:50.687516 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:53:50.687459    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/504db21a-2c51-47ec-94d4-c3921ba0831e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "504db21a-2c51-47ec-94d4-c3921ba0831e" (UID: "504db21a-2c51-47ec-94d4-c3921ba0831e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 14:53:50.687735 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:53:50.687641    2559 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/504db21a-2c51-47ec-94d4-c3921ba0831e-credential-socket\") on node \"ip-10-0-133-112.ec2.internal\" DevicePath \"\""
Jul 06 14:53:50.687735 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:53:50.687660    2559 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/504db21a-2c51-47ec-94d4-c3921ba0831e-workload-certs\") on node \"ip-10-0-133-112.ec2.internal\" DevicePath \"\""
Jul 06 14:53:50.687735 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:53:50.687673    2559 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/504db21a-2c51-47ec-94d4-c3921ba0831e-workload-socket\") on node \"ip-10-0-133-112.ec2.internal\" DevicePath \"\""
Jul 06 14:53:50.687735 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:53:50.687671    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/504db21a-2c51-47ec-94d4-c3921ba0831e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "504db21a-2c51-47ec-94d4-c3921ba0831e" (UID: "504db21a-2c51-47ec-94d4-c3921ba0831e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 14:53:50.688387 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:53:50.688355    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/504db21a-2c51-47ec-94d4-c3921ba0831e-istio-data" (OuterVolumeSpecName: "istio-data") pod "504db21a-2c51-47ec-94d4-c3921ba0831e" (UID: "504db21a-2c51-47ec-94d4-c3921ba0831e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 14:53:50.689438 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:53:50.689405    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/504db21a-2c51-47ec-94d4-c3921ba0831e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "504db21a-2c51-47ec-94d4-c3921ba0831e" (UID: "504db21a-2c51-47ec-94d4-c3921ba0831e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 06 14:53:50.689563 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:53:50.689544    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/504db21a-2c51-47ec-94d4-c3921ba0831e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "504db21a-2c51-47ec-94d4-c3921ba0831e" (UID: "504db21a-2c51-47ec-94d4-c3921ba0831e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 14:53:50.689885 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:53:50.689863    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/504db21a-2c51-47ec-94d4-c3921ba0831e-istio-token" (OuterVolumeSpecName: "istio-token") pod "504db21a-2c51-47ec-94d4-c3921ba0831e" (UID: "504db21a-2c51-47ec-94d4-c3921ba0831e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 14:53:50.689979 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:53:50.689889    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/504db21a-2c51-47ec-94d4-c3921ba0831e-kube-api-access-f57jj" (OuterVolumeSpecName: "kube-api-access-f57jj") pod "504db21a-2c51-47ec-94d4-c3921ba0831e" (UID: "504db21a-2c51-47ec-94d4-c3921ba0831e"). InnerVolumeSpecName "kube-api-access-f57jj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 14:53:50.789066 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:53:50.789026    2559 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/504db21a-2c51-47ec-94d4-c3921ba0831e-istio-token\") on node \"ip-10-0-133-112.ec2.internal\" DevicePath \"\""
Jul 06 14:53:50.789066 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:53:50.789060    2559 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/504db21a-2c51-47ec-94d4-c3921ba0831e-istio-podinfo\") on node \"ip-10-0-133-112.ec2.internal\" DevicePath \"\""
Jul 06 14:53:50.789066 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:53:50.789069    2559 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/504db21a-2c51-47ec-94d4-c3921ba0831e-istio-envoy\") on node \"ip-10-0-133-112.ec2.internal\" DevicePath \"\""
Jul 06 14:53:50.789066 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:53:50.789080    2559 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/504db21a-2c51-47ec-94d4-c3921ba0831e-istiod-ca-cert\") on node \"ip-10-0-133-112.ec2.internal\" DevicePath \"\""
Jul 06 14:53:50.789352 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:53:50.789090    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f57jj\" (UniqueName: \"kubernetes.io/projected/504db21a-2c51-47ec-94d4-c3921ba0831e-kube-api-access-f57jj\") on node \"ip-10-0-133-112.ec2.internal\" DevicePath \"\""
Jul 06 14:53:50.789352 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:53:50.789098    2559 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/504db21a-2c51-47ec-94d4-c3921ba0831e-istio-data\") on node \"ip-10-0-133-112.ec2.internal\" DevicePath \"\""
Jul 06 14:53:50.978056 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:53:50.978018    2559 generic.go:356] "Generic (PLEG): container finished" podID="504db21a-2c51-47ec-94d4-c3921ba0831e" containerID="7919c496884c1cfb1a0d5e283a01e7bd28991b96442c803da4dd5da9791088f6" exitCode=0
Jul 06 14:53:50.978233 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:53:50.978076    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-3f9274-openshift-default-56c654ff75-9xq2r"
Jul 06 14:53:50.978233 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:53:50.978095    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-3f9274-openshift-default-56c654ff75-9xq2r" event={"ID":"504db21a-2c51-47ec-94d4-c3921ba0831e","Type":"ContainerDied","Data":"7919c496884c1cfb1a0d5e283a01e7bd28991b96442c803da4dd5da9791088f6"}
Jul 06 14:53:50.978233 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:53:50.978145    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-3f9274-openshift-default-56c654ff75-9xq2r" event={"ID":"504db21a-2c51-47ec-94d4-c3921ba0831e","Type":"ContainerDied","Data":"6bb7fab634e96f262a103a043d2f85a9f997752c08a0ece7e445293c3ee54a3f"}
Jul 06 14:53:50.978233 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:53:50.978168    2559 scope.go:117] "RemoveContainer" containerID="7919c496884c1cfb1a0d5e283a01e7bd28991b96442c803da4dd5da9791088f6"
Jul 06 14:53:50.987335 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:53:50.987317    2559 scope.go:117] "RemoveContainer" containerID="c7e5f428956d0a92df2776b590d8291132fbd7e11cc0a0d50115fc2efe8d2798"
Jul 06 14:53:50.995482 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:53:50.995461    2559 scope.go:117] "RemoveContainer" containerID="7919c496884c1cfb1a0d5e283a01e7bd28991b96442c803da4dd5da9791088f6"
Jul 06 14:53:50.995782 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:53:50.995761    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7919c496884c1cfb1a0d5e283a01e7bd28991b96442c803da4dd5da9791088f6\": container with ID starting with 7919c496884c1cfb1a0d5e283a01e7bd28991b96442c803da4dd5da9791088f6 not found: ID does not exist" containerID="7919c496884c1cfb1a0d5e283a01e7bd28991b96442c803da4dd5da9791088f6"
Jul 06 14:53:50.995862 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:53:50.995790    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7919c496884c1cfb1a0d5e283a01e7bd28991b96442c803da4dd5da9791088f6"} err="failed to get container status \"7919c496884c1cfb1a0d5e283a01e7bd28991b96442c803da4dd5da9791088f6\": rpc error: code = NotFound desc = could not find container \"7919c496884c1cfb1a0d5e283a01e7bd28991b96442c803da4dd5da9791088f6\": container with ID starting with 7919c496884c1cfb1a0d5e283a01e7bd28991b96442c803da4dd5da9791088f6 not found: ID does not exist"
Jul 06 14:53:50.995862 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:53:50.995808    2559 scope.go:117] "RemoveContainer" containerID="c7e5f428956d0a92df2776b590d8291132fbd7e11cc0a0d50115fc2efe8d2798"
Jul 06 14:53:50.996057 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:53:50.996041    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c7e5f428956d0a92df2776b590d8291132fbd7e11cc0a0d50115fc2efe8d2798\": container with ID starting with c7e5f428956d0a92df2776b590d8291132fbd7e11cc0a0d50115fc2efe8d2798 not found: ID does not exist" containerID="c7e5f428956d0a92df2776b590d8291132fbd7e11cc0a0d50115fc2efe8d2798"
Jul 06 14:53:50.996096 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:53:50.996063    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c7e5f428956d0a92df2776b590d8291132fbd7e11cc0a0d50115fc2efe8d2798"} err="failed to get container status \"c7e5f428956d0a92df2776b590d8291132fbd7e11cc0a0d50115fc2efe8d2798\": rpc error: code = NotFound desc = could not find container \"c7e5f428956d0a92df2776b590d8291132fbd7e11cc0a0d50115fc2efe8d2798\": container with ID starting with c7e5f428956d0a92df2776b590d8291132fbd7e11cc0a0d50115fc2efe8d2798 not found: ID does not exist"
Jul 06 14:53:51.005146 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:53:51.005117    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-3f9274-openshift-default-56c654ff75-9xq2r"]
Jul 06 14:53:51.018317 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:53:51.018286    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-3f9274-openshift-default-56c654ff75-9xq2r"]
Jul 06 14:53:52.228177 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:53:52.228142    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="504db21a-2c51-47ec-94d4-c3921ba0831e" path="/var/lib/kubelet/pods/504db21a-2c51-47ec-94d4-c3921ba0831e/volumes"
Jul 06 14:53:54.153384 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:53:54.153352    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-5c757b-openshift-default-7d9c6fc49-rqn6k"]
Jul 06 14:53:54.153793 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:53:54.153742    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="504db21a-2c51-47ec-94d4-c3921ba0831e" containerName="istio-proxy"
Jul 06 14:53:54.153793 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:53:54.153754    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="504db21a-2c51-47ec-94d4-c3921ba0831e" containerName="istio-proxy"
Jul 06 14:53:54.153793 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:53:54.153775    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5bb9f763-587f-4123-a2cc-255d1934dead" containerName="maas-api"
Jul 06 14:53:54.153793 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:53:54.153781    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="5bb9f763-587f-4123-a2cc-255d1934dead" containerName="maas-api"
Jul 06 14:53:54.153793 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:53:54.153794    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a921fb9a-a0e1-42a4-bfd3-f0d50bfd7609" containerName="maas-api"
Jul 06 14:53:54.153959 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:53:54.153799    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="a921fb9a-a0e1-42a4-bfd3-f0d50bfd7609" containerName="maas-api"
Jul 06 14:53:54.153959 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:53:54.153854    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="5bb9f763-587f-4123-a2cc-255d1934dead" containerName="maas-api"
Jul 06 14:53:54.153959 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:53:54.153865    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="504db21a-2c51-47ec-94d4-c3921ba0831e" containerName="istio-proxy"
Jul 06 14:53:54.153959 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:53:54.153873    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="a921fb9a-a0e1-42a4-bfd3-f0d50bfd7609" containerName="maas-api"
Jul 06 14:53:54.153959 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:53:54.153879    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="504db21a-2c51-47ec-94d4-c3921ba0831e" containerName="istio-proxy"
Jul 06 14:53:54.158326 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:53:54.158306    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-5c757b-openshift-default-7d9c6fc49-rqn6k"
Jul 06 14:53:54.161049 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:53:54.161026    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-5c757b-openshift-default-dockercfg-96ztf\""
Jul 06 14:53:54.171287 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:53:54.171258    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-5c757b-openshift-default-7d9c6fc49-rqn6k"]
Jul 06 14:53:54.218368 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:53:54.218330    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d7c70639-a47b-4adf-8a6b-5ff2cbbe458c-workload-certs\") pod \"e2e-api-a-5c757b-openshift-default-7d9c6fc49-rqn6k\" (UID: \"d7c70639-a47b-4adf-8a6b-5ff2cbbe458c\") " pod="openshift-ingress/e2e-api-a-5c757b-openshift-default-7d9c6fc49-rqn6k"
Jul 06 14:53:54.218600 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:53:54.218379    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7c70639-a47b-4adf-8a6b-5ff2cbbe458c-credential-socket\") pod \"e2e-api-a-5c757b-openshift-default-7d9c6fc49-rqn6k\" (UID: \"d7c70639-a47b-4adf-8a6b-5ff2cbbe458c\") " pod="openshift-ingress/e2e-api-a-5c757b-openshift-default-7d9c6fc49-rqn6k"
Jul 06 14:53:54.218600 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:53:54.218397    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d7c70639-a47b-4adf-8a6b-5ff2cbbe458c-istio-envoy\") pod \"e2e-api-a-5c757b-openshift-default-7d9c6fc49-rqn6k\" (UID: \"d7c70639-a47b-4adf-8a6b-5ff2cbbe458c\") " pod="openshift-ingress/e2e-api-a-5c757b-openshift-default-7d9c6fc49-rqn6k"
Jul 06 14:53:54.218600 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:53:54.218511    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7c70639-a47b-4adf-8a6b-5ff2cbbe458c-workload-socket\") pod \"e2e-api-a-5c757b-openshift-default-7d9c6fc49-rqn6k\" (UID: \"d7c70639-a47b-4adf-8a6b-5ff2cbbe458c\") " pod="openshift-ingress/e2e-api-a-5c757b-openshift-default-7d9c6fc49-rqn6k"
Jul 06 14:53:54.218600 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:53:54.218542    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d7c70639-a47b-4adf-8a6b-5ff2cbbe458c-istio-data\") pod \"e2e-api-a-5c757b-openshift-default-7d9c6fc49-rqn6k\" (UID: \"d7c70639-a47b-4adf-8a6b-5ff2cbbe458c\") " pod="openshift-ingress/e2e-api-a-5c757b-openshift-default-7d9c6fc49-rqn6k"
Jul 06 14:53:54.218600 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:53:54.218566    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d7c70639-a47b-4adf-8a6b-5ff2cbbe458c-istio-token\") pod \"e2e-api-a-5c757b-openshift-default-7d9c6fc49-rqn6k\" (UID: \"d7c70639-a47b-4adf-8a6b-5ff2cbbe458c\") " pod="openshift-ingress/e2e-api-a-5c757b-openshift-default-7d9c6fc49-rqn6k"
Jul 06 14:53:54.218808 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:53:54.218629    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d7c70639-a47b-4adf-8a6b-5ff2cbbe458c-istiod-ca-cert\") pod \"e2e-api-a-5c757b-openshift-default-7d9c6fc49-rqn6k\" (UID: \"d7c70639-a47b-4adf-8a6b-5ff2cbbe458c\") " pod="openshift-ingress/e2e-api-a-5c757b-openshift-default-7d9c6fc49-rqn6k"
Jul 06 14:53:54.218808 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:53:54.218707    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d7c70639-a47b-4adf-8a6b-5ff2cbbe458c-istio-podinfo\") pod \"e2e-api-a-5c757b-openshift-default-7d9c6fc49-rqn6k\" (UID: \"d7c70639-a47b-4adf-8a6b-5ff2cbbe458c\") " pod="openshift-ingress/e2e-api-a-5c757b-openshift-default-7d9c6fc49-rqn6k"
Jul 06 14:53:54.218808 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:53:54.218731    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dpd5d\" (UniqueName: \"kubernetes.io/projected/d7c70639-a47b-4adf-8a6b-5ff2cbbe458c-kube-api-access-dpd5d\") pod \"e2e-api-a-5c757b-openshift-default-7d9c6fc49-rqn6k\" (UID: \"d7c70639-a47b-4adf-8a6b-5ff2cbbe458c\") " pod="openshift-ingress/e2e-api-a-5c757b-openshift-default-7d9c6fc49-rqn6k"
Jul 06 14:53:54.319492 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:53:54.319443    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d7c70639-a47b-4adf-8a6b-5ff2cbbe458c-workload-certs\") pod \"e2e-api-a-5c757b-openshift-default-7d9c6fc49-rqn6k\" (UID: \"d7c70639-a47b-4adf-8a6b-5ff2cbbe458c\") " pod="openshift-ingress/e2e-api-a-5c757b-openshift-default-7d9c6fc49-rqn6k"
Jul 06 14:53:54.319764 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:53:54.319527    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7c70639-a47b-4adf-8a6b-5ff2cbbe458c-credential-socket\") pod \"e2e-api-a-5c757b-openshift-default-7d9c6fc49-rqn6k\" (UID: \"d7c70639-a47b-4adf-8a6b-5ff2cbbe458c\") " pod="openshift-ingress/e2e-api-a-5c757b-openshift-default-7d9c6fc49-rqn6k"
Jul 06 14:53:54.319764 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:53:54.319551    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d7c70639-a47b-4adf-8a6b-5ff2cbbe458c-istio-envoy\") pod \"e2e-api-a-5c757b-openshift-default-7d9c6fc49-rqn6k\" (UID: \"d7c70639-a47b-4adf-8a6b-5ff2cbbe458c\") " pod="openshift-ingress/e2e-api-a-5c757b-openshift-default-7d9c6fc49-rqn6k"
Jul 06 14:53:54.319764 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:53:54.319652    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7c70639-a47b-4adf-8a6b-5ff2cbbe458c-workload-socket\") pod \"e2e-api-a-5c757b-openshift-default-7d9c6fc49-rqn6k\" (UID: \"d7c70639-a47b-4adf-8a6b-5ff2cbbe458c\") " pod="openshift-ingress/e2e-api-a-5c757b-openshift-default-7d9c6fc49-rqn6k"
Jul 06 14:53:54.319764 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:53:54.319687    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d7c70639-a47b-4adf-8a6b-5ff2cbbe458c-istio-data\") pod \"e2e-api-a-5c757b-openshift-default-7d9c6fc49-rqn6k\" (UID: \"d7c70639-a47b-4adf-8a6b-5ff2cbbe458c\") " pod="openshift-ingress/e2e-api-a-5c757b-openshift-default-7d9c6fc49-rqn6k"
Jul 06 14:53:54.319764 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:53:54.319724    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d7c70639-a47b-4adf-8a6b-5ff2cbbe458c-istio-token\") pod \"e2e-api-a-5c757b-openshift-default-7d9c6fc49-rqn6k\" (UID: \"d7c70639-a47b-4adf-8a6b-5ff2cbbe458c\") " pod="openshift-ingress/e2e-api-a-5c757b-openshift-default-7d9c6fc49-rqn6k"
Jul 06 14:53:54.319764 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:53:54.319766    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d7c70639-a47b-4adf-8a6b-5ff2cbbe458c-istiod-ca-cert\") pod \"e2e-api-a-5c757b-openshift-default-7d9c6fc49-rqn6k\" (UID: \"d7c70639-a47b-4adf-8a6b-5ff2cbbe458c\") " pod="openshift-ingress/e2e-api-a-5c757b-openshift-default-7d9c6fc49-rqn6k"
Jul 06 14:53:54.320097 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:53:54.319943    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d7c70639-a47b-4adf-8a6b-5ff2cbbe458c-istio-podinfo\") pod \"e2e-api-a-5c757b-openshift-default-7d9c6fc49-rqn6k\" (UID: \"d7c70639-a47b-4adf-8a6b-5ff2cbbe458c\") " pod="openshift-ingress/e2e-api-a-5c757b-openshift-default-7d9c6fc49-rqn6k"
Jul 06 14:53:54.320097 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:53:54.319991    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dpd5d\" (UniqueName: \"kubernetes.io/projected/d7c70639-a47b-4adf-8a6b-5ff2cbbe458c-kube-api-access-dpd5d\") pod \"e2e-api-a-5c757b-openshift-default-7d9c6fc49-rqn6k\" (UID: \"d7c70639-a47b-4adf-8a6b-5ff2cbbe458c\") " pod="openshift-ingress/e2e-api-a-5c757b-openshift-default-7d9c6fc49-rqn6k"
Jul 06 14:53:54.320210 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:53:54.320168    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d7c70639-a47b-4adf-8a6b-5ff2cbbe458c-istio-data\") pod \"e2e-api-a-5c757b-openshift-default-7d9c6fc49-rqn6k\" (UID: \"d7c70639-a47b-4adf-8a6b-5ff2cbbe458c\") " pod="openshift-ingress/e2e-api-a-5c757b-openshift-default-7d9c6fc49-rqn6k"
Jul 06 14:53:54.320268 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:53:54.320244    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7c70639-a47b-4adf-8a6b-5ff2cbbe458c-workload-socket\") pod \"e2e-api-a-5c757b-openshift-default-7d9c6fc49-rqn6k\" (UID: \"d7c70639-a47b-4adf-8a6b-5ff2cbbe458c\") " pod="openshift-ingress/e2e-api-a-5c757b-openshift-default-7d9c6fc49-rqn6k"
Jul 06 14:53:54.320504 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:53:54.320478    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7c70639-a47b-4adf-8a6b-5ff2cbbe458c-credential-socket\") pod \"e2e-api-a-5c757b-openshift-default-7d9c6fc49-rqn6k\" (UID: \"d7c70639-a47b-4adf-8a6b-5ff2cbbe458c\") " pod="openshift-ingress/e2e-api-a-5c757b-openshift-default-7d9c6fc49-rqn6k"
Jul 06 14:53:54.320711 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:53:54.320557    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d7c70639-a47b-4adf-8a6b-5ff2cbbe458c-istiod-ca-cert\") pod \"e2e-api-a-5c757b-openshift-default-7d9c6fc49-rqn6k\" (UID: \"d7c70639-a47b-4adf-8a6b-5ff2cbbe458c\") " pod="openshift-ingress/e2e-api-a-5c757b-openshift-default-7d9c6fc49-rqn6k"
Jul 06 14:53:54.320711 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:53:54.320619    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d7c70639-a47b-4adf-8a6b-5ff2cbbe458c-workload-certs\") pod \"e2e-api-a-5c757b-openshift-default-7d9c6fc49-rqn6k\" (UID: \"d7c70639-a47b-4adf-8a6b-5ff2cbbe458c\") " pod="openshift-ingress/e2e-api-a-5c757b-openshift-default-7d9c6fc49-rqn6k"
Jul 06 14:53:54.322518 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:53:54.322495    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d7c70639-a47b-4adf-8a6b-5ff2cbbe458c-istio-envoy\") pod \"e2e-api-a-5c757b-openshift-default-7d9c6fc49-rqn6k\" (UID: \"d7c70639-a47b-4adf-8a6b-5ff2cbbe458c\") " pod="openshift-ingress/e2e-api-a-5c757b-openshift-default-7d9c6fc49-rqn6k"
Jul 06 14:53:54.322681 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:53:54.322662    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d7c70639-a47b-4adf-8a6b-5ff2cbbe458c-istio-podinfo\") pod \"e2e-api-a-5c757b-openshift-default-7d9c6fc49-rqn6k\" (UID: \"d7c70639-a47b-4adf-8a6b-5ff2cbbe458c\") " pod="openshift-ingress/e2e-api-a-5c757b-openshift-default-7d9c6fc49-rqn6k"
Jul 06 14:53:54.329675 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:53:54.329644    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d7c70639-a47b-4adf-8a6b-5ff2cbbe458c-istio-token\") pod \"e2e-api-a-5c757b-openshift-default-7d9c6fc49-rqn6k\" (UID: \"d7c70639-a47b-4adf-8a6b-5ff2cbbe458c\") " pod="openshift-ingress/e2e-api-a-5c757b-openshift-default-7d9c6fc49-rqn6k"
Jul 06 14:53:54.330303 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:53:54.330274    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dpd5d\" (UniqueName: \"kubernetes.io/projected/d7c70639-a47b-4adf-8a6b-5ff2cbbe458c-kube-api-access-dpd5d\") pod \"e2e-api-a-5c757b-openshift-default-7d9c6fc49-rqn6k\" (UID: \"d7c70639-a47b-4adf-8a6b-5ff2cbbe458c\") " pod="openshift-ingress/e2e-api-a-5c757b-openshift-default-7d9c6fc49-rqn6k"
Jul 06 14:53:54.470398 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:53:54.470356    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-5c757b-openshift-default-7d9c6fc49-rqn6k"
Jul 06 14:53:54.624448 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:53:54.624422    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-5c757b-openshift-default-7d9c6fc49-rqn6k"]
Jul 06 14:53:54.626718 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:53:54.626688    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7c70639_a47b_4adf_8a6b_5ff2cbbe458c.slice/crio-825b06c1466b98b7d4e32827b123aceebbe8c56aca54cf7b392a240dfe178ff3 WatchSource:0}: Error finding container 825b06c1466b98b7d4e32827b123aceebbe8c56aca54cf7b392a240dfe178ff3: Status 404 returned error can't find the container with id 825b06c1466b98b7d4e32827b123aceebbe8c56aca54cf7b392a240dfe178ff3
Jul 06 14:53:54.628774 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:53:54.628738    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 06 14:53:54.628868 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:53:54.628805    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 06 14:53:54.628868 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:53:54.628840    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 06 14:53:55.002821 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:53:55.002785    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-5c757b-openshift-default-7d9c6fc49-rqn6k" event={"ID":"d7c70639-a47b-4adf-8a6b-5ff2cbbe458c","Type":"ContainerStarted","Data":"3e04734f648241506ba9e6eb9f78ba73b34eb9db0b85271ea758a1708da18659"}
Jul 06 14:53:55.002997 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:53:55.002845    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-5c757b-openshift-default-7d9c6fc49-rqn6k" event={"ID":"d7c70639-a47b-4adf-8a6b-5ff2cbbe458c","Type":"ContainerStarted","Data":"825b06c1466b98b7d4e32827b123aceebbe8c56aca54cf7b392a240dfe178ff3"}
Jul 06 14:53:55.029402 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:53:55.029352    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-5c757b-openshift-default-7d9c6fc49-rqn6k" podStartSLOduration=1.029336632 podStartE2EDuration="1.029336632s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:53:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:53:55.027206971 +0000 UTC m=+2057.334499067" watchObservedRunningTime="2026-07-06 14:53:55.029336632 +0000 UTC m=+2057.336628773"
Jul 06 14:53:55.470907 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:53:55.470868    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-5c757b-openshift-default-7d9c6fc49-rqn6k"
Jul 06 14:53:55.475758 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:53:55.475731    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-5c757b-openshift-default-7d9c6fc49-rqn6k"
Jul 06 14:53:56.008322 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:53:56.008287    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-5c757b-openshift-default-7d9c6fc49-rqn6k"
Jul 06 14:53:56.009196 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:53:56.009174    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-5c757b-openshift-default-7d9c6fc49-rqn6k"
Jul 06 14:53:57.220480 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:53:57.220447    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-api-a-5c757b-677c4fbf7b-4sffr"]
Jul 06 14:53:57.220879 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:53:57.220866    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="504db21a-2c51-47ec-94d4-c3921ba0831e" containerName="istio-proxy"
Jul 06 14:53:57.220879 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:53:57.220878    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="504db21a-2c51-47ec-94d4-c3921ba0831e" containerName="istio-proxy"
Jul 06 14:53:57.223848 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:53:57.223830    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-5c757b-677c4fbf7b-4sffr"
Jul 06 14:53:57.226399 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:53:57.226377    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-api-a-5c757b-serving-cert\""
Jul 06 14:53:57.232865 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:53:57.232842    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-a-5c757b-677c4fbf7b-4sffr"]
Jul 06 14:53:57.248462 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:53:57.248429    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/32e5d6bd-559d-407e-b3c2-a1f6c9fe7bf8-openshift-service-ca\") pod \"maas-api-e2e-api-a-5c757b-677c4fbf7b-4sffr\" (UID: \"32e5d6bd-559d-407e-b3c2-a1f6c9fe7bf8\") " pod="opendatahub/maas-api-e2e-api-a-5c757b-677c4fbf7b-4sffr"
Jul 06 14:53:57.248659 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:53:57.248472    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/32e5d6bd-559d-407e-b3c2-a1f6c9fe7bf8-maas-api-tls\") pod \"maas-api-e2e-api-a-5c757b-677c4fbf7b-4sffr\" (UID: \"32e5d6bd-559d-407e-b3c2-a1f6c9fe7bf8\") " pod="opendatahub/maas-api-e2e-api-a-5c757b-677c4fbf7b-4sffr"
Jul 06 14:53:57.248659 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:53:57.248560    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r9cmh\" (UniqueName: \"kubernetes.io/projected/32e5d6bd-559d-407e-b3c2-a1f6c9fe7bf8-kube-api-access-r9cmh\") pod \"maas-api-e2e-api-a-5c757b-677c4fbf7b-4sffr\" (UID: \"32e5d6bd-559d-407e-b3c2-a1f6c9fe7bf8\") " pod="opendatahub/maas-api-e2e-api-a-5c757b-677c4fbf7b-4sffr"
Jul 06 14:53:57.349423 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:53:57.349382    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r9cmh\" (UniqueName: \"kubernetes.io/projected/32e5d6bd-559d-407e-b3c2-a1f6c9fe7bf8-kube-api-access-r9cmh\") pod \"maas-api-e2e-api-a-5c757b-677c4fbf7b-4sffr\" (UID: \"32e5d6bd-559d-407e-b3c2-a1f6c9fe7bf8\") " pod="opendatahub/maas-api-e2e-api-a-5c757b-677c4fbf7b-4sffr"
Jul 06 14:53:57.349640 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:53:57.349502    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/32e5d6bd-559d-407e-b3c2-a1f6c9fe7bf8-openshift-service-ca\") pod \"maas-api-e2e-api-a-5c757b-677c4fbf7b-4sffr\" (UID: \"32e5d6bd-559d-407e-b3c2-a1f6c9fe7bf8\") " pod="opendatahub/maas-api-e2e-api-a-5c757b-677c4fbf7b-4sffr"
Jul 06 14:53:57.349640 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:53:57.349536    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/32e5d6bd-559d-407e-b3c2-a1f6c9fe7bf8-maas-api-tls\") pod \"maas-api-e2e-api-a-5c757b-677c4fbf7b-4sffr\" (UID: \"32e5d6bd-559d-407e-b3c2-a1f6c9fe7bf8\") " pod="opendatahub/maas-api-e2e-api-a-5c757b-677c4fbf7b-4sffr"
Jul 06 14:53:57.349726 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:53:57.349692    2559 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-api-a-5c757b-serving-cert: secret "maas-api-e2e-api-a-5c757b-serving-cert" not found
Jul 06 14:53:57.349770 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:53:57.349759    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/32e5d6bd-559d-407e-b3c2-a1f6c9fe7bf8-maas-api-tls podName:32e5d6bd-559d-407e-b3c2-a1f6c9fe7bf8 nodeName:}" failed. No retries permitted until 2026-07-06 14:53:57.8497395 +0000 UTC m=+2060.157031561 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/32e5d6bd-559d-407e-b3c2-a1f6c9fe7bf8-maas-api-tls") pod "maas-api-e2e-api-a-5c757b-677c4fbf7b-4sffr" (UID: "32e5d6bd-559d-407e-b3c2-a1f6c9fe7bf8") : secret "maas-api-e2e-api-a-5c757b-serving-cert" not found
Jul 06 14:53:57.350174 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:53:57.350155    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/32e5d6bd-559d-407e-b3c2-a1f6c9fe7bf8-openshift-service-ca\") pod \"maas-api-e2e-api-a-5c757b-677c4fbf7b-4sffr\" (UID: \"32e5d6bd-559d-407e-b3c2-a1f6c9fe7bf8\") " pod="opendatahub/maas-api-e2e-api-a-5c757b-677c4fbf7b-4sffr"
Jul 06 14:53:57.362654 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:53:57.362624    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r9cmh\" (UniqueName: \"kubernetes.io/projected/32e5d6bd-559d-407e-b3c2-a1f6c9fe7bf8-kube-api-access-r9cmh\") pod \"maas-api-e2e-api-a-5c757b-677c4fbf7b-4sffr\" (UID: \"32e5d6bd-559d-407e-b3c2-a1f6c9fe7bf8\") " pod="opendatahub/maas-api-e2e-api-a-5c757b-677c4fbf7b-4sffr"
Jul 06 14:53:57.854151 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:53:57.854111    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/32e5d6bd-559d-407e-b3c2-a1f6c9fe7bf8-maas-api-tls\") pod \"maas-api-e2e-api-a-5c757b-677c4fbf7b-4sffr\" (UID: \"32e5d6bd-559d-407e-b3c2-a1f6c9fe7bf8\") " pod="opendatahub/maas-api-e2e-api-a-5c757b-677c4fbf7b-4sffr"
Jul 06 14:53:57.856599 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:53:57.856554    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/32e5d6bd-559d-407e-b3c2-a1f6c9fe7bf8-maas-api-tls\") pod \"maas-api-e2e-api-a-5c757b-677c4fbf7b-4sffr\" (UID: \"32e5d6bd-559d-407e-b3c2-a1f6c9fe7bf8\") " pod="opendatahub/maas-api-e2e-api-a-5c757b-677c4fbf7b-4sffr"
Jul 06 14:53:58.136374 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:53:58.136275    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-5c757b-677c4fbf7b-4sffr"
Jul 06 14:53:58.286766 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:53:58.286729    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-a-5c757b-677c4fbf7b-4sffr"]
Jul 06 14:53:58.288299 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:53:58.288271    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32e5d6bd_559d_407e_b3c2_a1f6c9fe7bf8.slice/crio-eafc0dfc3025cf63c4a542f6c8f8b1a71a7f3e5f6966fe44d801498523373b60 WatchSource:0}: Error finding container eafc0dfc3025cf63c4a542f6c8f8b1a71a7f3e5f6966fe44d801498523373b60: Status 404 returned error can't find the container with id eafc0dfc3025cf63c4a542f6c8f8b1a71a7f3e5f6966fe44d801498523373b60
Jul 06 14:53:59.022813 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:53:59.022779    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-5c757b-677c4fbf7b-4sffr" event={"ID":"32e5d6bd-559d-407e-b3c2-a1f6c9fe7bf8","Type":"ContainerStarted","Data":"de0087672bdabcb60b59058c08f79272970bba10fa65f43614d24b9190339940"}
Jul 06 14:53:59.022991 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:53:59.022836    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-5c757b-677c4fbf7b-4sffr" event={"ID":"32e5d6bd-559d-407e-b3c2-a1f6c9fe7bf8","Type":"ContainerStarted","Data":"eafc0dfc3025cf63c4a542f6c8f8b1a71a7f3e5f6966fe44d801498523373b60"}
Jul 06 14:53:59.022991 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:53:59.022864    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-api-a-5c757b-677c4fbf7b-4sffr"
Jul 06 14:53:59.041547 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:53:59.041498    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-api-a-5c757b-677c4fbf7b-4sffr" podStartSLOduration=1.407308045 podStartE2EDuration="2.041482875s" totalImagesPullingTime="634.17483ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:53:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:53:59.04098991 +0000 UTC m=+2061.348281995" watchObservedRunningTime="2026-07-06 14:53:59.041482875 +0000 UTC m=+2061.348774957"
Jul 06 14:54:02.435146 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:02.435051    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-6c8b7d-openshift-default-746555f497-8mn74"]
Jul 06 14:54:02.439048 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:02.439024    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-6c8b7d-openshift-default-746555f497-8mn74"
Jul 06 14:54:02.447788 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:02.447764    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-6c8b7d-openshift-default-dockercfg-njx4m\""
Jul 06 14:54:02.464437 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:02.464405    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-6c8b7d-openshift-default-746555f497-8mn74"]
Jul 06 14:54:02.507786 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:02.507495    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ede5beb-23e8-4a16-8a23-ca419a6a83d9-credential-socket\") pod \"e2e-api-b-6c8b7d-openshift-default-746555f497-8mn74\" (UID: \"3ede5beb-23e8-4a16-8a23-ca419a6a83d9\") " pod="openshift-ingress/e2e-api-b-6c8b7d-openshift-default-746555f497-8mn74"
Jul 06 14:54:02.508005 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:02.507872    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3ede5beb-23e8-4a16-8a23-ca419a6a83d9-workload-certs\") pod \"e2e-api-b-6c8b7d-openshift-default-746555f497-8mn74\" (UID: \"3ede5beb-23e8-4a16-8a23-ca419a6a83d9\") " pod="openshift-ingress/e2e-api-b-6c8b7d-openshift-default-746555f497-8mn74"
Jul 06 14:54:02.508005 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:02.507930    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-br2kc\" (UniqueName: \"kubernetes.io/projected/3ede5beb-23e8-4a16-8a23-ca419a6a83d9-kube-api-access-br2kc\") pod \"e2e-api-b-6c8b7d-openshift-default-746555f497-8mn74\" (UID: \"3ede5beb-23e8-4a16-8a23-ca419a6a83d9\") " pod="openshift-ingress/e2e-api-b-6c8b7d-openshift-default-746555f497-8mn74"
Jul 06 14:54:02.508005 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:02.507959    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3ede5beb-23e8-4a16-8a23-ca419a6a83d9-istio-podinfo\") pod \"e2e-api-b-6c8b7d-openshift-default-746555f497-8mn74\" (UID: \"3ede5beb-23e8-4a16-8a23-ca419a6a83d9\") " pod="openshift-ingress/e2e-api-b-6c8b7d-openshift-default-746555f497-8mn74"
Jul 06 14:54:02.508198 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:02.508021    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3ede5beb-23e8-4a16-8a23-ca419a6a83d9-istio-token\") pod \"e2e-api-b-6c8b7d-openshift-default-746555f497-8mn74\" (UID: \"3ede5beb-23e8-4a16-8a23-ca419a6a83d9\") " pod="openshift-ingress/e2e-api-b-6c8b7d-openshift-default-746555f497-8mn74"
Jul 06 14:54:02.508198 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:02.508053    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3ede5beb-23e8-4a16-8a23-ca419a6a83d9-istio-data\") pod \"e2e-api-b-6c8b7d-openshift-default-746555f497-8mn74\" (UID: \"3ede5beb-23e8-4a16-8a23-ca419a6a83d9\") " pod="openshift-ingress/e2e-api-b-6c8b7d-openshift-default-746555f497-8mn74"
Jul 06 14:54:02.508198 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:02.508088    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ede5beb-23e8-4a16-8a23-ca419a6a83d9-workload-socket\") pod \"e2e-api-b-6c8b7d-openshift-default-746555f497-8mn74\" (UID: \"3ede5beb-23e8-4a16-8a23-ca419a6a83d9\") " pod="openshift-ingress/e2e-api-b-6c8b7d-openshift-default-746555f497-8mn74"
Jul 06 14:54:02.508198 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:02.508109    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3ede5beb-23e8-4a16-8a23-ca419a6a83d9-istio-envoy\") pod \"e2e-api-b-6c8b7d-openshift-default-746555f497-8mn74\" (UID: \"3ede5beb-23e8-4a16-8a23-ca419a6a83d9\") " pod="openshift-ingress/e2e-api-b-6c8b7d-openshift-default-746555f497-8mn74"
Jul 06 14:54:02.508198 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:02.508139    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3ede5beb-23e8-4a16-8a23-ca419a6a83d9-istiod-ca-cert\") pod \"e2e-api-b-6c8b7d-openshift-default-746555f497-8mn74\" (UID: \"3ede5beb-23e8-4a16-8a23-ca419a6a83d9\") " pod="openshift-ingress/e2e-api-b-6c8b7d-openshift-default-746555f497-8mn74"
Jul 06 14:54:02.609391 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:02.609349    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3ede5beb-23e8-4a16-8a23-ca419a6a83d9-istio-podinfo\") pod \"e2e-api-b-6c8b7d-openshift-default-746555f497-8mn74\" (UID: \"3ede5beb-23e8-4a16-8a23-ca419a6a83d9\") " pod="openshift-ingress/e2e-api-b-6c8b7d-openshift-default-746555f497-8mn74"
Jul 06 14:54:02.609605 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:02.609417    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3ede5beb-23e8-4a16-8a23-ca419a6a83d9-istio-token\") pod \"e2e-api-b-6c8b7d-openshift-default-746555f497-8mn74\" (UID: \"3ede5beb-23e8-4a16-8a23-ca419a6a83d9\") " pod="openshift-ingress/e2e-api-b-6c8b7d-openshift-default-746555f497-8mn74"
Jul 06 14:54:02.609605 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:02.609450    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3ede5beb-23e8-4a16-8a23-ca419a6a83d9-istio-data\") pod \"e2e-api-b-6c8b7d-openshift-default-746555f497-8mn74\" (UID: \"3ede5beb-23e8-4a16-8a23-ca419a6a83d9\") " pod="openshift-ingress/e2e-api-b-6c8b7d-openshift-default-746555f497-8mn74"
Jul 06 14:54:02.609605 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:02.609483    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ede5beb-23e8-4a16-8a23-ca419a6a83d9-workload-socket\") pod \"e2e-api-b-6c8b7d-openshift-default-746555f497-8mn74\" (UID: \"3ede5beb-23e8-4a16-8a23-ca419a6a83d9\") " pod="openshift-ingress/e2e-api-b-6c8b7d-openshift-default-746555f497-8mn74"
Jul 06 14:54:02.609605 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:02.609505    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3ede5beb-23e8-4a16-8a23-ca419a6a83d9-istio-envoy\") pod \"e2e-api-b-6c8b7d-openshift-default-746555f497-8mn74\" (UID: \"3ede5beb-23e8-4a16-8a23-ca419a6a83d9\") " pod="openshift-ingress/e2e-api-b-6c8b7d-openshift-default-746555f497-8mn74"
Jul 06 14:54:02.609605 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:02.609535    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3ede5beb-23e8-4a16-8a23-ca419a6a83d9-istiod-ca-cert\") pod \"e2e-api-b-6c8b7d-openshift-default-746555f497-8mn74\" (UID: \"3ede5beb-23e8-4a16-8a23-ca419a6a83d9\") " pod="openshift-ingress/e2e-api-b-6c8b7d-openshift-default-746555f497-8mn74"
Jul 06 14:54:02.609605 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:02.609598    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ede5beb-23e8-4a16-8a23-ca419a6a83d9-credential-socket\") pod \"e2e-api-b-6c8b7d-openshift-default-746555f497-8mn74\" (UID: \"3ede5beb-23e8-4a16-8a23-ca419a6a83d9\") " pod="openshift-ingress/e2e-api-b-6c8b7d-openshift-default-746555f497-8mn74"
Jul 06 14:54:02.609999 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:02.609685    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3ede5beb-23e8-4a16-8a23-ca419a6a83d9-workload-certs\") pod \"e2e-api-b-6c8b7d-openshift-default-746555f497-8mn74\" (UID: \"3ede5beb-23e8-4a16-8a23-ca419a6a83d9\") " pod="openshift-ingress/e2e-api-b-6c8b7d-openshift-default-746555f497-8mn74"
Jul 06 14:54:02.609999 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:02.609729    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-br2kc\" (UniqueName: \"kubernetes.io/projected/3ede5beb-23e8-4a16-8a23-ca419a6a83d9-kube-api-access-br2kc\") pod \"e2e-api-b-6c8b7d-openshift-default-746555f497-8mn74\" (UID: \"3ede5beb-23e8-4a16-8a23-ca419a6a83d9\") " pod="openshift-ingress/e2e-api-b-6c8b7d-openshift-default-746555f497-8mn74"
Jul 06 14:54:02.610119 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:02.610077    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ede5beb-23e8-4a16-8a23-ca419a6a83d9-workload-socket\") pod \"e2e-api-b-6c8b7d-openshift-default-746555f497-8mn74\" (UID: \"3ede5beb-23e8-4a16-8a23-ca419a6a83d9\") " pod="openshift-ingress/e2e-api-b-6c8b7d-openshift-default-746555f497-8mn74"
Jul 06 14:54:02.610835 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:02.610285    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3ede5beb-23e8-4a16-8a23-ca419a6a83d9-istio-data\") pod \"e2e-api-b-6c8b7d-openshift-default-746555f497-8mn74\" (UID: \"3ede5beb-23e8-4a16-8a23-ca419a6a83d9\") " pod="openshift-ingress/e2e-api-b-6c8b7d-openshift-default-746555f497-8mn74"
Jul 06 14:54:02.610835 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:02.610317    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ede5beb-23e8-4a16-8a23-ca419a6a83d9-credential-socket\") pod \"e2e-api-b-6c8b7d-openshift-default-746555f497-8mn74\" (UID: \"3ede5beb-23e8-4a16-8a23-ca419a6a83d9\") " pod="openshift-ingress/e2e-api-b-6c8b7d-openshift-default-746555f497-8mn74"
Jul 06 14:54:02.610835 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:02.610538    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3ede5beb-23e8-4a16-8a23-ca419a6a83d9-workload-certs\") pod \"e2e-api-b-6c8b7d-openshift-default-746555f497-8mn74\" (UID: \"3ede5beb-23e8-4a16-8a23-ca419a6a83d9\") " pod="openshift-ingress/e2e-api-b-6c8b7d-openshift-default-746555f497-8mn74"
Jul 06 14:54:02.610835 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:02.610557    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3ede5beb-23e8-4a16-8a23-ca419a6a83d9-istiod-ca-cert\") pod \"e2e-api-b-6c8b7d-openshift-default-746555f497-8mn74\" (UID: \"3ede5beb-23e8-4a16-8a23-ca419a6a83d9\") " pod="openshift-ingress/e2e-api-b-6c8b7d-openshift-default-746555f497-8mn74"
Jul 06 14:54:02.613060 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:02.612628    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3ede5beb-23e8-4a16-8a23-ca419a6a83d9-istio-envoy\") pod \"e2e-api-b-6c8b7d-openshift-default-746555f497-8mn74\" (UID: \"3ede5beb-23e8-4a16-8a23-ca419a6a83d9\") " pod="openshift-ingress/e2e-api-b-6c8b7d-openshift-default-746555f497-8mn74"
Jul 06 14:54:02.613060 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:02.612946    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3ede5beb-23e8-4a16-8a23-ca419a6a83d9-istio-podinfo\") pod \"e2e-api-b-6c8b7d-openshift-default-746555f497-8mn74\" (UID: \"3ede5beb-23e8-4a16-8a23-ca419a6a83d9\") " pod="openshift-ingress/e2e-api-b-6c8b7d-openshift-default-746555f497-8mn74"
Jul 06 14:54:02.620711 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:02.620476    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3ede5beb-23e8-4a16-8a23-ca419a6a83d9-istio-token\") pod \"e2e-api-b-6c8b7d-openshift-default-746555f497-8mn74\" (UID: \"3ede5beb-23e8-4a16-8a23-ca419a6a83d9\") " pod="openshift-ingress/e2e-api-b-6c8b7d-openshift-default-746555f497-8mn74"
Jul 06 14:54:02.620711 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:02.620691    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-br2kc\" (UniqueName: \"kubernetes.io/projected/3ede5beb-23e8-4a16-8a23-ca419a6a83d9-kube-api-access-br2kc\") pod \"e2e-api-b-6c8b7d-openshift-default-746555f497-8mn74\" (UID: \"3ede5beb-23e8-4a16-8a23-ca419a6a83d9\") " pod="openshift-ingress/e2e-api-b-6c8b7d-openshift-default-746555f497-8mn74"
Jul 06 14:54:02.751880 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:02.751782    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-6c8b7d-openshift-default-746555f497-8mn74"
Jul 06 14:54:02.914417 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:02.914372    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-6c8b7d-openshift-default-746555f497-8mn74"]
Jul 06 14:54:02.915354 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:54:02.915321    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ede5beb_23e8_4a16_8a23_ca419a6a83d9.slice/crio-fd251e94ce5e3fbf5eec6456a8a1af3c27f5f2bdafd95ef0d47c01b6ad62edd9 WatchSource:0}: Error finding container fd251e94ce5e3fbf5eec6456a8a1af3c27f5f2bdafd95ef0d47c01b6ad62edd9: Status 404 returned error can't find the container with id fd251e94ce5e3fbf5eec6456a8a1af3c27f5f2bdafd95ef0d47c01b6ad62edd9
Jul 06 14:54:02.917757 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:02.917708    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 06 14:54:02.917844 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:02.917819    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 06 14:54:02.917888 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:02.917872    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 06 14:54:03.045462 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:03.045292    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-6c8b7d-openshift-default-746555f497-8mn74" event={"ID":"3ede5beb-23e8-4a16-8a23-ca419a6a83d9","Type":"ContainerStarted","Data":"bc3827a083e3560045679a4bfe36d15228bc29d1c99d163a3878d809e1ceb074"}
Jul 06 14:54:03.045462 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:03.045364    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-6c8b7d-openshift-default-746555f497-8mn74" event={"ID":"3ede5beb-23e8-4a16-8a23-ca419a6a83d9","Type":"ContainerStarted","Data":"fd251e94ce5e3fbf5eec6456a8a1af3c27f5f2bdafd95ef0d47c01b6ad62edd9"}
Jul 06 14:54:03.751922 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:03.751885    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-6c8b7d-openshift-default-746555f497-8mn74"
Jul 06 14:54:03.756948 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:03.756919    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-6c8b7d-openshift-default-746555f497-8mn74"
Jul 06 14:54:03.817212 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:03.816984    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-6c8b7d-openshift-default-746555f497-8mn74" podStartSLOduration=1.816963273 podStartE2EDuration="1.816963273s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:54:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:54:03.092401893 +0000 UTC m=+2065.399693976" watchObservedRunningTime="2026-07-06 14:54:03.816963273 +0000 UTC m=+2066.124255359"
Jul 06 14:54:04.050779 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:04.050681    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-6c8b7d-openshift-default-746555f497-8mn74"
Jul 06 14:54:04.051874 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:04.051855    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-6c8b7d-openshift-default-746555f497-8mn74"
Jul 06 14:54:05.033931 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:05.033901    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-api-a-5c757b-677c4fbf7b-4sffr"
Jul 06 14:54:05.373960 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:05.373862    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-api-b-6c8b7d-7d98b97fb8-75tm7"]
Jul 06 14:54:05.378085 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:05.378060    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-6c8b7d-7d98b97fb8-75tm7"
Jul 06 14:54:05.385642 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:05.383165    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-api-b-6c8b7d-serving-cert\""
Jul 06 14:54:05.395604 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:05.389808    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-b-6c8b7d-7d98b97fb8-75tm7"]
Jul 06 14:54:05.439220 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:05.439170    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8f5fc6d8-3a9d-4786-adff-4f13542d8d97-openshift-service-ca\") pod \"maas-api-e2e-api-b-6c8b7d-7d98b97fb8-75tm7\" (UID: \"8f5fc6d8-3a9d-4786-adff-4f13542d8d97\") " pod="opendatahub/maas-api-e2e-api-b-6c8b7d-7d98b97fb8-75tm7"
Jul 06 14:54:05.439220 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:05.439214    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ndwp2\" (UniqueName: \"kubernetes.io/projected/8f5fc6d8-3a9d-4786-adff-4f13542d8d97-kube-api-access-ndwp2\") pod \"maas-api-e2e-api-b-6c8b7d-7d98b97fb8-75tm7\" (UID: \"8f5fc6d8-3a9d-4786-adff-4f13542d8d97\") " pod="opendatahub/maas-api-e2e-api-b-6c8b7d-7d98b97fb8-75tm7"
Jul 06 14:54:05.439507 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:05.439255    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8f5fc6d8-3a9d-4786-adff-4f13542d8d97-maas-api-tls\") pod \"maas-api-e2e-api-b-6c8b7d-7d98b97fb8-75tm7\" (UID: \"8f5fc6d8-3a9d-4786-adff-4f13542d8d97\") " pod="opendatahub/maas-api-e2e-api-b-6c8b7d-7d98b97fb8-75tm7"
Jul 06 14:54:05.539832 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:05.539796    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8f5fc6d8-3a9d-4786-adff-4f13542d8d97-openshift-service-ca\") pod \"maas-api-e2e-api-b-6c8b7d-7d98b97fb8-75tm7\" (UID: \"8f5fc6d8-3a9d-4786-adff-4f13542d8d97\") " pod="opendatahub/maas-api-e2e-api-b-6c8b7d-7d98b97fb8-75tm7"
Jul 06 14:54:05.540021 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:05.539839    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ndwp2\" (UniqueName: \"kubernetes.io/projected/8f5fc6d8-3a9d-4786-adff-4f13542d8d97-kube-api-access-ndwp2\") pod \"maas-api-e2e-api-b-6c8b7d-7d98b97fb8-75tm7\" (UID: \"8f5fc6d8-3a9d-4786-adff-4f13542d8d97\") " pod="opendatahub/maas-api-e2e-api-b-6c8b7d-7d98b97fb8-75tm7"
Jul 06 14:54:05.540021 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:05.539884    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8f5fc6d8-3a9d-4786-adff-4f13542d8d97-maas-api-tls\") pod \"maas-api-e2e-api-b-6c8b7d-7d98b97fb8-75tm7\" (UID: \"8f5fc6d8-3a9d-4786-adff-4f13542d8d97\") " pod="opendatahub/maas-api-e2e-api-b-6c8b7d-7d98b97fb8-75tm7"
Jul 06 14:54:05.540021 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:54:05.540010    2559 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-api-b-6c8b7d-serving-cert: secret "maas-api-e2e-api-b-6c8b7d-serving-cert" not found
Jul 06 14:54:05.540147 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:54:05.540081    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8f5fc6d8-3a9d-4786-adff-4f13542d8d97-maas-api-tls podName:8f5fc6d8-3a9d-4786-adff-4f13542d8d97 nodeName:}" failed. No retries permitted until 2026-07-06 14:54:06.040060578 +0000 UTC m=+2068.347352640 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/8f5fc6d8-3a9d-4786-adff-4f13542d8d97-maas-api-tls") pod "maas-api-e2e-api-b-6c8b7d-7d98b97fb8-75tm7" (UID: "8f5fc6d8-3a9d-4786-adff-4f13542d8d97") : secret "maas-api-e2e-api-b-6c8b7d-serving-cert" not found
Jul 06 14:54:05.540543 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:05.540523    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8f5fc6d8-3a9d-4786-adff-4f13542d8d97-openshift-service-ca\") pod \"maas-api-e2e-api-b-6c8b7d-7d98b97fb8-75tm7\" (UID: \"8f5fc6d8-3a9d-4786-adff-4f13542d8d97\") " pod="opendatahub/maas-api-e2e-api-b-6c8b7d-7d98b97fb8-75tm7"
Jul 06 14:54:05.553210 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:05.553177    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ndwp2\" (UniqueName: \"kubernetes.io/projected/8f5fc6d8-3a9d-4786-adff-4f13542d8d97-kube-api-access-ndwp2\") pod \"maas-api-e2e-api-b-6c8b7d-7d98b97fb8-75tm7\" (UID: \"8f5fc6d8-3a9d-4786-adff-4f13542d8d97\") " pod="opendatahub/maas-api-e2e-api-b-6c8b7d-7d98b97fb8-75tm7"
Jul 06 14:54:06.046429 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:06.046381    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8f5fc6d8-3a9d-4786-adff-4f13542d8d97-maas-api-tls\") pod \"maas-api-e2e-api-b-6c8b7d-7d98b97fb8-75tm7\" (UID: \"8f5fc6d8-3a9d-4786-adff-4f13542d8d97\") " pod="opendatahub/maas-api-e2e-api-b-6c8b7d-7d98b97fb8-75tm7"
Jul 06 14:54:06.048914 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:06.048882    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8f5fc6d8-3a9d-4786-adff-4f13542d8d97-maas-api-tls\") pod \"maas-api-e2e-api-b-6c8b7d-7d98b97fb8-75tm7\" (UID: \"8f5fc6d8-3a9d-4786-adff-4f13542d8d97\") " pod="opendatahub/maas-api-e2e-api-b-6c8b7d-7d98b97fb8-75tm7"
Jul 06 14:54:06.313541 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:06.313438    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-6c8b7d-7d98b97fb8-75tm7"
Jul 06 14:54:06.505639 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:06.504639    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-b-6c8b7d-7d98b97fb8-75tm7"]
Jul 06 14:54:07.067630 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:07.067593    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-6c8b7d-7d98b97fb8-75tm7" event={"ID":"8f5fc6d8-3a9d-4786-adff-4f13542d8d97","Type":"ContainerStarted","Data":"bebdba22a4a8d95aa311844d1a6466daaadc3e3d68a9a71829b133371602059b"}
Jul 06 14:54:07.068149 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:07.067656    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-6c8b7d-7d98b97fb8-75tm7" event={"ID":"8f5fc6d8-3a9d-4786-adff-4f13542d8d97","Type":"ContainerStarted","Data":"fbfebc98b702fd8f9bd14be41a9816482953753a1ebc63f2ff51bbf9385f1a92"}
Jul 06 14:54:07.068149 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:07.067708    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-api-b-6c8b7d-7d98b97fb8-75tm7"
Jul 06 14:54:07.088309 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:07.088234    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-api-b-6c8b7d-7d98b97fb8-75tm7" podStartSLOduration=1.675532411 podStartE2EDuration="2.08821246s" totalImagesPullingTime="412.680049ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:54:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:54:07.086619777 +0000 UTC m=+2069.393911856" watchObservedRunningTime="2026-07-06 14:54:07.08821246 +0000 UTC m=+2069.395504544"
Jul 06 14:54:11.010410 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:11.010374    2559 patch_prober.go:28] interesting pod/e2e-api-a-5c757b-openshift-default-7d9c6fc49-rqn6k container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body=
Jul 06 14:54:11.010881 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:11.010433    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-5c757b-openshift-default-7d9c6fc49-rqn6k" podUID="d7c70639-a47b-4adf-8a6b-5ff2cbbe458c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused"
Jul 06 14:54:11.093898 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:11.093862    2559 generic.go:356] "Generic (PLEG): container finished" podID="d7c70639-a47b-4adf-8a6b-5ff2cbbe458c" containerID="3e04734f648241506ba9e6eb9f78ba73b34eb9db0b85271ea758a1708da18659" exitCode=0
Jul 06 14:54:11.094085 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:11.093938    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-5c757b-openshift-default-7d9c6fc49-rqn6k" event={"ID":"d7c70639-a47b-4adf-8a6b-5ff2cbbe458c","Type":"ContainerDied","Data":"3e04734f648241506ba9e6eb9f78ba73b34eb9db0b85271ea758a1708da18659"}
Jul 06 14:54:11.094376 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:11.094356    2559 scope.go:117] "RemoveContainer" containerID="3e04734f648241506ba9e6eb9f78ba73b34eb9db0b85271ea758a1708da18659"
Jul 06 14:54:11.095400 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:11.095367    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 06 14:54:11.095469 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:11.095436    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 06 14:54:11.095514 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:11.095468    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 06 14:54:12.105101 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:12.105050    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-5c757b-openshift-default-7d9c6fc49-rqn6k" event={"ID":"d7c70639-a47b-4adf-8a6b-5ff2cbbe458c","Type":"ContainerStarted","Data":"3ab4885c4172bf9498b7e25b32523dd37a7fc79d5317dfdea2be9ecfc54941c3"}
Jul 06 14:54:12.470862 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:12.470832    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-5c757b-openshift-default-7d9c6fc49-rqn6k"
Jul 06 14:54:12.471114 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:12.471091    2559 patch_prober.go:28] interesting pod/e2e-api-a-5c757b-openshift-default-7d9c6fc49-rqn6k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body=
Jul 06 14:54:12.471164 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:12.471150    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-5c757b-openshift-default-7d9c6fc49-rqn6k" podUID="d7c70639-a47b-4adf-8a6b-5ff2cbbe458c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused"
Jul 06 14:54:13.078487 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:13.078454    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-api-b-6c8b7d-7d98b97fb8-75tm7"
Jul 06 14:54:13.471383 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:13.471340    2559 patch_prober.go:28] interesting pod/e2e-api-a-5c757b-openshift-default-7d9c6fc49-rqn6k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body=
Jul 06 14:54:13.471924 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:13.471412    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-5c757b-openshift-default-7d9c6fc49-rqn6k" podUID="d7c70639-a47b-4adf-8a6b-5ff2cbbe458c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused"
Jul 06 14:54:14.471511 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:14.471475    2559 patch_prober.go:28] interesting pod/e2e-api-a-5c757b-openshift-default-7d9c6fc49-rqn6k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body=
Jul 06 14:54:14.471952 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:14.471541    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-5c757b-openshift-default-7d9c6fc49-rqn6k" podUID="d7c70639-a47b-4adf-8a6b-5ff2cbbe458c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused"
Jul 06 14:54:15.471176 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:15.471137    2559 patch_prober.go:28] interesting pod/e2e-api-a-5c757b-openshift-default-7d9c6fc49-rqn6k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body=
Jul 06 14:54:15.471387 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:15.471196    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-5c757b-openshift-default-7d9c6fc49-rqn6k" podUID="d7c70639-a47b-4adf-8a6b-5ff2cbbe458c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused"
Jul 06 14:54:16.475523 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:16.475490    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-5c757b-openshift-default-7d9c6fc49-rqn6k"
Jul 06 14:54:16.475981 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:16.475816    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-5c757b-openshift-default-7d9c6fc49-rqn6k"
Jul 06 14:54:16.476469 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:16.476450    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-5c757b-openshift-default-7d9c6fc49-rqn6k"
Jul 06 14:54:18.712912 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:18.712870    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-a-5c757b-677c4fbf7b-4sffr"]
Jul 06 14:54:18.713390 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:18.713143    2559 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-api-a-5c757b-677c4fbf7b-4sffr" podUID="32e5d6bd-559d-407e-b3c2-a1f6c9fe7bf8" containerName="maas-api" containerID="cri-o://de0087672bdabcb60b59058c08f79272970bba10fa65f43614d24b9190339940" gracePeriod=30
Jul 06 14:54:19.003971 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:19.003943    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-5c757b-677c4fbf7b-4sffr"
Jul 06 14:54:19.079243 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:19.079208    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r9cmh\" (UniqueName: \"kubernetes.io/projected/32e5d6bd-559d-407e-b3c2-a1f6c9fe7bf8-kube-api-access-r9cmh\") pod \"32e5d6bd-559d-407e-b3c2-a1f6c9fe7bf8\" (UID: \"32e5d6bd-559d-407e-b3c2-a1f6c9fe7bf8\") "
Jul 06 14:54:19.079472 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:19.079278    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/32e5d6bd-559d-407e-b3c2-a1f6c9fe7bf8-maas-api-tls\") pod \"32e5d6bd-559d-407e-b3c2-a1f6c9fe7bf8\" (UID: \"32e5d6bd-559d-407e-b3c2-a1f6c9fe7bf8\") "
Jul 06 14:54:19.079472 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:19.079337    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/32e5d6bd-559d-407e-b3c2-a1f6c9fe7bf8-openshift-service-ca\") pod \"32e5d6bd-559d-407e-b3c2-a1f6c9fe7bf8\" (UID: \"32e5d6bd-559d-407e-b3c2-a1f6c9fe7bf8\") "
Jul 06 14:54:19.080048 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:19.080023    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/32e5d6bd-559d-407e-b3c2-a1f6c9fe7bf8-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "32e5d6bd-559d-407e-b3c2-a1f6c9fe7bf8" (UID: "32e5d6bd-559d-407e-b3c2-a1f6c9fe7bf8"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 14:54:19.082090 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:19.082055    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/32e5d6bd-559d-407e-b3c2-a1f6c9fe7bf8-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "32e5d6bd-559d-407e-b3c2-a1f6c9fe7bf8" (UID: "32e5d6bd-559d-407e-b3c2-a1f6c9fe7bf8"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 14:54:19.082722 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:19.082695    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/32e5d6bd-559d-407e-b3c2-a1f6c9fe7bf8-kube-api-access-r9cmh" (OuterVolumeSpecName: "kube-api-access-r9cmh") pod "32e5d6bd-559d-407e-b3c2-a1f6c9fe7bf8" (UID: "32e5d6bd-559d-407e-b3c2-a1f6c9fe7bf8"). InnerVolumeSpecName "kube-api-access-r9cmh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 14:54:19.150560 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:19.150521    2559 generic.go:356] "Generic (PLEG): container finished" podID="32e5d6bd-559d-407e-b3c2-a1f6c9fe7bf8" containerID="de0087672bdabcb60b59058c08f79272970bba10fa65f43614d24b9190339940" exitCode=0
Jul 06 14:54:19.150770 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:19.150592    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-5c757b-677c4fbf7b-4sffr"
Jul 06 14:54:19.150770 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:19.150625    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-5c757b-677c4fbf7b-4sffr" event={"ID":"32e5d6bd-559d-407e-b3c2-a1f6c9fe7bf8","Type":"ContainerDied","Data":"de0087672bdabcb60b59058c08f79272970bba10fa65f43614d24b9190339940"}
Jul 06 14:54:19.150770 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:19.150679    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-5c757b-677c4fbf7b-4sffr" event={"ID":"32e5d6bd-559d-407e-b3c2-a1f6c9fe7bf8","Type":"ContainerDied","Data":"eafc0dfc3025cf63c4a542f6c8f8b1a71a7f3e5f6966fe44d801498523373b60"}
Jul 06 14:54:19.150770 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:19.150705    2559 scope.go:117] "RemoveContainer" containerID="de0087672bdabcb60b59058c08f79272970bba10fa65f43614d24b9190339940"
Jul 06 14:54:19.160963 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:19.160942    2559 scope.go:117] "RemoveContainer" containerID="de0087672bdabcb60b59058c08f79272970bba10fa65f43614d24b9190339940"
Jul 06 14:54:19.161312 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:54:19.161292    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"de0087672bdabcb60b59058c08f79272970bba10fa65f43614d24b9190339940\": container with ID starting with de0087672bdabcb60b59058c08f79272970bba10fa65f43614d24b9190339940 not found: ID does not exist" containerID="de0087672bdabcb60b59058c08f79272970bba10fa65f43614d24b9190339940"
Jul 06 14:54:19.161383 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:19.161330    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"de0087672bdabcb60b59058c08f79272970bba10fa65f43614d24b9190339940"} err="failed to get container status \"de0087672bdabcb60b59058c08f79272970bba10fa65f43614d24b9190339940\": rpc error: code = NotFound desc = could not find container \"de0087672bdabcb60b59058c08f79272970bba10fa65f43614d24b9190339940\": container with ID starting with de0087672bdabcb60b59058c08f79272970bba10fa65f43614d24b9190339940 not found: ID does not exist"
Jul 06 14:54:19.176978 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:19.176944    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-a-5c757b-677c4fbf7b-4sffr"]
Jul 06 14:54:19.181141 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:19.181106    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r9cmh\" (UniqueName: \"kubernetes.io/projected/32e5d6bd-559d-407e-b3c2-a1f6c9fe7bf8-kube-api-access-r9cmh\") on node \"ip-10-0-133-112.ec2.internal\" DevicePath \"\""
Jul 06 14:54:19.181141 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:19.181137    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/32e5d6bd-559d-407e-b3c2-a1f6c9fe7bf8-maas-api-tls\") on node \"ip-10-0-133-112.ec2.internal\" DevicePath \"\""
Jul 06 14:54:19.181141 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:19.181148    2559 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/32e5d6bd-559d-407e-b3c2-a1f6c9fe7bf8-openshift-service-ca\") on node \"ip-10-0-133-112.ec2.internal\" DevicePath \"\""
Jul 06 14:54:19.183595 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:19.183550    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-api-a-5c757b-677c4fbf7b-4sffr"]
Jul 06 14:54:20.092793 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:20.092743    2559 patch_prober.go:28] interesting pod/e2e-api-b-6c8b7d-openshift-default-746555f497-8mn74 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 06 14:54:20.093172 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:20.092823    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-6c8b7d-openshift-default-746555f497-8mn74" podUID="3ede5beb-23e8-4a16-8a23-ca419a6a83d9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 06 14:54:20.228180 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:20.228146    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="32e5d6bd-559d-407e-b3c2-a1f6c9fe7bf8" path="/var/lib/kubelet/pods/32e5d6bd-559d-407e-b3c2-a1f6c9fe7bf8/volumes"
Jul 06 14:54:21.162921 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:21.162889    2559 generic.go:356] "Generic (PLEG): container finished" podID="3ede5beb-23e8-4a16-8a23-ca419a6a83d9" containerID="bc3827a083e3560045679a4bfe36d15228bc29d1c99d163a3878d809e1ceb074" exitCode=0
Jul 06 14:54:21.163362 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:21.162968    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-6c8b7d-openshift-default-746555f497-8mn74" event={"ID":"3ede5beb-23e8-4a16-8a23-ca419a6a83d9","Type":"ContainerDied","Data":"bc3827a083e3560045679a4bfe36d15228bc29d1c99d163a3878d809e1ceb074"}
Jul 06 14:54:21.163434 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:21.163370    2559 scope.go:117] "RemoveContainer" containerID="bc3827a083e3560045679a4bfe36d15228bc29d1c99d163a3878d809e1ceb074"
Jul 06 14:54:21.164408 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:21.164379    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 06 14:54:21.164470 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:21.164450    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 06 14:54:21.164507 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:21.164482    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 06 14:54:22.171128 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:22.171093    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-6c8b7d-openshift-default-746555f497-8mn74" event={"ID":"3ede5beb-23e8-4a16-8a23-ca419a6a83d9","Type":"ContainerStarted","Data":"20579e81aca2aae35e1f661bbd2bafae1dd5003eb96575bcc865cfbe44c3a74b"}
Jul 06 14:54:22.752837 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:22.752796    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-6c8b7d-openshift-default-746555f497-8mn74"
Jul 06 14:54:22.753113 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:22.753087    2559 patch_prober.go:28] interesting pod/e2e-api-b-6c8b7d-openshift-default-746555f497-8mn74 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body=
Jul 06 14:54:22.753206 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:22.753151    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-6c8b7d-openshift-default-746555f497-8mn74" podUID="3ede5beb-23e8-4a16-8a23-ca419a6a83d9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused"
Jul 06 14:54:23.752340 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:23.752299    2559 patch_prober.go:28] interesting pod/e2e-api-b-6c8b7d-openshift-default-746555f497-8mn74 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body=
Jul 06 14:54:23.752860 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:23.752359    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-6c8b7d-openshift-default-746555f497-8mn74" podUID="3ede5beb-23e8-4a16-8a23-ca419a6a83d9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused"
Jul 06 14:54:24.753022 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:24.752984    2559 patch_prober.go:28] interesting pod/e2e-api-b-6c8b7d-openshift-default-746555f497-8mn74 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body=
Jul 06 14:54:24.753430 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:24.753042    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-6c8b7d-openshift-default-746555f497-8mn74" podUID="3ede5beb-23e8-4a16-8a23-ca419a6a83d9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused"
Jul 06 14:54:25.056388 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:25.056296    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-5c757b-openshift-default-7d9c6fc49-rqn6k"]
Jul 06 14:54:25.056546 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:25.056497    2559 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-5c757b-openshift-default-7d9c6fc49-rqn6k" podUID="d7c70639-a47b-4adf-8a6b-5ff2cbbe458c" containerName="istio-proxy" containerID="cri-o://3ab4885c4172bf9498b7e25b32523dd37a7fc79d5317dfdea2be9ecfc54941c3" gracePeriod=30
Jul 06 14:54:25.752440 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:25.752402    2559 patch_prober.go:28] interesting pod/e2e-api-b-6c8b7d-openshift-default-746555f497-8mn74 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body=
Jul 06 14:54:25.752697 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:25.752466    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-6c8b7d-openshift-default-746555f497-8mn74" podUID="3ede5beb-23e8-4a16-8a23-ca419a6a83d9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused"
Jul 06 14:54:26.231400 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:26.231370    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-b-6c8b7d-7d98b97fb8-75tm7"]
Jul 06 14:54:26.231927 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:26.231615    2559 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-api-b-6c8b7d-7d98b97fb8-75tm7" podUID="8f5fc6d8-3a9d-4786-adff-4f13542d8d97" containerName="maas-api" containerID="cri-o://bebdba22a4a8d95aa311844d1a6466daaadc3e3d68a9a71829b133371602059b" gracePeriod=30
Jul 06 14:54:26.506977 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:26.506942    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-6c8b7d-7d98b97fb8-75tm7"
Jul 06 14:54:26.657451 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:26.657412    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8f5fc6d8-3a9d-4786-adff-4f13542d8d97-maas-api-tls\") pod \"8f5fc6d8-3a9d-4786-adff-4f13542d8d97\" (UID: \"8f5fc6d8-3a9d-4786-adff-4f13542d8d97\") "
Jul 06 14:54:26.657679 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:26.657463    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ndwp2\" (UniqueName: \"kubernetes.io/projected/8f5fc6d8-3a9d-4786-adff-4f13542d8d97-kube-api-access-ndwp2\") pod \"8f5fc6d8-3a9d-4786-adff-4f13542d8d97\" (UID: \"8f5fc6d8-3a9d-4786-adff-4f13542d8d97\") "
Jul 06 14:54:26.657679 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:26.657634    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8f5fc6d8-3a9d-4786-adff-4f13542d8d97-openshift-service-ca\") pod \"8f5fc6d8-3a9d-4786-adff-4f13542d8d97\" (UID: \"8f5fc6d8-3a9d-4786-adff-4f13542d8d97\") "
Jul 06 14:54:26.658006 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:26.657972    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8f5fc6d8-3a9d-4786-adff-4f13542d8d97-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "8f5fc6d8-3a9d-4786-adff-4f13542d8d97" (UID: "8f5fc6d8-3a9d-4786-adff-4f13542d8d97"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 14:54:26.659540 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:26.659516    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8f5fc6d8-3a9d-4786-adff-4f13542d8d97-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "8f5fc6d8-3a9d-4786-adff-4f13542d8d97" (UID: "8f5fc6d8-3a9d-4786-adff-4f13542d8d97"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 14:54:26.659665 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:26.659621    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8f5fc6d8-3a9d-4786-adff-4f13542d8d97-kube-api-access-ndwp2" (OuterVolumeSpecName: "kube-api-access-ndwp2") pod "8f5fc6d8-3a9d-4786-adff-4f13542d8d97" (UID: "8f5fc6d8-3a9d-4786-adff-4f13542d8d97"). InnerVolumeSpecName "kube-api-access-ndwp2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 14:54:26.756273 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:26.756197    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-6c8b7d-openshift-default-746555f497-8mn74"
Jul 06 14:54:26.756473 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:26.756452    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-6c8b7d-openshift-default-746555f497-8mn74"
Jul 06 14:54:26.757228 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:26.757213    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-6c8b7d-openshift-default-746555f497-8mn74"
Jul 06 14:54:26.758606 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:26.758588    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8f5fc6d8-3a9d-4786-adff-4f13542d8d97-maas-api-tls\") on node \"ip-10-0-133-112.ec2.internal\" DevicePath \"\""
Jul 06 14:54:26.758658 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:26.758610    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ndwp2\" (UniqueName: \"kubernetes.io/projected/8f5fc6d8-3a9d-4786-adff-4f13542d8d97-kube-api-access-ndwp2\") on node \"ip-10-0-133-112.ec2.internal\" DevicePath \"\""
Jul 06 14:54:26.758658 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:26.758620    2559 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8f5fc6d8-3a9d-4786-adff-4f13542d8d97-openshift-service-ca\") on node \"ip-10-0-133-112.ec2.internal\" DevicePath \"\""
Jul 06 14:54:27.210510 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:27.210477    2559 generic.go:356] "Generic (PLEG): container finished" podID="8f5fc6d8-3a9d-4786-adff-4f13542d8d97" containerID="bebdba22a4a8d95aa311844d1a6466daaadc3e3d68a9a71829b133371602059b" exitCode=0
Jul 06 14:54:27.210715 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:27.210531    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-6c8b7d-7d98b97fb8-75tm7"
Jul 06 14:54:27.210715 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:27.210560    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-6c8b7d-7d98b97fb8-75tm7" event={"ID":"8f5fc6d8-3a9d-4786-adff-4f13542d8d97","Type":"ContainerDied","Data":"bebdba22a4a8d95aa311844d1a6466daaadc3e3d68a9a71829b133371602059b"}
Jul 06 14:54:27.210715 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:27.210666    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-6c8b7d-7d98b97fb8-75tm7" event={"ID":"8f5fc6d8-3a9d-4786-adff-4f13542d8d97","Type":"ContainerDied","Data":"fbfebc98b702fd8f9bd14be41a9816482953753a1ebc63f2ff51bbf9385f1a92"}
Jul 06 14:54:27.210715 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:27.210694    2559 scope.go:117] "RemoveContainer" containerID="bebdba22a4a8d95aa311844d1a6466daaadc3e3d68a9a71829b133371602059b"
Jul 06 14:54:27.220686 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:27.220663    2559 scope.go:117] "RemoveContainer" containerID="bebdba22a4a8d95aa311844d1a6466daaadc3e3d68a9a71829b133371602059b"
Jul 06 14:54:27.220986 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:54:27.220965    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bebdba22a4a8d95aa311844d1a6466daaadc3e3d68a9a71829b133371602059b\": container with ID starting with bebdba22a4a8d95aa311844d1a6466daaadc3e3d68a9a71829b133371602059b not found: ID does not exist" containerID="bebdba22a4a8d95aa311844d1a6466daaadc3e3d68a9a71829b133371602059b"
Jul 06 14:54:27.221067 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:27.220999    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bebdba22a4a8d95aa311844d1a6466daaadc3e3d68a9a71829b133371602059b"} err="failed to get container status \"bebdba22a4a8d95aa311844d1a6466daaadc3e3d68a9a71829b133371602059b\": rpc error: code = NotFound desc = could not find container \"bebdba22a4a8d95aa311844d1a6466daaadc3e3d68a9a71829b133371602059b\": container with ID starting with bebdba22a4a8d95aa311844d1a6466daaadc3e3d68a9a71829b133371602059b not found: ID does not exist"
Jul 06 14:54:27.233166 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:27.233138    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-b-6c8b7d-7d98b97fb8-75tm7"]
Jul 06 14:54:27.237711 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:27.237685    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-api-b-6c8b7d-7d98b97fb8-75tm7"]
Jul 06 14:54:28.227236 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:28.227202    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8f5fc6d8-3a9d-4786-adff-4f13542d8d97" path="/var/lib/kubelet/pods/8f5fc6d8-3a9d-4786-adff-4f13542d8d97/volumes"
Jul 06 14:54:30.230046 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:30.230008    2559 generic.go:356] "Generic (PLEG): container finished" podID="d7c70639-a47b-4adf-8a6b-5ff2cbbe458c" containerID="3ab4885c4172bf9498b7e25b32523dd37a7fc79d5317dfdea2be9ecfc54941c3" exitCode=0
Jul 06 14:54:30.230450 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:30.230153    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-5c757b-openshift-default-7d9c6fc49-rqn6k" event={"ID":"d7c70639-a47b-4adf-8a6b-5ff2cbbe458c","Type":"ContainerDied","Data":"3ab4885c4172bf9498b7e25b32523dd37a7fc79d5317dfdea2be9ecfc54941c3"}
Jul 06 14:54:30.230450 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:30.230209    2559 scope.go:117] "RemoveContainer" containerID="3e04734f648241506ba9e6eb9f78ba73b34eb9db0b85271ea758a1708da18659"
Jul 06 14:54:30.322828 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:30.322760    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-5c757b-openshift-default-7d9c6fc49-rqn6k"
Jul 06 14:54:30.493157 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:30.493104    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7c70639-a47b-4adf-8a6b-5ff2cbbe458c-credential-socket\") pod \"d7c70639-a47b-4adf-8a6b-5ff2cbbe458c\" (UID: \"d7c70639-a47b-4adf-8a6b-5ff2cbbe458c\") "
Jul 06 14:54:30.493157 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:30.493167    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d7c70639-a47b-4adf-8a6b-5ff2cbbe458c-istio-podinfo\") pod \"d7c70639-a47b-4adf-8a6b-5ff2cbbe458c\" (UID: \"d7c70639-a47b-4adf-8a6b-5ff2cbbe458c\") "
Jul 06 14:54:30.493427 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:30.493193    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d7c70639-a47b-4adf-8a6b-5ff2cbbe458c-istio-data\") pod \"d7c70639-a47b-4adf-8a6b-5ff2cbbe458c\" (UID: \"d7c70639-a47b-4adf-8a6b-5ff2cbbe458c\") "
Jul 06 14:54:30.493427 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:30.493222    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d7c70639-a47b-4adf-8a6b-5ff2cbbe458c-istiod-ca-cert\") pod \"d7c70639-a47b-4adf-8a6b-5ff2cbbe458c\" (UID: \"d7c70639-a47b-4adf-8a6b-5ff2cbbe458c\") "
Jul 06 14:54:30.493427 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:30.493259    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d7c70639-a47b-4adf-8a6b-5ff2cbbe458c-istio-envoy\") pod \"d7c70639-a47b-4adf-8a6b-5ff2cbbe458c\" (UID: \"d7c70639-a47b-4adf-8a6b-5ff2cbbe458c\") "
Jul 06 14:54:30.493427 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:30.493292    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d7c70639-a47b-4adf-8a6b-5ff2cbbe458c-istio-token\") pod \"d7c70639-a47b-4adf-8a6b-5ff2cbbe458c\" (UID: \"d7c70639-a47b-4adf-8a6b-5ff2cbbe458c\") "
Jul 06 14:54:30.493427 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:30.493317    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7c70639-a47b-4adf-8a6b-5ff2cbbe458c-workload-socket\") pod \"d7c70639-a47b-4adf-8a6b-5ff2cbbe458c\" (UID: \"d7c70639-a47b-4adf-8a6b-5ff2cbbe458c\") "
Jul 06 14:54:30.493427 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:30.493405    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d7c70639-a47b-4adf-8a6b-5ff2cbbe458c-workload-certs\") pod \"d7c70639-a47b-4adf-8a6b-5ff2cbbe458c\" (UID: \"d7c70639-a47b-4adf-8a6b-5ff2cbbe458c\") "
Jul 06 14:54:30.493774 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:30.493429    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dpd5d\" (UniqueName: \"kubernetes.io/projected/d7c70639-a47b-4adf-8a6b-5ff2cbbe458c-kube-api-access-dpd5d\") pod \"d7c70639-a47b-4adf-8a6b-5ff2cbbe458c\" (UID: \"d7c70639-a47b-4adf-8a6b-5ff2cbbe458c\") "
Jul 06 14:54:30.493774 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:30.493471    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7c70639-a47b-4adf-8a6b-5ff2cbbe458c-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d7c70639-a47b-4adf-8a6b-5ff2cbbe458c" (UID: "d7c70639-a47b-4adf-8a6b-5ff2cbbe458c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 14:54:30.493774 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:30.493738    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7c70639-a47b-4adf-8a6b-5ff2cbbe458c-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d7c70639-a47b-4adf-8a6b-5ff2cbbe458c" (UID: "d7c70639-a47b-4adf-8a6b-5ff2cbbe458c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 14:54:30.493774 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:30.493768    2559 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7c70639-a47b-4adf-8a6b-5ff2cbbe458c-credential-socket\") on node \"ip-10-0-133-112.ec2.internal\" DevicePath \"\""
Jul 06 14:54:30.493983 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:30.493767    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d7c70639-a47b-4adf-8a6b-5ff2cbbe458c-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d7c70639-a47b-4adf-8a6b-5ff2cbbe458c" (UID: "d7c70639-a47b-4adf-8a6b-5ff2cbbe458c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 14:54:30.493983 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:30.493890    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7c70639-a47b-4adf-8a6b-5ff2cbbe458c-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d7c70639-a47b-4adf-8a6b-5ff2cbbe458c" (UID: "d7c70639-a47b-4adf-8a6b-5ff2cbbe458c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 14:54:30.494146 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:30.494123    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7c70639-a47b-4adf-8a6b-5ff2cbbe458c-istio-data" (OuterVolumeSpecName: "istio-data") pod "d7c70639-a47b-4adf-8a6b-5ff2cbbe458c" (UID: "d7c70639-a47b-4adf-8a6b-5ff2cbbe458c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 14:54:30.495607 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:30.495561    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d7c70639-a47b-4adf-8a6b-5ff2cbbe458c-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d7c70639-a47b-4adf-8a6b-5ff2cbbe458c" (UID: "d7c70639-a47b-4adf-8a6b-5ff2cbbe458c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 06 14:54:30.495607 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:30.495589    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7c70639-a47b-4adf-8a6b-5ff2cbbe458c-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d7c70639-a47b-4adf-8a6b-5ff2cbbe458c" (UID: "d7c70639-a47b-4adf-8a6b-5ff2cbbe458c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 14:54:30.495788 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:30.495773    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d7c70639-a47b-4adf-8a6b-5ff2cbbe458c-istio-token" (OuterVolumeSpecName: "istio-token") pod "d7c70639-a47b-4adf-8a6b-5ff2cbbe458c" (UID: "d7c70639-a47b-4adf-8a6b-5ff2cbbe458c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 14:54:30.495855 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:30.495831    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d7c70639-a47b-4adf-8a6b-5ff2cbbe458c-kube-api-access-dpd5d" (OuterVolumeSpecName: "kube-api-access-dpd5d") pod "d7c70639-a47b-4adf-8a6b-5ff2cbbe458c" (UID: "d7c70639-a47b-4adf-8a6b-5ff2cbbe458c"). InnerVolumeSpecName "kube-api-access-dpd5d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 14:54:30.594505 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:30.594407    2559 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d7c70639-a47b-4adf-8a6b-5ff2cbbe458c-istio-data\") on node \"ip-10-0-133-112.ec2.internal\" DevicePath \"\""
Jul 06 14:54:30.594505 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:30.594443    2559 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d7c70639-a47b-4adf-8a6b-5ff2cbbe458c-istiod-ca-cert\") on node \"ip-10-0-133-112.ec2.internal\" DevicePath \"\""
Jul 06 14:54:30.594505 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:30.594451    2559 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d7c70639-a47b-4adf-8a6b-5ff2cbbe458c-istio-envoy\") on node \"ip-10-0-133-112.ec2.internal\" DevicePath \"\""
Jul 06 14:54:30.594505 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:30.594459    2559 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d7c70639-a47b-4adf-8a6b-5ff2cbbe458c-istio-token\") on node \"ip-10-0-133-112.ec2.internal\" DevicePath \"\""
Jul 06 14:54:30.594505 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:30.594467    2559 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7c70639-a47b-4adf-8a6b-5ff2cbbe458c-workload-socket\") on node \"ip-10-0-133-112.ec2.internal\" DevicePath \"\""
Jul 06 14:54:30.594505 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:30.594475    2559 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d7c70639-a47b-4adf-8a6b-5ff2cbbe458c-workload-certs\") on node \"ip-10-0-133-112.ec2.internal\" DevicePath \"\""
Jul 06 14:54:30.594505 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:30.594484    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dpd5d\" (UniqueName: \"kubernetes.io/projected/d7c70639-a47b-4adf-8a6b-5ff2cbbe458c-kube-api-access-dpd5d\") on node \"ip-10-0-133-112.ec2.internal\" DevicePath \"\""
Jul 06 14:54:30.594505 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:30.594492    2559 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d7c70639-a47b-4adf-8a6b-5ff2cbbe458c-istio-podinfo\") on node \"ip-10-0-133-112.ec2.internal\" DevicePath \"\""
Jul 06 14:54:31.236614 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:31.236564    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-5c757b-openshift-default-7d9c6fc49-rqn6k"
Jul 06 14:54:31.236614 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:31.236606    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-5c757b-openshift-default-7d9c6fc49-rqn6k" event={"ID":"d7c70639-a47b-4adf-8a6b-5ff2cbbe458c","Type":"ContainerDied","Data":"825b06c1466b98b7d4e32827b123aceebbe8c56aca54cf7b392a240dfe178ff3"}
Jul 06 14:54:31.237114 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:31.236669    2559 scope.go:117] "RemoveContainer" containerID="3ab4885c4172bf9498b7e25b32523dd37a7fc79d5317dfdea2be9ecfc54941c3"
Jul 06 14:54:31.261527 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:31.261497    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-5c757b-openshift-default-7d9c6fc49-rqn6k"]
Jul 06 14:54:31.266253 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:31.266230    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-5c757b-openshift-default-7d9c6fc49-rqn6k"]
Jul 06 14:54:32.228239 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:32.228206    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d7c70639-a47b-4adf-8a6b-5ff2cbbe458c" path="/var/lib/kubelet/pods/d7c70639-a47b-4adf-8a6b-5ff2cbbe458c/volumes"
Jul 06 14:54:32.343792 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:32.343759    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-6c8b7d-openshift-default-746555f497-8mn74"]
Jul 06 14:54:32.344193 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:32.343979    2559 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-6c8b7d-openshift-default-746555f497-8mn74" podUID="3ede5beb-23e8-4a16-8a23-ca419a6a83d9" containerName="istio-proxy" containerID="cri-o://20579e81aca2aae35e1f661bbd2bafae1dd5003eb96575bcc865cfbe44c3a74b" gracePeriod=30
Jul 06 14:54:33.195461 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:33.195421    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-bb4316-openshift-default-79cd6875-cjg58"]
Jul 06 14:54:33.195860 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:33.195847    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8f5fc6d8-3a9d-4786-adff-4f13542d8d97" containerName="maas-api"
Jul 06 14:54:33.195913 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:33.195862    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="8f5fc6d8-3a9d-4786-adff-4f13542d8d97" containerName="maas-api"
Jul 06 14:54:33.195913 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:33.195878    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d7c70639-a47b-4adf-8a6b-5ff2cbbe458c" containerName="istio-proxy"
Jul 06 14:54:33.195913 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:33.195884    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="d7c70639-a47b-4adf-8a6b-5ff2cbbe458c" containerName="istio-proxy"
Jul 06 14:54:33.195913 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:33.195890    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d7c70639-a47b-4adf-8a6b-5ff2cbbe458c" containerName="istio-proxy"
Jul 06 14:54:33.195913 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:33.195895    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="d7c70639-a47b-4adf-8a6b-5ff2cbbe458c" containerName="istio-proxy"
Jul 06 14:54:33.195913 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:33.195904    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="32e5d6bd-559d-407e-b3c2-a1f6c9fe7bf8" containerName="maas-api"
Jul 06 14:54:33.195913 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:33.195910    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="32e5d6bd-559d-407e-b3c2-a1f6c9fe7bf8" containerName="maas-api"
Jul 06 14:54:33.196126 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:33.195972    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="d7c70639-a47b-4adf-8a6b-5ff2cbbe458c" containerName="istio-proxy"
Jul 06 14:54:33.196126 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:33.195980    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="32e5d6bd-559d-407e-b3c2-a1f6c9fe7bf8" containerName="maas-api"
Jul 06 14:54:33.196126 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:33.195986    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="8f5fc6d8-3a9d-4786-adff-4f13542d8d97" containerName="maas-api"
Jul 06 14:54:33.196126 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:33.195992    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="d7c70639-a47b-4adf-8a6b-5ff2cbbe458c" containerName="istio-proxy"
Jul 06 14:54:33.200743 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:33.200715    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-bb4316-openshift-default-79cd6875-cjg58"
Jul 06 14:54:33.203435 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:33.203414    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-bb4316-openshift-default-dockercfg-5xn9l\""
Jul 06 14:54:33.213435 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:33.213411    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-bb4316-openshift-default-79cd6875-cjg58"]
Jul 06 14:54:33.215702 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:33.215675    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8b9bfd85-5b75-4113-b0f6-656bd17770d4-istio-envoy\") pod \"e2e-shared-a-bb4316-openshift-default-79cd6875-cjg58\" (UID: \"8b9bfd85-5b75-4113-b0f6-656bd17770d4\") " pod="openshift-ingress/e2e-shared-a-bb4316-openshift-default-79cd6875-cjg58"
Jul 06 14:54:33.215814 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:33.215711    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8b9bfd85-5b75-4113-b0f6-656bd17770d4-istiod-ca-cert\") pod \"e2e-shared-a-bb4316-openshift-default-79cd6875-cjg58\" (UID: \"8b9bfd85-5b75-4113-b0f6-656bd17770d4\") " pod="openshift-ingress/e2e-shared-a-bb4316-openshift-default-79cd6875-cjg58"
Jul 06 14:54:33.215814 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:33.215770    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8b9bfd85-5b75-4113-b0f6-656bd17770d4-istio-podinfo\") pod \"e2e-shared-a-bb4316-openshift-default-79cd6875-cjg58\" (UID: \"8b9bfd85-5b75-4113-b0f6-656bd17770d4\") " pod="openshift-ingress/e2e-shared-a-bb4316-openshift-default-79cd6875-cjg58"
Jul 06 14:54:33.215814 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:33.215801    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b9bfd85-5b75-4113-b0f6-656bd17770d4-credential-socket\") pod \"e2e-shared-a-bb4316-openshift-default-79cd6875-cjg58\" (UID: \"8b9bfd85-5b75-4113-b0f6-656bd17770d4\") " pod="openshift-ingress/e2e-shared-a-bb4316-openshift-default-79cd6875-cjg58"
Jul 06 14:54:33.215939 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:33.215856    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8b9bfd85-5b75-4113-b0f6-656bd17770d4-istio-token\") pod \"e2e-shared-a-bb4316-openshift-default-79cd6875-cjg58\" (UID: \"8b9bfd85-5b75-4113-b0f6-656bd17770d4\") " pod="openshift-ingress/e2e-shared-a-bb4316-openshift-default-79cd6875-cjg58"
Jul 06 14:54:33.215939 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:33.215876    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b9bfd85-5b75-4113-b0f6-656bd17770d4-workload-socket\") pod \"e2e-shared-a-bb4316-openshift-default-79cd6875-cjg58\" (UID: \"8b9bfd85-5b75-4113-b0f6-656bd17770d4\") " pod="openshift-ingress/e2e-shared-a-bb4316-openshift-default-79cd6875-cjg58"
Jul 06 14:54:33.215939 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:33.215894    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8b9bfd85-5b75-4113-b0f6-656bd17770d4-workload-certs\") pod \"e2e-shared-a-bb4316-openshift-default-79cd6875-cjg58\" (UID: \"8b9bfd85-5b75-4113-b0f6-656bd17770d4\") " pod="openshift-ingress/e2e-shared-a-bb4316-openshift-default-79cd6875-cjg58"
Jul 06 14:54:33.215939 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:33.215914    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8b9bfd85-5b75-4113-b0f6-656bd17770d4-istio-data\") pod \"e2e-shared-a-bb4316-openshift-default-79cd6875-cjg58\" (UID: \"8b9bfd85-5b75-4113-b0f6-656bd17770d4\") " pod="openshift-ingress/e2e-shared-a-bb4316-openshift-default-79cd6875-cjg58"
Jul 06 14:54:33.215939 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:33.215936    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d99rz\" (UniqueName: \"kubernetes.io/projected/8b9bfd85-5b75-4113-b0f6-656bd17770d4-kube-api-access-d99rz\") pod \"e2e-shared-a-bb4316-openshift-default-79cd6875-cjg58\" (UID: \"8b9bfd85-5b75-4113-b0f6-656bd17770d4\") " pod="openshift-ingress/e2e-shared-a-bb4316-openshift-default-79cd6875-cjg58"
Jul 06 14:54:33.316429 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:33.316393    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8b9bfd85-5b75-4113-b0f6-656bd17770d4-istio-envoy\") pod \"e2e-shared-a-bb4316-openshift-default-79cd6875-cjg58\" (UID: \"8b9bfd85-5b75-4113-b0f6-656bd17770d4\") " pod="openshift-ingress/e2e-shared-a-bb4316-openshift-default-79cd6875-cjg58"
Jul 06 14:54:33.316652 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:33.316437    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8b9bfd85-5b75-4113-b0f6-656bd17770d4-istiod-ca-cert\") pod \"e2e-shared-a-bb4316-openshift-default-79cd6875-cjg58\" (UID: \"8b9bfd85-5b75-4113-b0f6-656bd17770d4\") " pod="openshift-ingress/e2e-shared-a-bb4316-openshift-default-79cd6875-cjg58"
Jul 06 14:54:33.316652 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:33.316494    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8b9bfd85-5b75-4113-b0f6-656bd17770d4-istio-podinfo\") pod \"e2e-shared-a-bb4316-openshift-default-79cd6875-cjg58\" (UID: \"8b9bfd85-5b75-4113-b0f6-656bd17770d4\") " pod="openshift-ingress/e2e-shared-a-bb4316-openshift-default-79cd6875-cjg58"
Jul 06 14:54:33.316652 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:33.316637    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b9bfd85-5b75-4113-b0f6-656bd17770d4-credential-socket\") pod \"e2e-shared-a-bb4316-openshift-default-79cd6875-cjg58\" (UID: \"8b9bfd85-5b75-4113-b0f6-656bd17770d4\") " pod="openshift-ingress/e2e-shared-a-bb4316-openshift-default-79cd6875-cjg58"
Jul 06 14:54:33.316834 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:33.316705    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8b9bfd85-5b75-4113-b0f6-656bd17770d4-istio-token\") pod \"e2e-shared-a-bb4316-openshift-default-79cd6875-cjg58\" (UID: \"8b9bfd85-5b75-4113-b0f6-656bd17770d4\") " pod="openshift-ingress/e2e-shared-a-bb4316-openshift-default-79cd6875-cjg58"
Jul 06 14:54:33.316834 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:33.316731    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b9bfd85-5b75-4113-b0f6-656bd17770d4-workload-socket\") pod \"e2e-shared-a-bb4316-openshift-default-79cd6875-cjg58\" (UID: \"8b9bfd85-5b75-4113-b0f6-656bd17770d4\") " pod="openshift-ingress/e2e-shared-a-bb4316-openshift-default-79cd6875-cjg58"
Jul 06 14:54:33.316834 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:33.316746    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8b9bfd85-5b75-4113-b0f6-656bd17770d4-workload-certs\") pod \"e2e-shared-a-bb4316-openshift-default-79cd6875-cjg58\" (UID: \"8b9bfd85-5b75-4113-b0f6-656bd17770d4\") " pod="openshift-ingress/e2e-shared-a-bb4316-openshift-default-79cd6875-cjg58"
Jul 06 14:54:33.316834 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:33.316773    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8b9bfd85-5b75-4113-b0f6-656bd17770d4-istio-data\") pod \"e2e-shared-a-bb4316-openshift-default-79cd6875-cjg58\" (UID: \"8b9bfd85-5b75-4113-b0f6-656bd17770d4\") " pod="openshift-ingress/e2e-shared-a-bb4316-openshift-default-79cd6875-cjg58"
Jul 06 14:54:33.316834 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:33.316811    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d99rz\" (UniqueName: \"kubernetes.io/projected/8b9bfd85-5b75-4113-b0f6-656bd17770d4-kube-api-access-d99rz\") pod \"e2e-shared-a-bb4316-openshift-default-79cd6875-cjg58\" (UID: \"8b9bfd85-5b75-4113-b0f6-656bd17770d4\") " pod="openshift-ingress/e2e-shared-a-bb4316-openshift-default-79cd6875-cjg58"
Jul 06 14:54:33.317091 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:33.317029    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b9bfd85-5b75-4113-b0f6-656bd17770d4-credential-socket\") pod \"e2e-shared-a-bb4316-openshift-default-79cd6875-cjg58\" (UID: \"8b9bfd85-5b75-4113-b0f6-656bd17770d4\") " pod="openshift-ingress/e2e-shared-a-bb4316-openshift-default-79cd6875-cjg58"
Jul 06 14:54:33.317091 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:33.317074    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8b9bfd85-5b75-4113-b0f6-656bd17770d4-workload-certs\") pod \"e2e-shared-a-bb4316-openshift-default-79cd6875-cjg58\" (UID: \"8b9bfd85-5b75-4113-b0f6-656bd17770d4\") " pod="openshift-ingress/e2e-shared-a-bb4316-openshift-default-79cd6875-cjg58"
Jul 06 14:54:33.317257 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:33.317238    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8b9bfd85-5b75-4113-b0f6-656bd17770d4-istio-data\") pod \"e2e-shared-a-bb4316-openshift-default-79cd6875-cjg58\" (UID: \"8b9bfd85-5b75-4113-b0f6-656bd17770d4\") " pod="openshift-ingress/e2e-shared-a-bb4316-openshift-default-79cd6875-cjg58"
Jul 06 14:54:33.317303 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:33.317258    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b9bfd85-5b75-4113-b0f6-656bd17770d4-workload-socket\") pod \"e2e-shared-a-bb4316-openshift-default-79cd6875-cjg58\" (UID: \"8b9bfd85-5b75-4113-b0f6-656bd17770d4\") " pod="openshift-ingress/e2e-shared-a-bb4316-openshift-default-79cd6875-cjg58"
Jul 06 14:54:33.317343 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:33.317317    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8b9bfd85-5b75-4113-b0f6-656bd17770d4-istiod-ca-cert\") pod \"e2e-shared-a-bb4316-openshift-default-79cd6875-cjg58\" (UID: \"8b9bfd85-5b75-4113-b0f6-656bd17770d4\") " pod="openshift-ingress/e2e-shared-a-bb4316-openshift-default-79cd6875-cjg58"
Jul 06 14:54:33.319086 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:33.319064    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8b9bfd85-5b75-4113-b0f6-656bd17770d4-istio-envoy\") pod \"e2e-shared-a-bb4316-openshift-default-79cd6875-cjg58\" (UID: \"8b9bfd85-5b75-4113-b0f6-656bd17770d4\") " pod="openshift-ingress/e2e-shared-a-bb4316-openshift-default-79cd6875-cjg58"
Jul 06 14:54:33.319191 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:33.319096    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8b9bfd85-5b75-4113-b0f6-656bd17770d4-istio-podinfo\") pod \"e2e-shared-a-bb4316-openshift-default-79cd6875-cjg58\" (UID: \"8b9bfd85-5b75-4113-b0f6-656bd17770d4\") " pod="openshift-ingress/e2e-shared-a-bb4316-openshift-default-79cd6875-cjg58"
Jul 06 14:54:33.325419 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:33.325393    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d99rz\" (UniqueName: \"kubernetes.io/projected/8b9bfd85-5b75-4113-b0f6-656bd17770d4-kube-api-access-d99rz\") pod \"e2e-shared-a-bb4316-openshift-default-79cd6875-cjg58\" (UID: \"8b9bfd85-5b75-4113-b0f6-656bd17770d4\") " pod="openshift-ingress/e2e-shared-a-bb4316-openshift-default-79cd6875-cjg58"
Jul 06 14:54:33.326607 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:33.326562    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8b9bfd85-5b75-4113-b0f6-656bd17770d4-istio-token\") pod \"e2e-shared-a-bb4316-openshift-default-79cd6875-cjg58\" (UID: \"8b9bfd85-5b75-4113-b0f6-656bd17770d4\") " pod="openshift-ingress/e2e-shared-a-bb4316-openshift-default-79cd6875-cjg58"
Jul 06 14:54:33.511354 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:33.511273    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-bb4316-openshift-default-79cd6875-cjg58"
Jul 06 14:54:33.657394 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:33.657366    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-bb4316-openshift-default-79cd6875-cjg58"]
Jul 06 14:54:33.659039 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:54:33.659011    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b9bfd85_5b75_4113_b0f6_656bd17770d4.slice/crio-638b851f37857089bc3c7052601913ea515aab81339de62b0e2e6a4634b1a3da WatchSource:0}: Error finding container 638b851f37857089bc3c7052601913ea515aab81339de62b0e2e6a4634b1a3da: Status 404 returned error can't find the container with id 638b851f37857089bc3c7052601913ea515aab81339de62b0e2e6a4634b1a3da
Jul 06 14:54:33.661210 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:33.661176    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 06 14:54:33.661301 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:33.661243    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 06 14:54:33.661301 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:33.661269    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 06 14:54:34.254440 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:34.254402    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-bb4316-openshift-default-79cd6875-cjg58" event={"ID":"8b9bfd85-5b75-4113-b0f6-656bd17770d4","Type":"ContainerStarted","Data":"5bd2442ff3ca80f9d4e4851d365d661526102cd2689080444a0af3de63188a12"}
Jul 06 14:54:34.254634 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:34.254468    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-bb4316-openshift-default-79cd6875-cjg58" event={"ID":"8b9bfd85-5b75-4113-b0f6-656bd17770d4","Type":"ContainerStarted","Data":"638b851f37857089bc3c7052601913ea515aab81339de62b0e2e6a4634b1a3da"}
Jul 06 14:54:34.277039 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:34.276979    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-bb4316-openshift-default-79cd6875-cjg58" podStartSLOduration=1.276962664 podStartE2EDuration="1.276962664s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:54:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:54:34.275560805 +0000 UTC m=+2096.582852902" watchObservedRunningTime="2026-07-06 14:54:34.276962664 +0000 UTC m=+2096.584254788"
Jul 06 14:54:34.512350 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:34.512260    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-bb4316-openshift-default-79cd6875-cjg58"
Jul 06 14:54:34.517263 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:34.517238    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-bb4316-openshift-default-79cd6875-cjg58"
Jul 06 14:54:35.259798 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:35.259770    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-bb4316-openshift-default-79cd6875-cjg58"
Jul 06 14:54:35.260681 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:35.260663    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-bb4316-openshift-default-79cd6875-cjg58"
Jul 06 14:54:36.528306 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:36.528277    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-shared-a-bb4316-6d6d64dddd-24qw7"]
Jul 06 14:54:36.532008 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:36.531987    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-bb4316-6d6d64dddd-24qw7"
Jul 06 14:54:36.534334 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:36.534315    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-shared-a-bb4316-serving-cert\""
Jul 06 14:54:36.543091 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:36.543069    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-bb4316-6d6d64dddd-24qw7"]
Jul 06 14:54:36.546080 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:36.546051    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rdgcp\" (UniqueName: \"kubernetes.io/projected/9f4c88f0-8bd9-43a1-a648-a312759fe26d-kube-api-access-rdgcp\") pod \"maas-api-e2e-shared-a-bb4316-6d6d64dddd-24qw7\" (UID: \"9f4c88f0-8bd9-43a1-a648-a312759fe26d\") " pod="opendatahub/maas-api-e2e-shared-a-bb4316-6d6d64dddd-24qw7"
Jul 06 14:54:36.546213 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:36.546106    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9f4c88f0-8bd9-43a1-a648-a312759fe26d-maas-api-tls\") pod \"maas-api-e2e-shared-a-bb4316-6d6d64dddd-24qw7\" (UID: \"9f4c88f0-8bd9-43a1-a648-a312759fe26d\") " pod="opendatahub/maas-api-e2e-shared-a-bb4316-6d6d64dddd-24qw7"
Jul 06 14:54:36.546287 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:36.546235    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9f4c88f0-8bd9-43a1-a648-a312759fe26d-openshift-service-ca\") pod \"maas-api-e2e-shared-a-bb4316-6d6d64dddd-24qw7\" (UID: \"9f4c88f0-8bd9-43a1-a648-a312759fe26d\") " pod="opendatahub/maas-api-e2e-shared-a-bb4316-6d6d64dddd-24qw7"
Jul 06 14:54:36.647563 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:36.647527    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rdgcp\" (UniqueName: \"kubernetes.io/projected/9f4c88f0-8bd9-43a1-a648-a312759fe26d-kube-api-access-rdgcp\") pod \"maas-api-e2e-shared-a-bb4316-6d6d64dddd-24qw7\" (UID: \"9f4c88f0-8bd9-43a1-a648-a312759fe26d\") " pod="opendatahub/maas-api-e2e-shared-a-bb4316-6d6d64dddd-24qw7"
Jul 06 14:54:36.647563 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:36.647566    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9f4c88f0-8bd9-43a1-a648-a312759fe26d-maas-api-tls\") pod \"maas-api-e2e-shared-a-bb4316-6d6d64dddd-24qw7\" (UID: \"9f4c88f0-8bd9-43a1-a648-a312759fe26d\") " pod="opendatahub/maas-api-e2e-shared-a-bb4316-6d6d64dddd-24qw7"
Jul 06 14:54:36.647799 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:36.647635    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9f4c88f0-8bd9-43a1-a648-a312759fe26d-openshift-service-ca\") pod \"maas-api-e2e-shared-a-bb4316-6d6d64dddd-24qw7\" (UID: \"9f4c88f0-8bd9-43a1-a648-a312759fe26d\") " pod="opendatahub/maas-api-e2e-shared-a-bb4316-6d6d64dddd-24qw7"
Jul 06 14:54:36.647799 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:54:36.647746    2559 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-shared-a-bb4316-serving-cert: secret "maas-api-e2e-shared-a-bb4316-serving-cert" not found
Jul 06 14:54:36.647894 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:54:36.647809    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9f4c88f0-8bd9-43a1-a648-a312759fe26d-maas-api-tls podName:9f4c88f0-8bd9-43a1-a648-a312759fe26d nodeName:}" failed. No retries permitted until 2026-07-06 14:54:37.147791376 +0000 UTC m=+2099.455083442 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/9f4c88f0-8bd9-43a1-a648-a312759fe26d-maas-api-tls") pod "maas-api-e2e-shared-a-bb4316-6d6d64dddd-24qw7" (UID: "9f4c88f0-8bd9-43a1-a648-a312759fe26d") : secret "maas-api-e2e-shared-a-bb4316-serving-cert" not found
Jul 06 14:54:36.648310 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:36.648292    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9f4c88f0-8bd9-43a1-a648-a312759fe26d-openshift-service-ca\") pod \"maas-api-e2e-shared-a-bb4316-6d6d64dddd-24qw7\" (UID: \"9f4c88f0-8bd9-43a1-a648-a312759fe26d\") " pod="opendatahub/maas-api-e2e-shared-a-bb4316-6d6d64dddd-24qw7"
Jul 06 14:54:36.659636 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:36.659612    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rdgcp\" (UniqueName: \"kubernetes.io/projected/9f4c88f0-8bd9-43a1-a648-a312759fe26d-kube-api-access-rdgcp\") pod \"maas-api-e2e-shared-a-bb4316-6d6d64dddd-24qw7\" (UID: \"9f4c88f0-8bd9-43a1-a648-a312759fe26d\") " pod="opendatahub/maas-api-e2e-shared-a-bb4316-6d6d64dddd-24qw7"
Jul 06 14:54:37.152626 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:37.152557    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9f4c88f0-8bd9-43a1-a648-a312759fe26d-maas-api-tls\") pod \"maas-api-e2e-shared-a-bb4316-6d6d64dddd-24qw7\" (UID: \"9f4c88f0-8bd9-43a1-a648-a312759fe26d\") " pod="opendatahub/maas-api-e2e-shared-a-bb4316-6d6d64dddd-24qw7"
Jul 06 14:54:37.155078 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:37.155049    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9f4c88f0-8bd9-43a1-a648-a312759fe26d-maas-api-tls\") pod \"maas-api-e2e-shared-a-bb4316-6d6d64dddd-24qw7\" (UID: \"9f4c88f0-8bd9-43a1-a648-a312759fe26d\") " pod="opendatahub/maas-api-e2e-shared-a-bb4316-6d6d64dddd-24qw7"
Jul 06 14:54:37.443707 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:37.443681    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-bb4316-6d6d64dddd-24qw7"
Jul 06 14:54:37.606330 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:37.606298    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-bb4316-6d6d64dddd-24qw7"]
Jul 06 14:54:37.608523 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:54:37.608498    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f4c88f0_8bd9_43a1_a648_a312759fe26d.slice/crio-ec9ae4e1692c0f53e866b5a43bc5940ec6dcb23767f53632ce0260255b71ba6b WatchSource:0}: Error finding container ec9ae4e1692c0f53e866b5a43bc5940ec6dcb23767f53632ce0260255b71ba6b: Status 404 returned error can't find the container with id ec9ae4e1692c0f53e866b5a43bc5940ec6dcb23767f53632ce0260255b71ba6b
Jul 06 14:54:37.610207 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:37.610183    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-6c8b7d-openshift-default-746555f497-8mn74"
Jul 06 14:54:37.657921 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:37.657893    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3ede5beb-23e8-4a16-8a23-ca419a6a83d9-workload-certs\") pod \"3ede5beb-23e8-4a16-8a23-ca419a6a83d9\" (UID: \"3ede5beb-23e8-4a16-8a23-ca419a6a83d9\") "
Jul 06 14:54:37.658087 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:37.657973    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3ede5beb-23e8-4a16-8a23-ca419a6a83d9-istiod-ca-cert\") pod \"3ede5beb-23e8-4a16-8a23-ca419a6a83d9\" (UID: \"3ede5beb-23e8-4a16-8a23-ca419a6a83d9\") "
Jul 06 14:54:37.658087 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:37.658006    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-br2kc\" (UniqueName: \"kubernetes.io/projected/3ede5beb-23e8-4a16-8a23-ca419a6a83d9-kube-api-access-br2kc\") pod \"3ede5beb-23e8-4a16-8a23-ca419a6a83d9\" (UID: \"3ede5beb-23e8-4a16-8a23-ca419a6a83d9\") "
Jul 06 14:54:37.658087 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:37.658028    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3ede5beb-23e8-4a16-8a23-ca419a6a83d9-istio-data\") pod \"3ede5beb-23e8-4a16-8a23-ca419a6a83d9\" (UID: \"3ede5beb-23e8-4a16-8a23-ca419a6a83d9\") "
Jul 06 14:54:37.658087 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:37.658054    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3ede5beb-23e8-4a16-8a23-ca419a6a83d9-istio-envoy\") pod \"3ede5beb-23e8-4a16-8a23-ca419a6a83d9\" (UID: \"3ede5beb-23e8-4a16-8a23-ca419a6a83d9\") "
Jul 06 14:54:37.658087 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:37.658082    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ede5beb-23e8-4a16-8a23-ca419a6a83d9-workload-socket\") pod \"3ede5beb-23e8-4a16-8a23-ca419a6a83d9\" (UID: \"3ede5beb-23e8-4a16-8a23-ca419a6a83d9\") "
Jul 06 14:54:37.658359 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:37.658111    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3ede5beb-23e8-4a16-8a23-ca419a6a83d9-istio-token\") pod \"3ede5beb-23e8-4a16-8a23-ca419a6a83d9\" (UID: \"3ede5beb-23e8-4a16-8a23-ca419a6a83d9\") "
Jul 06 14:54:37.658359 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:37.658161    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3ede5beb-23e8-4a16-8a23-ca419a6a83d9-istio-podinfo\") pod \"3ede5beb-23e8-4a16-8a23-ca419a6a83d9\" (UID: \"3ede5beb-23e8-4a16-8a23-ca419a6a83d9\") "
Jul 06 14:54:37.658359 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:37.658170    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3ede5beb-23e8-4a16-8a23-ca419a6a83d9-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "3ede5beb-23e8-4a16-8a23-ca419a6a83d9" (UID: "3ede5beb-23e8-4a16-8a23-ca419a6a83d9"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 14:54:37.658359 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:37.658222    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ede5beb-23e8-4a16-8a23-ca419a6a83d9-credential-socket\") pod \"3ede5beb-23e8-4a16-8a23-ca419a6a83d9\" (UID: \"3ede5beb-23e8-4a16-8a23-ca419a6a83d9\") "
Jul 06 14:54:37.658599 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:37.658478    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3ede5beb-23e8-4a16-8a23-ca419a6a83d9-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "3ede5beb-23e8-4a16-8a23-ca419a6a83d9" (UID: "3ede5beb-23e8-4a16-8a23-ca419a6a83d9"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 14:54:37.658599 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:37.658542    2559 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3ede5beb-23e8-4a16-8a23-ca419a6a83d9-workload-certs\") on node \"ip-10-0-133-112.ec2.internal\" DevicePath \"\""
Jul 06 14:54:37.658936 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:37.658726    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3ede5beb-23e8-4a16-8a23-ca419a6a83d9-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "3ede5beb-23e8-4a16-8a23-ca419a6a83d9" (UID: "3ede5beb-23e8-4a16-8a23-ca419a6a83d9"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 14:54:37.658936 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:37.658772    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3ede5beb-23e8-4a16-8a23-ca419a6a83d9-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "3ede5beb-23e8-4a16-8a23-ca419a6a83d9" (UID: "3ede5beb-23e8-4a16-8a23-ca419a6a83d9"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 14:54:37.658936 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:37.658891    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3ede5beb-23e8-4a16-8a23-ca419a6a83d9-istio-data" (OuterVolumeSpecName: "istio-data") pod "3ede5beb-23e8-4a16-8a23-ca419a6a83d9" (UID: "3ede5beb-23e8-4a16-8a23-ca419a6a83d9"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 14:54:37.660806 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:37.660773    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3ede5beb-23e8-4a16-8a23-ca419a6a83d9-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "3ede5beb-23e8-4a16-8a23-ca419a6a83d9" (UID: "3ede5beb-23e8-4a16-8a23-ca419a6a83d9"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 14:54:37.660806 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:37.660795    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3ede5beb-23e8-4a16-8a23-ca419a6a83d9-istio-token" (OuterVolumeSpecName: "istio-token") pod "3ede5beb-23e8-4a16-8a23-ca419a6a83d9" (UID: "3ede5beb-23e8-4a16-8a23-ca419a6a83d9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 14:54:37.660954 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:37.660844    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3ede5beb-23e8-4a16-8a23-ca419a6a83d9-kube-api-access-br2kc" (OuterVolumeSpecName: "kube-api-access-br2kc") pod "3ede5beb-23e8-4a16-8a23-ca419a6a83d9" (UID: "3ede5beb-23e8-4a16-8a23-ca419a6a83d9"). InnerVolumeSpecName "kube-api-access-br2kc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 14:54:37.660954 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:37.660929    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3ede5beb-23e8-4a16-8a23-ca419a6a83d9-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "3ede5beb-23e8-4a16-8a23-ca419a6a83d9" (UID: "3ede5beb-23e8-4a16-8a23-ca419a6a83d9"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 06 14:54:37.759873 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:37.759783    2559 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ede5beb-23e8-4a16-8a23-ca419a6a83d9-workload-socket\") on node \"ip-10-0-133-112.ec2.internal\" DevicePath \"\""
Jul 06 14:54:37.759873 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:37.759814    2559 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3ede5beb-23e8-4a16-8a23-ca419a6a83d9-istio-token\") on node \"ip-10-0-133-112.ec2.internal\" DevicePath \"\""
Jul 06 14:54:37.759873 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:37.759823    2559 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3ede5beb-23e8-4a16-8a23-ca419a6a83d9-istio-podinfo\") on node \"ip-10-0-133-112.ec2.internal\" DevicePath \"\""
Jul 06 14:54:37.759873 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:37.759831    2559 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ede5beb-23e8-4a16-8a23-ca419a6a83d9-credential-socket\") on node \"ip-10-0-133-112.ec2.internal\" DevicePath \"\""
Jul 06 14:54:37.759873 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:37.759842    2559 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3ede5beb-23e8-4a16-8a23-ca419a6a83d9-istiod-ca-cert\") on node \"ip-10-0-133-112.ec2.internal\" DevicePath \"\""
Jul 06 14:54:37.759873 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:37.759851    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-br2kc\" (UniqueName: \"kubernetes.io/projected/3ede5beb-23e8-4a16-8a23-ca419a6a83d9-kube-api-access-br2kc\") on node \"ip-10-0-133-112.ec2.internal\" DevicePath \"\""
Jul 06 14:54:37.759873 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:37.759859    2559 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3ede5beb-23e8-4a16-8a23-ca419a6a83d9-istio-data\") on node \"ip-10-0-133-112.ec2.internal\" DevicePath \"\""
Jul 06 14:54:37.759873 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:37.759867    2559 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3ede5beb-23e8-4a16-8a23-ca419a6a83d9-istio-envoy\") on node \"ip-10-0-133-112.ec2.internal\" DevicePath \"\""
Jul 06 14:54:38.276759 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:38.276716    2559 generic.go:356] "Generic (PLEG): container finished" podID="3ede5beb-23e8-4a16-8a23-ca419a6a83d9" containerID="20579e81aca2aae35e1f661bbd2bafae1dd5003eb96575bcc865cfbe44c3a74b" exitCode=0
Jul 06 14:54:38.276957 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:38.276761    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-6c8b7d-openshift-default-746555f497-8mn74"
Jul 06 14:54:38.276957 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:38.276886    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-6c8b7d-openshift-default-746555f497-8mn74" event={"ID":"3ede5beb-23e8-4a16-8a23-ca419a6a83d9","Type":"ContainerDied","Data":"20579e81aca2aae35e1f661bbd2bafae1dd5003eb96575bcc865cfbe44c3a74b"}
Jul 06 14:54:38.276957 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:38.276928    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-6c8b7d-openshift-default-746555f497-8mn74" event={"ID":"3ede5beb-23e8-4a16-8a23-ca419a6a83d9","Type":"ContainerDied","Data":"fd251e94ce5e3fbf5eec6456a8a1af3c27f5f2bdafd95ef0d47c01b6ad62edd9"}
Jul 06 14:54:38.276957 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:38.276949    2559 scope.go:117] "RemoveContainer" containerID="20579e81aca2aae35e1f661bbd2bafae1dd5003eb96575bcc865cfbe44c3a74b"
Jul 06 14:54:38.278538 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:38.278509    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-bb4316-6d6d64dddd-24qw7" event={"ID":"9f4c88f0-8bd9-43a1-a648-a312759fe26d","Type":"ContainerStarted","Data":"e91a8582f157dd3559601ca2335c18fc2da998dd6ff6fdad58dfde063f5dab6f"}
Jul 06 14:54:38.278721 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:38.278560    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-bb4316-6d6d64dddd-24qw7" event={"ID":"9f4c88f0-8bd9-43a1-a648-a312759fe26d","Type":"ContainerStarted","Data":"ec9ae4e1692c0f53e866b5a43bc5940ec6dcb23767f53632ce0260255b71ba6b"}
Jul 06 14:54:38.278721 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:38.278692    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-shared-a-bb4316-6d6d64dddd-24qw7"
Jul 06 14:54:38.289472 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:38.289406    2559 scope.go:117] "RemoveContainer" containerID="bc3827a083e3560045679a4bfe36d15228bc29d1c99d163a3878d809e1ceb074"
Jul 06 14:54:38.292233 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:38.292213    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-g5w5m_e35b0247-7c4e-4bd1-bc28-c084dc40ffa1/console-operator/1.log"
Jul 06 14:54:38.298478 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:38.298444    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-g5w5m_e35b0247-7c4e-4bd1-bc28-c084dc40ffa1/console-operator/1.log"
Jul 06 14:54:38.298722 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:38.298700    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-6c8b7d-openshift-default-746555f497-8mn74"]
Jul 06 14:54:38.300371 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:38.300362    2559 scope.go:117] "RemoveContainer" containerID="20579e81aca2aae35e1f661bbd2bafae1dd5003eb96575bcc865cfbe44c3a74b"
Jul 06 14:54:38.300667 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:54:38.300640    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"20579e81aca2aae35e1f661bbd2bafae1dd5003eb96575bcc865cfbe44c3a74b\": container with ID starting with 20579e81aca2aae35e1f661bbd2bafae1dd5003eb96575bcc865cfbe44c3a74b not found: ID does not exist" containerID="20579e81aca2aae35e1f661bbd2bafae1dd5003eb96575bcc865cfbe44c3a74b"
Jul 06 14:54:38.300778 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:38.300675    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"20579e81aca2aae35e1f661bbd2bafae1dd5003eb96575bcc865cfbe44c3a74b"} err="failed to get container status \"20579e81aca2aae35e1f661bbd2bafae1dd5003eb96575bcc865cfbe44c3a74b\": rpc error: code = NotFound desc = could not find container \"20579e81aca2aae35e1f661bbd2bafae1dd5003eb96575bcc865cfbe44c3a74b\": container with ID starting with 20579e81aca2aae35e1f661bbd2bafae1dd5003eb96575bcc865cfbe44c3a74b not found: ID does not exist"
Jul 06 14:54:38.300778 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:38.300694    2559 scope.go:117] "RemoveContainer" containerID="bc3827a083e3560045679a4bfe36d15228bc29d1c99d163a3878d809e1ceb074"
Jul 06 14:54:38.300968 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:54:38.300941    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bc3827a083e3560045679a4bfe36d15228bc29d1c99d163a3878d809e1ceb074\": container with ID starting with bc3827a083e3560045679a4bfe36d15228bc29d1c99d163a3878d809e1ceb074 not found: ID does not exist" containerID="bc3827a083e3560045679a4bfe36d15228bc29d1c99d163a3878d809e1ceb074"
Jul 06 14:54:38.301078 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:38.300976    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bc3827a083e3560045679a4bfe36d15228bc29d1c99d163a3878d809e1ceb074"} err="failed to get container status \"bc3827a083e3560045679a4bfe36d15228bc29d1c99d163a3878d809e1ceb074\": rpc error: code = NotFound desc = could not find container \"bc3827a083e3560045679a4bfe36d15228bc29d1c99d163a3878d809e1ceb074\": container with ID starting with bc3827a083e3560045679a4bfe36d15228bc29d1c99d163a3878d809e1ceb074 not found: ID does not exist"
Jul 06 14:54:38.305436 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:38.305415    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-6c8b7d-openshift-default-746555f497-8mn74"]
Jul 06 14:54:38.323594 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:38.323529    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-shared-a-bb4316-6d6d64dddd-24qw7" podStartSLOduration=1.813418118 podStartE2EDuration="2.323512734s" totalImagesPullingTime="510.094616ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:54:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:54:38.322643358 +0000 UTC m=+2100.629935441" watchObservedRunningTime="2026-07-06 14:54:38.323512734 +0000 UTC m=+2100.630804816"
Jul 06 14:54:40.228288 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:40.228250    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3ede5beb-23e8-4a16-8a23-ca419a6a83d9" path="/var/lib/kubelet/pods/3ede5beb-23e8-4a16-8a23-ca419a6a83d9/volumes"
Jul 06 14:54:41.483490 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:41.483446    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-aa7c3f-openshift-default-85c8575b7f-blhgz"]
Jul 06 14:54:41.483892 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:41.483866    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3ede5beb-23e8-4a16-8a23-ca419a6a83d9" containerName="istio-proxy"
Jul 06 14:54:41.483892 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:41.483878    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="3ede5beb-23e8-4a16-8a23-ca419a6a83d9" containerName="istio-proxy"
Jul 06 14:54:41.483892 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:41.483890    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3ede5beb-23e8-4a16-8a23-ca419a6a83d9" containerName="istio-proxy"
Jul 06 14:54:41.484003 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:41.483896    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="3ede5beb-23e8-4a16-8a23-ca419a6a83d9" containerName="istio-proxy"
Jul 06 14:54:41.484003 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:41.483960    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="3ede5beb-23e8-4a16-8a23-ca419a6a83d9" containerName="istio-proxy"
Jul 06 14:54:41.484003 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:41.483968    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="3ede5beb-23e8-4a16-8a23-ca419a6a83d9" containerName="istio-proxy"
Jul 06 14:54:41.489000 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:41.488983    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-aa7c3f-openshift-default-85c8575b7f-blhgz"
Jul 06 14:54:41.492274 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:41.492249    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-aa7c3f-openshift-default-dockercfg-ljptb\""
Jul 06 14:54:41.511963 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:41.511935    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-aa7c3f-openshift-default-85c8575b7f-blhgz"]
Jul 06 14:54:41.591892 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:41.591816    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a9dcab5f-6209-4edd-8832-d31608f3c7d3-credential-socket\") pod \"e2e-shared-b-aa7c3f-openshift-default-85c8575b7f-blhgz\" (UID: \"a9dcab5f-6209-4edd-8832-d31608f3c7d3\") " pod="openshift-ingress/e2e-shared-b-aa7c3f-openshift-default-85c8575b7f-blhgz"
Jul 06 14:54:41.592179 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:41.592157    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a9dcab5f-6209-4edd-8832-d31608f3c7d3-workload-socket\") pod \"e2e-shared-b-aa7c3f-openshift-default-85c8575b7f-blhgz\" (UID: \"a9dcab5f-6209-4edd-8832-d31608f3c7d3\") " pod="openshift-ingress/e2e-shared-b-aa7c3f-openshift-default-85c8575b7f-blhgz"
Jul 06 14:54:41.592325 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:41.592301    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a9dcab5f-6209-4edd-8832-d31608f3c7d3-istio-podinfo\") pod \"e2e-shared-b-aa7c3f-openshift-default-85c8575b7f-blhgz\" (UID: \"a9dcab5f-6209-4edd-8832-d31608f3c7d3\") " pod="openshift-ingress/e2e-shared-b-aa7c3f-openshift-default-85c8575b7f-blhgz"
Jul 06 14:54:41.592325 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:41.592338    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a9dcab5f-6209-4edd-8832-d31608f3c7d3-workload-certs\") pod \"e2e-shared-b-aa7c3f-openshift-default-85c8575b7f-blhgz\" (UID: \"a9dcab5f-6209-4edd-8832-d31608f3c7d3\") " pod="openshift-ingress/e2e-shared-b-aa7c3f-openshift-default-85c8575b7f-blhgz"
Jul 06 14:54:41.592517 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:41.592371    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a9dcab5f-6209-4edd-8832-d31608f3c7d3-istiod-ca-cert\") pod \"e2e-shared-b-aa7c3f-openshift-default-85c8575b7f-blhgz\" (UID: \"a9dcab5f-6209-4edd-8832-d31608f3c7d3\") " pod="openshift-ingress/e2e-shared-b-aa7c3f-openshift-default-85c8575b7f-blhgz"
Jul 06 14:54:41.592517 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:41.592408    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a9dcab5f-6209-4edd-8832-d31608f3c7d3-istio-token\") pod \"e2e-shared-b-aa7c3f-openshift-default-85c8575b7f-blhgz\" (UID: \"a9dcab5f-6209-4edd-8832-d31608f3c7d3\") " pod="openshift-ingress/e2e-shared-b-aa7c3f-openshift-default-85c8575b7f-blhgz"
Jul 06 14:54:41.592517 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:41.592445    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a9dcab5f-6209-4edd-8832-d31608f3c7d3-istio-data\") pod \"e2e-shared-b-aa7c3f-openshift-default-85c8575b7f-blhgz\" (UID: \"a9dcab5f-6209-4edd-8832-d31608f3c7d3\") " pod="openshift-ingress/e2e-shared-b-aa7c3f-openshift-default-85c8575b7f-blhgz"
Jul 06 14:54:41.592711 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:41.592522    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g48nr\" (UniqueName: \"kubernetes.io/projected/a9dcab5f-6209-4edd-8832-d31608f3c7d3-kube-api-access-g48nr\") pod \"e2e-shared-b-aa7c3f-openshift-default-85c8575b7f-blhgz\" (UID: \"a9dcab5f-6209-4edd-8832-d31608f3c7d3\") " pod="openshift-ingress/e2e-shared-b-aa7c3f-openshift-default-85c8575b7f-blhgz"
Jul 06 14:54:41.592711 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:41.592633    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a9dcab5f-6209-4edd-8832-d31608f3c7d3-istio-envoy\") pod \"e2e-shared-b-aa7c3f-openshift-default-85c8575b7f-blhgz\" (UID: \"a9dcab5f-6209-4edd-8832-d31608f3c7d3\") " pod="openshift-ingress/e2e-shared-b-aa7c3f-openshift-default-85c8575b7f-blhgz"
Jul 06 14:54:41.693862 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:41.693824    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g48nr\" (UniqueName: \"kubernetes.io/projected/a9dcab5f-6209-4edd-8832-d31608f3c7d3-kube-api-access-g48nr\") pod \"e2e-shared-b-aa7c3f-openshift-default-85c8575b7f-blhgz\" (UID: \"a9dcab5f-6209-4edd-8832-d31608f3c7d3\") " pod="openshift-ingress/e2e-shared-b-aa7c3f-openshift-default-85c8575b7f-blhgz"
Jul 06 14:54:41.694076 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:41.693879    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a9dcab5f-6209-4edd-8832-d31608f3c7d3-istio-envoy\") pod \"e2e-shared-b-aa7c3f-openshift-default-85c8575b7f-blhgz\" (UID: \"a9dcab5f-6209-4edd-8832-d31608f3c7d3\") " pod="openshift-ingress/e2e-shared-b-aa7c3f-openshift-default-85c8575b7f-blhgz"
Jul 06 14:54:41.694076 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:41.693925    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a9dcab5f-6209-4edd-8832-d31608f3c7d3-credential-socket\") pod \"e2e-shared-b-aa7c3f-openshift-default-85c8575b7f-blhgz\" (UID: \"a9dcab5f-6209-4edd-8832-d31608f3c7d3\") " pod="openshift-ingress/e2e-shared-b-aa7c3f-openshift-default-85c8575b7f-blhgz"
Jul 06 14:54:41.694076 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:41.693950    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a9dcab5f-6209-4edd-8832-d31608f3c7d3-workload-socket\") pod \"e2e-shared-b-aa7c3f-openshift-default-85c8575b7f-blhgz\" (UID: \"a9dcab5f-6209-4edd-8832-d31608f3c7d3\") " pod="openshift-ingress/e2e-shared-b-aa7c3f-openshift-default-85c8575b7f-blhgz"
Jul 06 14:54:41.694076 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:41.693970    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a9dcab5f-6209-4edd-8832-d31608f3c7d3-istio-podinfo\") pod \"e2e-shared-b-aa7c3f-openshift-default-85c8575b7f-blhgz\" (UID: \"a9dcab5f-6209-4edd-8832-d31608f3c7d3\") " pod="openshift-ingress/e2e-shared-b-aa7c3f-openshift-default-85c8575b7f-blhgz"
Jul 06 14:54:41.694076 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:41.694059    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a9dcab5f-6209-4edd-8832-d31608f3c7d3-workload-certs\") pod \"e2e-shared-b-aa7c3f-openshift-default-85c8575b7f-blhgz\" (UID: \"a9dcab5f-6209-4edd-8832-d31608f3c7d3\") " pod="openshift-ingress/e2e-shared-b-aa7c3f-openshift-default-85c8575b7f-blhgz"
Jul 06 14:54:41.694443 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:41.694106    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a9dcab5f-6209-4edd-8832-d31608f3c7d3-istiod-ca-cert\") pod \"e2e-shared-b-aa7c3f-openshift-default-85c8575b7f-blhgz\" (UID: \"a9dcab5f-6209-4edd-8832-d31608f3c7d3\") " pod="openshift-ingress/e2e-shared-b-aa7c3f-openshift-default-85c8575b7f-blhgz"
Jul 06 14:54:41.694443 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:41.694142    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a9dcab5f-6209-4edd-8832-d31608f3c7d3-istio-token\") pod \"e2e-shared-b-aa7c3f-openshift-default-85c8575b7f-blhgz\" (UID: \"a9dcab5f-6209-4edd-8832-d31608f3c7d3\") " pod="openshift-ingress/e2e-shared-b-aa7c3f-openshift-default-85c8575b7f-blhgz"
Jul 06 14:54:41.694443 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:41.694174    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a9dcab5f-6209-4edd-8832-d31608f3c7d3-istio-data\") pod \"e2e-shared-b-aa7c3f-openshift-default-85c8575b7f-blhgz\" (UID: \"a9dcab5f-6209-4edd-8832-d31608f3c7d3\") " pod="openshift-ingress/e2e-shared-b-aa7c3f-openshift-default-85c8575b7f-blhgz"
Jul 06 14:54:41.694443 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:41.694375    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a9dcab5f-6209-4edd-8832-d31608f3c7d3-credential-socket\") pod \"e2e-shared-b-aa7c3f-openshift-default-85c8575b7f-blhgz\" (UID: \"a9dcab5f-6209-4edd-8832-d31608f3c7d3\") " pod="openshift-ingress/e2e-shared-b-aa7c3f-openshift-default-85c8575b7f-blhgz"
Jul 06 14:54:41.694711 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:41.694428    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a9dcab5f-6209-4edd-8832-d31608f3c7d3-workload-socket\") pod \"e2e-shared-b-aa7c3f-openshift-default-85c8575b7f-blhgz\" (UID: \"a9dcab5f-6209-4edd-8832-d31608f3c7d3\") " pod="openshift-ingress/e2e-shared-b-aa7c3f-openshift-default-85c8575b7f-blhgz"
Jul 06 14:54:41.694711 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:41.694523    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a9dcab5f-6209-4edd-8832-d31608f3c7d3-workload-certs\") pod \"e2e-shared-b-aa7c3f-openshift-default-85c8575b7f-blhgz\" (UID: \"a9dcab5f-6209-4edd-8832-d31608f3c7d3\") " pod="openshift-ingress/e2e-shared-b-aa7c3f-openshift-default-85c8575b7f-blhgz"
Jul 06 14:54:41.694711 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:41.694553    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a9dcab5f-6209-4edd-8832-d31608f3c7d3-istio-data\") pod \"e2e-shared-b-aa7c3f-openshift-default-85c8575b7f-blhgz\" (UID: \"a9dcab5f-6209-4edd-8832-d31608f3c7d3\") " pod="openshift-ingress/e2e-shared-b-aa7c3f-openshift-default-85c8575b7f-blhgz"
Jul 06 14:54:41.694875 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:41.694855    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a9dcab5f-6209-4edd-8832-d31608f3c7d3-istiod-ca-cert\") pod \"e2e-shared-b-aa7c3f-openshift-default-85c8575b7f-blhgz\" (UID: \"a9dcab5f-6209-4edd-8832-d31608f3c7d3\") " pod="openshift-ingress/e2e-shared-b-aa7c3f-openshift-default-85c8575b7f-blhgz"
Jul 06 14:54:41.696463 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:41.696445    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a9dcab5f-6209-4edd-8832-d31608f3c7d3-istio-envoy\") pod \"e2e-shared-b-aa7c3f-openshift-default-85c8575b7f-blhgz\" (UID: \"a9dcab5f-6209-4edd-8832-d31608f3c7d3\") " pod="openshift-ingress/e2e-shared-b-aa7c3f-openshift-default-85c8575b7f-blhgz"
Jul 06 14:54:41.696554 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:41.696538    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a9dcab5f-6209-4edd-8832-d31608f3c7d3-istio-podinfo\") pod \"e2e-shared-b-aa7c3f-openshift-default-85c8575b7f-blhgz\" (UID: \"a9dcab5f-6209-4edd-8832-d31608f3c7d3\") " pod="openshift-ingress/e2e-shared-b-aa7c3f-openshift-default-85c8575b7f-blhgz"
Jul 06 14:54:41.703012 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:41.702980    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g48nr\" (UniqueName: \"kubernetes.io/projected/a9dcab5f-6209-4edd-8832-d31608f3c7d3-kube-api-access-g48nr\") pod \"e2e-shared-b-aa7c3f-openshift-default-85c8575b7f-blhgz\" (UID: \"a9dcab5f-6209-4edd-8832-d31608f3c7d3\") " pod="openshift-ingress/e2e-shared-b-aa7c3f-openshift-default-85c8575b7f-blhgz"
Jul 06 14:54:41.703101 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:41.703073    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a9dcab5f-6209-4edd-8832-d31608f3c7d3-istio-token\") pod \"e2e-shared-b-aa7c3f-openshift-default-85c8575b7f-blhgz\" (UID: \"a9dcab5f-6209-4edd-8832-d31608f3c7d3\") " pod="openshift-ingress/e2e-shared-b-aa7c3f-openshift-default-85c8575b7f-blhgz"
Jul 06 14:54:41.799464 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:41.799365    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-aa7c3f-openshift-default-85c8575b7f-blhgz"
Jul 06 14:54:41.944625 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:41.944599    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-aa7c3f-openshift-default-85c8575b7f-blhgz"]
Jul 06 14:54:41.946327 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:54:41.946298    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9dcab5f_6209_4edd_8832_d31608f3c7d3.slice/crio-9c19490f836cbf4af74d8ae9397e5f3c1034d46d5a4feee526486c253e81afd4 WatchSource:0}: Error finding container 9c19490f836cbf4af74d8ae9397e5f3c1034d46d5a4feee526486c253e81afd4: Status 404 returned error can't find the container with id 9c19490f836cbf4af74d8ae9397e5f3c1034d46d5a4feee526486c253e81afd4
Jul 06 14:54:41.948483 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:41.948452    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 06 14:54:41.948599 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:41.948524    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 06 14:54:41.948599 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:41.948583    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 06 14:54:42.301662 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:42.301628    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-aa7c3f-openshift-default-85c8575b7f-blhgz" event={"ID":"a9dcab5f-6209-4edd-8832-d31608f3c7d3","Type":"ContainerStarted","Data":"fec53383d2a08745763e163fbbf3efa194634a1b04935068c2fa60bdef007f62"}
Jul 06 14:54:42.301833 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:42.301682    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-aa7c3f-openshift-default-85c8575b7f-blhgz" event={"ID":"a9dcab5f-6209-4edd-8832-d31608f3c7d3","Type":"ContainerStarted","Data":"9c19490f836cbf4af74d8ae9397e5f3c1034d46d5a4feee526486c253e81afd4"}
Jul 06 14:54:42.324239 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:42.324181    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-aa7c3f-openshift-default-85c8575b7f-blhgz" podStartSLOduration=1.32416362 podStartE2EDuration="1.32416362s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:54:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:54:42.322692836 +0000 UTC m=+2104.629984921" watchObservedRunningTime="2026-07-06 14:54:42.32416362 +0000 UTC m=+2104.631455701"
Jul 06 14:54:42.799708 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:42.799666    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-aa7c3f-openshift-default-85c8575b7f-blhgz"
Jul 06 14:54:43.806974 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:43.806932    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-aa7c3f-openshift-default-85c8575b7f-blhgz"
Jul 06 14:54:44.297191 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:44.297156    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-shared-a-bb4316-6d6d64dddd-24qw7"
Jul 06 14:54:44.316584 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:44.316535    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-aa7c3f-openshift-default-85c8575b7f-blhgz"
Jul 06 14:54:44.317685 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:44.317661    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-aa7c3f-openshift-default-85c8575b7f-blhgz"
Jul 06 14:54:44.463801 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:44.463765    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-shared-b-aa7c3f-547545474d-qx2n7"]
Jul 06 14:54:44.467793 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:44.467770    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-aa7c3f-547545474d-qx2n7"
Jul 06 14:54:44.470448 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:44.470420    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-shared-b-aa7c3f-serving-cert\""
Jul 06 14:54:44.511715 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:44.511673    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-aa7c3f-547545474d-qx2n7"]
Jul 06 14:54:44.524496 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:44.524464    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wj22h\" (UniqueName: \"kubernetes.io/projected/d0b274dc-94ae-4af7-b562-8929db8b7ac3-kube-api-access-wj22h\") pod \"maas-api-e2e-shared-b-aa7c3f-547545474d-qx2n7\" (UID: \"d0b274dc-94ae-4af7-b562-8929db8b7ac3\") " pod="opendatahub/maas-api-e2e-shared-b-aa7c3f-547545474d-qx2n7"
Jul 06 14:54:44.524696 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:44.524512    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d0b274dc-94ae-4af7-b562-8929db8b7ac3-maas-api-tls\") pod \"maas-api-e2e-shared-b-aa7c3f-547545474d-qx2n7\" (UID: \"d0b274dc-94ae-4af7-b562-8929db8b7ac3\") " pod="opendatahub/maas-api-e2e-shared-b-aa7c3f-547545474d-qx2n7"
Jul 06 14:54:44.524696 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:44.524663    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d0b274dc-94ae-4af7-b562-8929db8b7ac3-openshift-service-ca\") pod \"maas-api-e2e-shared-b-aa7c3f-547545474d-qx2n7\" (UID: \"d0b274dc-94ae-4af7-b562-8929db8b7ac3\") " pod="opendatahub/maas-api-e2e-shared-b-aa7c3f-547545474d-qx2n7"
Jul 06 14:54:44.625641 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:44.625527    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wj22h\" (UniqueName: \"kubernetes.io/projected/d0b274dc-94ae-4af7-b562-8929db8b7ac3-kube-api-access-wj22h\") pod \"maas-api-e2e-shared-b-aa7c3f-547545474d-qx2n7\" (UID: \"d0b274dc-94ae-4af7-b562-8929db8b7ac3\") " pod="opendatahub/maas-api-e2e-shared-b-aa7c3f-547545474d-qx2n7"
Jul 06 14:54:44.625641 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:44.625588    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d0b274dc-94ae-4af7-b562-8929db8b7ac3-maas-api-tls\") pod \"maas-api-e2e-shared-b-aa7c3f-547545474d-qx2n7\" (UID: \"d0b274dc-94ae-4af7-b562-8929db8b7ac3\") " pod="opendatahub/maas-api-e2e-shared-b-aa7c3f-547545474d-qx2n7"
Jul 06 14:54:44.625830 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:44.625660    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d0b274dc-94ae-4af7-b562-8929db8b7ac3-openshift-service-ca\") pod \"maas-api-e2e-shared-b-aa7c3f-547545474d-qx2n7\" (UID: \"d0b274dc-94ae-4af7-b562-8929db8b7ac3\") " pod="opendatahub/maas-api-e2e-shared-b-aa7c3f-547545474d-qx2n7"
Jul 06 14:54:44.625830 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:54:44.625761    2559 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-shared-b-aa7c3f-serving-cert: secret "maas-api-e2e-shared-b-aa7c3f-serving-cert" not found
Jul 06 14:54:44.625908 ip-10-0-133-112 kubenswrapper[2559]: E0706 14:54:44.625846    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d0b274dc-94ae-4af7-b562-8929db8b7ac3-maas-api-tls podName:d0b274dc-94ae-4af7-b562-8929db8b7ac3 nodeName:}" failed. No retries permitted until 2026-07-06 14:54:45.125824898 +0000 UTC m=+2107.433116969 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/d0b274dc-94ae-4af7-b562-8929db8b7ac3-maas-api-tls") pod "maas-api-e2e-shared-b-aa7c3f-547545474d-qx2n7" (UID: "d0b274dc-94ae-4af7-b562-8929db8b7ac3") : secret "maas-api-e2e-shared-b-aa7c3f-serving-cert" not found
Jul 06 14:54:44.626342 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:44.626322    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d0b274dc-94ae-4af7-b562-8929db8b7ac3-openshift-service-ca\") pod \"maas-api-e2e-shared-b-aa7c3f-547545474d-qx2n7\" (UID: \"d0b274dc-94ae-4af7-b562-8929db8b7ac3\") " pod="opendatahub/maas-api-e2e-shared-b-aa7c3f-547545474d-qx2n7"
Jul 06 14:54:44.635314 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:44.635287    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wj22h\" (UniqueName: \"kubernetes.io/projected/d0b274dc-94ae-4af7-b562-8929db8b7ac3-kube-api-access-wj22h\") pod \"maas-api-e2e-shared-b-aa7c3f-547545474d-qx2n7\" (UID: \"d0b274dc-94ae-4af7-b562-8929db8b7ac3\") " pod="opendatahub/maas-api-e2e-shared-b-aa7c3f-547545474d-qx2n7"
Jul 06 14:54:45.130641 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:45.130605    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d0b274dc-94ae-4af7-b562-8929db8b7ac3-maas-api-tls\") pod \"maas-api-e2e-shared-b-aa7c3f-547545474d-qx2n7\" (UID: \"d0b274dc-94ae-4af7-b562-8929db8b7ac3\") " pod="opendatahub/maas-api-e2e-shared-b-aa7c3f-547545474d-qx2n7"
Jul 06 14:54:45.133031 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:45.133004    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d0b274dc-94ae-4af7-b562-8929db8b7ac3-maas-api-tls\") pod \"maas-api-e2e-shared-b-aa7c3f-547545474d-qx2n7\" (UID: \"d0b274dc-94ae-4af7-b562-8929db8b7ac3\") " pod="opendatahub/maas-api-e2e-shared-b-aa7c3f-547545474d-qx2n7"
Jul 06 14:54:45.383255 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:45.383160    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-aa7c3f-547545474d-qx2n7"
Jul 06 14:54:45.565799 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:45.565763    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-aa7c3f-547545474d-qx2n7"]
Jul 06 14:54:45.569158 ip-10-0-133-112 kubenswrapper[2559]: W0706 14:54:45.569128    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0b274dc_94ae_4af7_b562_8929db8b7ac3.slice/crio-a1c9deb75dae9377704837158318533ecccc3aeb53388485f55e24603d67abe2 WatchSource:0}: Error finding container a1c9deb75dae9377704837158318533ecccc3aeb53388485f55e24603d67abe2: Status 404 returned error can't find the container with id a1c9deb75dae9377704837158318533ecccc3aeb53388485f55e24603d67abe2
Jul 06 14:54:46.328581 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:46.328527    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-aa7c3f-547545474d-qx2n7" event={"ID":"d0b274dc-94ae-4af7-b562-8929db8b7ac3","Type":"ContainerStarted","Data":"743dbd5f35192a558f3d992ea8af2e90c424b0642cb01f773ff305bb5f6c0ad7"}
Jul 06 14:54:46.329070 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:46.328619    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-aa7c3f-547545474d-qx2n7" event={"ID":"d0b274dc-94ae-4af7-b562-8929db8b7ac3","Type":"ContainerStarted","Data":"a1c9deb75dae9377704837158318533ecccc3aeb53388485f55e24603d67abe2"}
Jul 06 14:54:46.350589 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:46.350521    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-shared-b-aa7c3f-547545474d-qx2n7" podStartSLOduration=1.812936234 podStartE2EDuration="2.350504403s" totalImagesPullingTime="537.568169ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:54:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:54:46.348597232 +0000 UTC m=+2108.655889312" watchObservedRunningTime="2026-07-06 14:54:46.350504403 +0000 UTC m=+2108.657796485"
Jul 06 14:54:47.334277 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:47.334245    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-shared-b-aa7c3f-547545474d-qx2n7"
Jul 06 14:54:50.261303 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:50.261269    2559 patch_prober.go:28] interesting pod/e2e-shared-a-bb4316-openshift-default-79cd6875-cjg58 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body=
Jul 06 14:54:50.261786 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:50.261325    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-bb4316-openshift-default-79cd6875-cjg58" podUID="8b9bfd85-5b75-4113-b0f6-656bd17770d4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused"
Jul 06 14:54:50.350898 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:50.350866    2559 generic.go:356] "Generic (PLEG): container finished" podID="8b9bfd85-5b75-4113-b0f6-656bd17770d4" containerID="5bd2442ff3ca80f9d4e4851d365d661526102cd2689080444a0af3de63188a12" exitCode=0
Jul 06 14:54:50.351089 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:50.350948    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-bb4316-openshift-default-79cd6875-cjg58" event={"ID":"8b9bfd85-5b75-4113-b0f6-656bd17770d4","Type":"ContainerDied","Data":"5bd2442ff3ca80f9d4e4851d365d661526102cd2689080444a0af3de63188a12"}
Jul 06 14:54:50.351385 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:50.351373    2559 scope.go:117] "RemoveContainer" containerID="5bd2442ff3ca80f9d4e4851d365d661526102cd2689080444a0af3de63188a12"
Jul 06 14:54:50.352520 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:50.352487    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 06 14:54:50.352620 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:50.352604    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 06 14:54:50.352664 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:50.352646    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 06 14:54:51.362155 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:51.362109    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-bb4316-openshift-default-79cd6875-cjg58" event={"ID":"8b9bfd85-5b75-4113-b0f6-656bd17770d4","Type":"ContainerStarted","Data":"fcd913d46799a0a04b1831b26d2b0dbb2584fcb53ec4cb8720b67c26eb79022e"}
Jul 06 14:54:51.512534 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:51.512488    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-bb4316-openshift-default-79cd6875-cjg58"
Jul 06 14:54:51.512773 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:51.512643    2559 patch_prober.go:28] interesting pod/e2e-shared-a-bb4316-openshift-default-79cd6875-cjg58 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body=
Jul 06 14:54:51.512773 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:51.512692    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-bb4316-openshift-default-79cd6875-cjg58" podUID="8b9bfd85-5b75-4113-b0f6-656bd17770d4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused"
Jul 06 14:54:52.512108 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:52.512070    2559 patch_prober.go:28] interesting pod/e2e-shared-a-bb4316-openshift-default-79cd6875-cjg58 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body=
Jul 06 14:54:52.513081 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:52.512680    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-bb4316-openshift-default-79cd6875-cjg58" podUID="8b9bfd85-5b75-4113-b0f6-656bd17770d4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused"
Jul 06 14:54:53.345222 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:53.345186    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-shared-b-aa7c3f-547545474d-qx2n7"
Jul 06 14:54:53.518594 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:53.517987    2559 patch_prober.go:28] interesting pod/e2e-shared-a-bb4316-openshift-default-79cd6875-cjg58 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body=
Jul 06 14:54:53.518594 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:53.518051    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-bb4316-openshift-default-79cd6875-cjg58" podUID="8b9bfd85-5b75-4113-b0f6-656bd17770d4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused"
Jul 06 14:54:54.512561 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:54.512517    2559 patch_prober.go:28] interesting pod/e2e-shared-a-bb4316-openshift-default-79cd6875-cjg58 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body=
Jul 06 14:54:54.512905 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:54.512870    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-bb4316-openshift-default-79cd6875-cjg58" podUID="8b9bfd85-5b75-4113-b0f6-656bd17770d4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused"
Jul 06 14:54:55.511971 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:55.511925    2559 patch_prober.go:28] interesting pod/e2e-shared-a-bb4316-openshift-default-79cd6875-cjg58 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body=
Jul 06 14:54:55.512374 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:55.511998    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-bb4316-openshift-default-79cd6875-cjg58" podUID="8b9bfd85-5b75-4113-b0f6-656bd17770d4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused"
Jul 06 14:54:56.517360 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:56.517327    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-bb4316-openshift-default-79cd6875-cjg58"
Jul 06 14:54:56.517772 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:56.517685    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-bb4316-openshift-default-79cd6875-cjg58"
Jul 06 14:54:56.518526 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:56.518504    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-bb4316-openshift-default-79cd6875-cjg58"
Jul 06 14:54:58.401821 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:58.401785    2559 generic.go:356] "Generic (PLEG): container finished" podID="a9dcab5f-6209-4edd-8832-d31608f3c7d3" containerID="fec53383d2a08745763e163fbbf3efa194634a1b04935068c2fa60bdef007f62" exitCode=0
Jul 06 14:54:58.402276 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:58.401859    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-aa7c3f-openshift-default-85c8575b7f-blhgz" event={"ID":"a9dcab5f-6209-4edd-8832-d31608f3c7d3","Type":"ContainerDied","Data":"fec53383d2a08745763e163fbbf3efa194634a1b04935068c2fa60bdef007f62"}
Jul 06 14:54:58.402346 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:58.402319    2559 scope.go:117] "RemoveContainer" containerID="fec53383d2a08745763e163fbbf3efa194634a1b04935068c2fa60bdef007f62"
Jul 06 14:54:58.403309 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:58.403280    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 06 14:54:58.403369 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:58.403351    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 06 14:54:58.403405 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:58.403385    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 06 14:54:59.316717 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:59.316681    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-aa7c3f-openshift-default-85c8575b7f-blhgz"
Jul 06 14:54:59.410120 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:59.410083    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-aa7c3f-openshift-default-85c8575b7f-blhgz" event={"ID":"a9dcab5f-6209-4edd-8832-d31608f3c7d3","Type":"ContainerStarted","Data":"f72cf7a3a7dea6534cb4de212b96dfbf801a15a4849eba159c3a5be9153a931a"}
Jul 06 14:54:59.801788 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:59.801741    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-aa7c3f-openshift-default-85c8575b7f-blhgz"
Jul 06 14:54:59.801989 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:59.801925    2559 patch_prober.go:28] interesting pod/e2e-shared-b-aa7c3f-openshift-default-85c8575b7f-blhgz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body=
Jul 06 14:54:59.801989 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:54:59.801973    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-aa7c3f-openshift-default-85c8575b7f-blhgz" podUID="a9dcab5f-6209-4edd-8832-d31608f3c7d3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused"
Jul 06 14:55:00.800876 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:55:00.800832    2559 patch_prober.go:28] interesting pod/e2e-shared-b-aa7c3f-openshift-default-85c8575b7f-blhgz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body=
Jul 06 14:55:00.801352 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:55:00.800908    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-aa7c3f-openshift-default-85c8575b7f-blhgz" podUID="a9dcab5f-6209-4edd-8832-d31608f3c7d3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused"
Jul 06 14:55:01.800412 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:55:01.800365    2559 patch_prober.go:28] interesting pod/e2e-shared-b-aa7c3f-openshift-default-85c8575b7f-blhgz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body=
Jul 06 14:55:01.800662 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:55:01.800442    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-aa7c3f-openshift-default-85c8575b7f-blhgz" podUID="a9dcab5f-6209-4edd-8832-d31608f3c7d3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused"
Jul 06 14:55:02.801025 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:55:02.800505    2559 patch_prober.go:28] interesting pod/e2e-shared-b-aa7c3f-openshift-default-85c8575b7f-blhgz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body=
Jul 06 14:55:02.801025 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:55:02.800603    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-aa7c3f-openshift-default-85c8575b7f-blhgz" podUID="a9dcab5f-6209-4edd-8832-d31608f3c7d3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused"
Jul 06 14:55:03.804055 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:55:03.804011    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-aa7c3f-openshift-default-85c8575b7f-blhgz"
Jul 06 14:55:03.804515 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:55:03.804267    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-aa7c3f-openshift-default-85c8575b7f-blhgz"
Jul 06 14:55:03.805040 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:55:03.805021    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-aa7c3f-openshift-default-85c8575b7f-blhgz"
Jul 06 14:59:38.324987 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:59:38.324948    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-g5w5m_e35b0247-7c4e-4bd1-bc28-c084dc40ffa1/console-operator/1.log"
Jul 06 14:59:38.331839 ip-10-0-133-112 kubenswrapper[2559]: I0706 14:59:38.331811    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-g5w5m_e35b0247-7c4e-4bd1-bc28-c084dc40ffa1/console-operator/1.log"
Jul 06 15:00:26.081102 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:00:26.080956    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-bb4316-6d6d64dddd-24qw7"]
Jul 06 15:00:26.083974 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:00:26.081793    2559 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-shared-a-bb4316-6d6d64dddd-24qw7" podUID="9f4c88f0-8bd9-43a1-a648-a312759fe26d" containerName="maas-api" containerID="cri-o://e91a8582f157dd3559601ca2335c18fc2da998dd6ff6fdad58dfde063f5dab6f" gracePeriod=30
Jul 06 15:00:26.345613 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:00:26.345532    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-bb4316-6d6d64dddd-24qw7"
Jul 06 15:00:26.454216 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:00:26.454176    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9f4c88f0-8bd9-43a1-a648-a312759fe26d-openshift-service-ca\") pod \"9f4c88f0-8bd9-43a1-a648-a312759fe26d\" (UID: \"9f4c88f0-8bd9-43a1-a648-a312759fe26d\") "
Jul 06 15:00:26.454216 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:00:26.454230    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rdgcp\" (UniqueName: \"kubernetes.io/projected/9f4c88f0-8bd9-43a1-a648-a312759fe26d-kube-api-access-rdgcp\") pod \"9f4c88f0-8bd9-43a1-a648-a312759fe26d\" (UID: \"9f4c88f0-8bd9-43a1-a648-a312759fe26d\") "
Jul 06 15:00:26.454483 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:00:26.454372    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9f4c88f0-8bd9-43a1-a648-a312759fe26d-maas-api-tls\") pod \"9f4c88f0-8bd9-43a1-a648-a312759fe26d\" (UID: \"9f4c88f0-8bd9-43a1-a648-a312759fe26d\") "
Jul 06 15:00:26.454679 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:00:26.454647    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9f4c88f0-8bd9-43a1-a648-a312759fe26d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "9f4c88f0-8bd9-43a1-a648-a312759fe26d" (UID: "9f4c88f0-8bd9-43a1-a648-a312759fe26d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 15:00:26.456376 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:00:26.456344    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9f4c88f0-8bd9-43a1-a648-a312759fe26d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "9f4c88f0-8bd9-43a1-a648-a312759fe26d" (UID: "9f4c88f0-8bd9-43a1-a648-a312759fe26d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 15:00:26.456480 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:00:26.456374    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9f4c88f0-8bd9-43a1-a648-a312759fe26d-kube-api-access-rdgcp" (OuterVolumeSpecName: "kube-api-access-rdgcp") pod "9f4c88f0-8bd9-43a1-a648-a312759fe26d" (UID: "9f4c88f0-8bd9-43a1-a648-a312759fe26d"). InnerVolumeSpecName "kube-api-access-rdgcp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 15:00:26.555281 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:00:26.555240    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9f4c88f0-8bd9-43a1-a648-a312759fe26d-maas-api-tls\") on node \"ip-10-0-133-112.ec2.internal\" DevicePath \"\""
Jul 06 15:00:26.555281 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:00:26.555276    2559 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9f4c88f0-8bd9-43a1-a648-a312759fe26d-openshift-service-ca\") on node \"ip-10-0-133-112.ec2.internal\" DevicePath \"\""
Jul 06 15:00:26.555281 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:00:26.555291    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rdgcp\" (UniqueName: \"kubernetes.io/projected/9f4c88f0-8bd9-43a1-a648-a312759fe26d-kube-api-access-rdgcp\") on node \"ip-10-0-133-112.ec2.internal\" DevicePath \"\""
Jul 06 15:00:27.063052 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:00:27.063014    2559 generic.go:356] "Generic (PLEG): container finished" podID="9f4c88f0-8bd9-43a1-a648-a312759fe26d" containerID="e91a8582f157dd3559601ca2335c18fc2da998dd6ff6fdad58dfde063f5dab6f" exitCode=0
Jul 06 15:00:27.063052 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:00:27.063055    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-bb4316-6d6d64dddd-24qw7"
Jul 06 15:00:27.063261 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:00:27.063104    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-bb4316-6d6d64dddd-24qw7" event={"ID":"9f4c88f0-8bd9-43a1-a648-a312759fe26d","Type":"ContainerDied","Data":"e91a8582f157dd3559601ca2335c18fc2da998dd6ff6fdad58dfde063f5dab6f"}
Jul 06 15:00:27.063261 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:00:27.063158    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-bb4316-6d6d64dddd-24qw7" event={"ID":"9f4c88f0-8bd9-43a1-a648-a312759fe26d","Type":"ContainerDied","Data":"ec9ae4e1692c0f53e866b5a43bc5940ec6dcb23767f53632ce0260255b71ba6b"}
Jul 06 15:00:27.063261 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:00:27.063187    2559 scope.go:117] "RemoveContainer" containerID="e91a8582f157dd3559601ca2335c18fc2da998dd6ff6fdad58dfde063f5dab6f"
Jul 06 15:00:27.072890 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:00:27.072871    2559 scope.go:117] "RemoveContainer" containerID="e91a8582f157dd3559601ca2335c18fc2da998dd6ff6fdad58dfde063f5dab6f"
Jul 06 15:00:27.073153 ip-10-0-133-112 kubenswrapper[2559]: E0706 15:00:27.073134    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e91a8582f157dd3559601ca2335c18fc2da998dd6ff6fdad58dfde063f5dab6f\": container with ID starting with e91a8582f157dd3559601ca2335c18fc2da998dd6ff6fdad58dfde063f5dab6f not found: ID does not exist" containerID="e91a8582f157dd3559601ca2335c18fc2da998dd6ff6fdad58dfde063f5dab6f"
Jul 06 15:00:27.073199 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:00:27.073162    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e91a8582f157dd3559601ca2335c18fc2da998dd6ff6fdad58dfde063f5dab6f"} err="failed to get container status \"e91a8582f157dd3559601ca2335c18fc2da998dd6ff6fdad58dfde063f5dab6f\": rpc error: code = NotFound desc = could not find container \"e91a8582f157dd3559601ca2335c18fc2da998dd6ff6fdad58dfde063f5dab6f\": container with ID starting with e91a8582f157dd3559601ca2335c18fc2da998dd6ff6fdad58dfde063f5dab6f not found: ID does not exist"
Jul 06 15:00:27.085000 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:00:27.084973    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-bb4316-6d6d64dddd-24qw7"]
Jul 06 15:00:27.089620 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:00:27.089598    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-bb4316-6d6d64dddd-24qw7"]
Jul 06 15:00:28.227310 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:00:28.227276    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9f4c88f0-8bd9-43a1-a648-a312759fe26d" path="/var/lib/kubelet/pods/9f4c88f0-8bd9-43a1-a648-a312759fe26d/volumes"
Jul 06 15:01:08.600839 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:08.600798    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-bb4316-openshift-default-79cd6875-cjg58"]
Jul 06 15:01:08.601505 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:08.601028    2559 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-bb4316-openshift-default-79cd6875-cjg58" podUID="8b9bfd85-5b75-4113-b0f6-656bd17770d4" containerName="istio-proxy" containerID="cri-o://fcd913d46799a0a04b1831b26d2b0dbb2584fcb53ec4cb8720b67c26eb79022e" gracePeriod=30
Jul 06 15:01:09.717650 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:09.717305    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-aa7c3f-547545474d-qx2n7"]
Jul 06 15:01:09.717650 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:09.717610    2559 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-shared-b-aa7c3f-547545474d-qx2n7" podUID="d0b274dc-94ae-4af7-b562-8929db8b7ac3" containerName="maas-api" containerID="cri-o://743dbd5f35192a558f3d992ea8af2e90c424b0642cb01f773ff305bb5f6c0ad7" gracePeriod=30
Jul 06 15:01:09.982678 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:09.982600    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-aa7c3f-547545474d-qx2n7"
Jul 06 15:01:10.072713 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:10.072669    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wj22h\" (UniqueName: \"kubernetes.io/projected/d0b274dc-94ae-4af7-b562-8929db8b7ac3-kube-api-access-wj22h\") pod \"d0b274dc-94ae-4af7-b562-8929db8b7ac3\" (UID: \"d0b274dc-94ae-4af7-b562-8929db8b7ac3\") "
Jul 06 15:01:10.072713 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:10.072722    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d0b274dc-94ae-4af7-b562-8929db8b7ac3-maas-api-tls\") pod \"d0b274dc-94ae-4af7-b562-8929db8b7ac3\" (UID: \"d0b274dc-94ae-4af7-b562-8929db8b7ac3\") "
Jul 06 15:01:10.072983 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:10.072848    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d0b274dc-94ae-4af7-b562-8929db8b7ac3-openshift-service-ca\") pod \"d0b274dc-94ae-4af7-b562-8929db8b7ac3\" (UID: \"d0b274dc-94ae-4af7-b562-8929db8b7ac3\") "
Jul 06 15:01:10.073200 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:10.073176    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d0b274dc-94ae-4af7-b562-8929db8b7ac3-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d0b274dc-94ae-4af7-b562-8929db8b7ac3" (UID: "d0b274dc-94ae-4af7-b562-8929db8b7ac3"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 15:01:10.074880 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:10.074851    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d0b274dc-94ae-4af7-b562-8929db8b7ac3-kube-api-access-wj22h" (OuterVolumeSpecName: "kube-api-access-wj22h") pod "d0b274dc-94ae-4af7-b562-8929db8b7ac3" (UID: "d0b274dc-94ae-4af7-b562-8929db8b7ac3"). InnerVolumeSpecName "kube-api-access-wj22h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 15:01:10.074999 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:10.074938    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d0b274dc-94ae-4af7-b562-8929db8b7ac3-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d0b274dc-94ae-4af7-b562-8929db8b7ac3" (UID: "d0b274dc-94ae-4af7-b562-8929db8b7ac3"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 15:01:10.173805 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:10.173766    2559 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d0b274dc-94ae-4af7-b562-8929db8b7ac3-openshift-service-ca\") on node \"ip-10-0-133-112.ec2.internal\" DevicePath \"\""
Jul 06 15:01:10.173805 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:10.173796    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wj22h\" (UniqueName: \"kubernetes.io/projected/d0b274dc-94ae-4af7-b562-8929db8b7ac3-kube-api-access-wj22h\") on node \"ip-10-0-133-112.ec2.internal\" DevicePath \"\""
Jul 06 15:01:10.173805 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:10.173807    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d0b274dc-94ae-4af7-b562-8929db8b7ac3-maas-api-tls\") on node \"ip-10-0-133-112.ec2.internal\" DevicePath \"\""
Jul 06 15:01:10.297219 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:10.297183    2559 generic.go:356] "Generic (PLEG): container finished" podID="d0b274dc-94ae-4af7-b562-8929db8b7ac3" containerID="743dbd5f35192a558f3d992ea8af2e90c424b0642cb01f773ff305bb5f6c0ad7" exitCode=0
Jul 06 15:01:10.297429 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:10.297228    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-aa7c3f-547545474d-qx2n7"
Jul 06 15:01:10.297429 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:10.297239    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-aa7c3f-547545474d-qx2n7" event={"ID":"d0b274dc-94ae-4af7-b562-8929db8b7ac3","Type":"ContainerDied","Data":"743dbd5f35192a558f3d992ea8af2e90c424b0642cb01f773ff305bb5f6c0ad7"}
Jul 06 15:01:10.297429 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:10.297282    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-aa7c3f-547545474d-qx2n7" event={"ID":"d0b274dc-94ae-4af7-b562-8929db8b7ac3","Type":"ContainerDied","Data":"a1c9deb75dae9377704837158318533ecccc3aeb53388485f55e24603d67abe2"}
Jul 06 15:01:10.297429 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:10.297303    2559 scope.go:117] "RemoveContainer" containerID="743dbd5f35192a558f3d992ea8af2e90c424b0642cb01f773ff305bb5f6c0ad7"
Jul 06 15:01:10.306591 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:10.306553    2559 scope.go:117] "RemoveContainer" containerID="743dbd5f35192a558f3d992ea8af2e90c424b0642cb01f773ff305bb5f6c0ad7"
Jul 06 15:01:10.306944 ip-10-0-133-112 kubenswrapper[2559]: E0706 15:01:10.306921    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"743dbd5f35192a558f3d992ea8af2e90c424b0642cb01f773ff305bb5f6c0ad7\": container with ID starting with 743dbd5f35192a558f3d992ea8af2e90c424b0642cb01f773ff305bb5f6c0ad7 not found: ID does not exist" containerID="743dbd5f35192a558f3d992ea8af2e90c424b0642cb01f773ff305bb5f6c0ad7"
Jul 06 15:01:10.307029 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:10.306954    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"743dbd5f35192a558f3d992ea8af2e90c424b0642cb01f773ff305bb5f6c0ad7"} err="failed to get container status \"743dbd5f35192a558f3d992ea8af2e90c424b0642cb01f773ff305bb5f6c0ad7\": rpc error: code = NotFound desc = could not find container \"743dbd5f35192a558f3d992ea8af2e90c424b0642cb01f773ff305bb5f6c0ad7\": container with ID starting with 743dbd5f35192a558f3d992ea8af2e90c424b0642cb01f773ff305bb5f6c0ad7 not found: ID does not exist"
Jul 06 15:01:10.318144 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:10.318103    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-aa7c3f-547545474d-qx2n7"]
Jul 06 15:01:10.322966 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:10.322929    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-aa7c3f-547545474d-qx2n7"]
Jul 06 15:01:11.520799 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:11.520762    2559 patch_prober.go:28] interesting pod/e2e-shared-a-bb4316-openshift-default-79cd6875-cjg58 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 06 15:01:11.521201 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:11.520826    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-bb4316-openshift-default-79cd6875-cjg58" podUID="8b9bfd85-5b75-4113-b0f6-656bd17770d4" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 06 15:01:12.228450 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:12.228416    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d0b274dc-94ae-4af7-b562-8929db8b7ac3" path="/var/lib/kubelet/pods/d0b274dc-94ae-4af7-b562-8929db8b7ac3/volumes"
Jul 06 15:01:13.857043 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:13.857013    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-bb4316-openshift-default-79cd6875-cjg58"
Jul 06 15:01:14.012414 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:14.012378    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8b9bfd85-5b75-4113-b0f6-656bd17770d4-istiod-ca-cert\") pod \"8b9bfd85-5b75-4113-b0f6-656bd17770d4\" (UID: \"8b9bfd85-5b75-4113-b0f6-656bd17770d4\") "
Jul 06 15:01:14.012625 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:14.012436    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8b9bfd85-5b75-4113-b0f6-656bd17770d4-istio-token\") pod \"8b9bfd85-5b75-4113-b0f6-656bd17770d4\" (UID: \"8b9bfd85-5b75-4113-b0f6-656bd17770d4\") "
Jul 06 15:01:14.012625 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:14.012456    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8b9bfd85-5b75-4113-b0f6-656bd17770d4-istio-envoy\") pod \"8b9bfd85-5b75-4113-b0f6-656bd17770d4\" (UID: \"8b9bfd85-5b75-4113-b0f6-656bd17770d4\") "
Jul 06 15:01:14.012625 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:14.012480    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8b9bfd85-5b75-4113-b0f6-656bd17770d4-istio-data\") pod \"8b9bfd85-5b75-4113-b0f6-656bd17770d4\" (UID: \"8b9bfd85-5b75-4113-b0f6-656bd17770d4\") "
Jul 06 15:01:14.012625 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:14.012543    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b9bfd85-5b75-4113-b0f6-656bd17770d4-credential-socket\") pod \"8b9bfd85-5b75-4113-b0f6-656bd17770d4\" (UID: \"8b9bfd85-5b75-4113-b0f6-656bd17770d4\") "
Jul 06 15:01:14.012914 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:14.012633    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8b9bfd85-5b75-4113-b0f6-656bd17770d4-workload-certs\") pod \"8b9bfd85-5b75-4113-b0f6-656bd17770d4\" (UID: \"8b9bfd85-5b75-4113-b0f6-656bd17770d4\") "
Jul 06 15:01:14.012914 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:14.012671    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8b9bfd85-5b75-4113-b0f6-656bd17770d4-istio-podinfo\") pod \"8b9bfd85-5b75-4113-b0f6-656bd17770d4\" (UID: \"8b9bfd85-5b75-4113-b0f6-656bd17770d4\") "
Jul 06 15:01:14.012914 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:14.012706    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d99rz\" (UniqueName: \"kubernetes.io/projected/8b9bfd85-5b75-4113-b0f6-656bd17770d4-kube-api-access-d99rz\") pod \"8b9bfd85-5b75-4113-b0f6-656bd17770d4\" (UID: \"8b9bfd85-5b75-4113-b0f6-656bd17770d4\") "
Jul 06 15:01:14.012914 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:14.012759    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b9bfd85-5b75-4113-b0f6-656bd17770d4-workload-socket\") pod \"8b9bfd85-5b75-4113-b0f6-656bd17770d4\" (UID: \"8b9bfd85-5b75-4113-b0f6-656bd17770d4\") "
Jul 06 15:01:14.012914 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:14.012854    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8b9bfd85-5b75-4113-b0f6-656bd17770d4-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "8b9bfd85-5b75-4113-b0f6-656bd17770d4" (UID: "8b9bfd85-5b75-4113-b0f6-656bd17770d4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 15:01:14.012914 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:14.012857    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b9bfd85-5b75-4113-b0f6-656bd17770d4-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "8b9bfd85-5b75-4113-b0f6-656bd17770d4" (UID: "8b9bfd85-5b75-4113-b0f6-656bd17770d4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 15:01:14.013239 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:14.013096    2559 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b9bfd85-5b75-4113-b0f6-656bd17770d4-credential-socket\") on node \"ip-10-0-133-112.ec2.internal\" DevicePath \"\""
Jul 06 15:01:14.013239 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:14.013114    2559 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8b9bfd85-5b75-4113-b0f6-656bd17770d4-istiod-ca-cert\") on node \"ip-10-0-133-112.ec2.internal\" DevicePath \"\""
Jul 06 15:01:14.013239 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:14.013130    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b9bfd85-5b75-4113-b0f6-656bd17770d4-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "8b9bfd85-5b75-4113-b0f6-656bd17770d4" (UID: "8b9bfd85-5b75-4113-b0f6-656bd17770d4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 15:01:14.013383 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:14.013244    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b9bfd85-5b75-4113-b0f6-656bd17770d4-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "8b9bfd85-5b75-4113-b0f6-656bd17770d4" (UID: "8b9bfd85-5b75-4113-b0f6-656bd17770d4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 15:01:14.013493 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:14.013467    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b9bfd85-5b75-4113-b0f6-656bd17770d4-istio-data" (OuterVolumeSpecName: "istio-data") pod "8b9bfd85-5b75-4113-b0f6-656bd17770d4" (UID: "8b9bfd85-5b75-4113-b0f6-656bd17770d4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 15:01:14.014874 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:14.014846    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8b9bfd85-5b75-4113-b0f6-656bd17770d4-istio-token" (OuterVolumeSpecName: "istio-token") pod "8b9bfd85-5b75-4113-b0f6-656bd17770d4" (UID: "8b9bfd85-5b75-4113-b0f6-656bd17770d4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 15:01:14.015018 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:14.014862    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b9bfd85-5b75-4113-b0f6-656bd17770d4-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "8b9bfd85-5b75-4113-b0f6-656bd17770d4" (UID: "8b9bfd85-5b75-4113-b0f6-656bd17770d4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 15:01:14.015149 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:14.015131    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8b9bfd85-5b75-4113-b0f6-656bd17770d4-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "8b9bfd85-5b75-4113-b0f6-656bd17770d4" (UID: "8b9bfd85-5b75-4113-b0f6-656bd17770d4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 06 15:01:14.015259 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:14.015240    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8b9bfd85-5b75-4113-b0f6-656bd17770d4-kube-api-access-d99rz" (OuterVolumeSpecName: "kube-api-access-d99rz") pod "8b9bfd85-5b75-4113-b0f6-656bd17770d4" (UID: "8b9bfd85-5b75-4113-b0f6-656bd17770d4"). InnerVolumeSpecName "kube-api-access-d99rz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 15:01:14.114217 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:14.114181    2559 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8b9bfd85-5b75-4113-b0f6-656bd17770d4-workload-certs\") on node \"ip-10-0-133-112.ec2.internal\" DevicePath \"\""
Jul 06 15:01:14.114217 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:14.114212    2559 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8b9bfd85-5b75-4113-b0f6-656bd17770d4-istio-podinfo\") on node \"ip-10-0-133-112.ec2.internal\" DevicePath \"\""
Jul 06 15:01:14.114217 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:14.114223    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d99rz\" (UniqueName: \"kubernetes.io/projected/8b9bfd85-5b75-4113-b0f6-656bd17770d4-kube-api-access-d99rz\") on node \"ip-10-0-133-112.ec2.internal\" DevicePath \"\""
Jul 06 15:01:14.114473 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:14.114234    2559 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b9bfd85-5b75-4113-b0f6-656bd17770d4-workload-socket\") on node \"ip-10-0-133-112.ec2.internal\" DevicePath \"\""
Jul 06 15:01:14.114473 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:14.114244    2559 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8b9bfd85-5b75-4113-b0f6-656bd17770d4-istio-token\") on node \"ip-10-0-133-112.ec2.internal\" DevicePath \"\""
Jul 06 15:01:14.114473 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:14.114252    2559 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8b9bfd85-5b75-4113-b0f6-656bd17770d4-istio-envoy\") on node \"ip-10-0-133-112.ec2.internal\" DevicePath \"\""
Jul 06 15:01:14.114473 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:14.114260    2559 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8b9bfd85-5b75-4113-b0f6-656bd17770d4-istio-data\") on node \"ip-10-0-133-112.ec2.internal\" DevicePath \"\""
Jul 06 15:01:14.320408 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:14.320309    2559 generic.go:356] "Generic (PLEG): container finished" podID="8b9bfd85-5b75-4113-b0f6-656bd17770d4" containerID="fcd913d46799a0a04b1831b26d2b0dbb2584fcb53ec4cb8720b67c26eb79022e" exitCode=0
Jul 06 15:01:14.320408 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:14.320353    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-bb4316-openshift-default-79cd6875-cjg58"
Jul 06 15:01:14.320408 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:14.320393    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-bb4316-openshift-default-79cd6875-cjg58" event={"ID":"8b9bfd85-5b75-4113-b0f6-656bd17770d4","Type":"ContainerDied","Data":"fcd913d46799a0a04b1831b26d2b0dbb2584fcb53ec4cb8720b67c26eb79022e"}
Jul 06 15:01:14.320679 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:14.320451    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-bb4316-openshift-default-79cd6875-cjg58" event={"ID":"8b9bfd85-5b75-4113-b0f6-656bd17770d4","Type":"ContainerDied","Data":"638b851f37857089bc3c7052601913ea515aab81339de62b0e2e6a4634b1a3da"}
Jul 06 15:01:14.320679 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:14.320474    2559 scope.go:117] "RemoveContainer" containerID="fcd913d46799a0a04b1831b26d2b0dbb2584fcb53ec4cb8720b67c26eb79022e"
Jul 06 15:01:14.329254 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:14.329234    2559 scope.go:117] "RemoveContainer" containerID="5bd2442ff3ca80f9d4e4851d365d661526102cd2689080444a0af3de63188a12"
Jul 06 15:01:14.337636 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:14.337618    2559 scope.go:117] "RemoveContainer" containerID="fcd913d46799a0a04b1831b26d2b0dbb2584fcb53ec4cb8720b67c26eb79022e"
Jul 06 15:01:14.337909 ip-10-0-133-112 kubenswrapper[2559]: E0706 15:01:14.337887    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fcd913d46799a0a04b1831b26d2b0dbb2584fcb53ec4cb8720b67c26eb79022e\": container with ID starting with fcd913d46799a0a04b1831b26d2b0dbb2584fcb53ec4cb8720b67c26eb79022e not found: ID does not exist" containerID="fcd913d46799a0a04b1831b26d2b0dbb2584fcb53ec4cb8720b67c26eb79022e"
Jul 06 15:01:14.337981 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:14.337921    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fcd913d46799a0a04b1831b26d2b0dbb2584fcb53ec4cb8720b67c26eb79022e"} err="failed to get container status \"fcd913d46799a0a04b1831b26d2b0dbb2584fcb53ec4cb8720b67c26eb79022e\": rpc error: code = NotFound desc = could not find container \"fcd913d46799a0a04b1831b26d2b0dbb2584fcb53ec4cb8720b67c26eb79022e\": container with ID starting with fcd913d46799a0a04b1831b26d2b0dbb2584fcb53ec4cb8720b67c26eb79022e not found: ID does not exist"
Jul 06 15:01:14.337981 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:14.337948    2559 scope.go:117] "RemoveContainer" containerID="5bd2442ff3ca80f9d4e4851d365d661526102cd2689080444a0af3de63188a12"
Jul 06 15:01:14.338192 ip-10-0-133-112 kubenswrapper[2559]: E0706 15:01:14.338173    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5bd2442ff3ca80f9d4e4851d365d661526102cd2689080444a0af3de63188a12\": container with ID starting with 5bd2442ff3ca80f9d4e4851d365d661526102cd2689080444a0af3de63188a12 not found: ID does not exist" containerID="5bd2442ff3ca80f9d4e4851d365d661526102cd2689080444a0af3de63188a12"
Jul 06 15:01:14.338231 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:14.338198    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5bd2442ff3ca80f9d4e4851d365d661526102cd2689080444a0af3de63188a12"} err="failed to get container status \"5bd2442ff3ca80f9d4e4851d365d661526102cd2689080444a0af3de63188a12\": rpc error: code = NotFound desc = could not find container \"5bd2442ff3ca80f9d4e4851d365d661526102cd2689080444a0af3de63188a12\": container with ID starting with 5bd2442ff3ca80f9d4e4851d365d661526102cd2689080444a0af3de63188a12 not found: ID does not exist"
Jul 06 15:01:14.342885 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:14.342863    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-bb4316-openshift-default-79cd6875-cjg58"]
Jul 06 15:01:14.351104 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:14.351078    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-bb4316-openshift-default-79cd6875-cjg58"]
Jul 06 15:01:15.923480 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:15.923448    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-aa7c3f-openshift-default-85c8575b7f-blhgz"]
Jul 06 15:01:15.923994 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:15.923668    2559 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-aa7c3f-openshift-default-85c8575b7f-blhgz" podUID="a9dcab5f-6209-4edd-8832-d31608f3c7d3" containerName="istio-proxy" containerID="cri-o://f72cf7a3a7dea6534cb4de212b96dfbf801a15a4849eba159c3a5be9153a931a" gracePeriod=30
Jul 06 15:01:16.228133 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:16.228045    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8b9bfd85-5b75-4113-b0f6-656bd17770d4" path="/var/lib/kubelet/pods/8b9bfd85-5b75-4113-b0f6-656bd17770d4/volumes"
Jul 06 15:01:18.807313 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:18.807268    2559 patch_prober.go:28] interesting pod/e2e-shared-b-aa7c3f-openshift-default-85c8575b7f-blhgz container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 06 15:01:18.807734 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:18.807331    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-aa7c3f-openshift-default-85c8575b7f-blhgz" podUID="a9dcab5f-6209-4edd-8832-d31608f3c7d3" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 06 15:01:21.188328 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:21.188253    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-aa7c3f-openshift-default-85c8575b7f-blhgz"
Jul 06 15:01:21.284740 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:21.284705    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a9dcab5f-6209-4edd-8832-d31608f3c7d3-workload-socket\") pod \"a9dcab5f-6209-4edd-8832-d31608f3c7d3\" (UID: \"a9dcab5f-6209-4edd-8832-d31608f3c7d3\") "
Jul 06 15:01:21.284740 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:21.284752    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a9dcab5f-6209-4edd-8832-d31608f3c7d3-workload-certs\") pod \"a9dcab5f-6209-4edd-8832-d31608f3c7d3\" (UID: \"a9dcab5f-6209-4edd-8832-d31608f3c7d3\") "
Jul 06 15:01:21.285019 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:21.284787    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g48nr\" (UniqueName: \"kubernetes.io/projected/a9dcab5f-6209-4edd-8832-d31608f3c7d3-kube-api-access-g48nr\") pod \"a9dcab5f-6209-4edd-8832-d31608f3c7d3\" (UID: \"a9dcab5f-6209-4edd-8832-d31608f3c7d3\") "
Jul 06 15:01:21.285019 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:21.284801    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a9dcab5f-6209-4edd-8832-d31608f3c7d3-istio-token\") pod \"a9dcab5f-6209-4edd-8832-d31608f3c7d3\" (UID: \"a9dcab5f-6209-4edd-8832-d31608f3c7d3\") "
Jul 06 15:01:21.285019 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:21.284940    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a9dcab5f-6209-4edd-8832-d31608f3c7d3-istio-envoy\") pod \"a9dcab5f-6209-4edd-8832-d31608f3c7d3\" (UID: \"a9dcab5f-6209-4edd-8832-d31608f3c7d3\") "
Jul 06 15:01:21.285019 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:21.285003    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a9dcab5f-6209-4edd-8832-d31608f3c7d3-istio-data\") pod \"a9dcab5f-6209-4edd-8832-d31608f3c7d3\" (UID: \"a9dcab5f-6209-4edd-8832-d31608f3c7d3\") "
Jul 06 15:01:21.285231 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:21.285060    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a9dcab5f-6209-4edd-8832-d31608f3c7d3-istiod-ca-cert\") pod \"a9dcab5f-6209-4edd-8832-d31608f3c7d3\" (UID: \"a9dcab5f-6209-4edd-8832-d31608f3c7d3\") "
Jul 06 15:01:21.285231 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:21.285074    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a9dcab5f-6209-4edd-8832-d31608f3c7d3-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a9dcab5f-6209-4edd-8832-d31608f3c7d3" (UID: "a9dcab5f-6209-4edd-8832-d31608f3c7d3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 15:01:21.285231 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:21.285084    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a9dcab5f-6209-4edd-8832-d31608f3c7d3-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a9dcab5f-6209-4edd-8832-d31608f3c7d3" (UID: "a9dcab5f-6209-4edd-8832-d31608f3c7d3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 15:01:21.285231 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:21.285094    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a9dcab5f-6209-4edd-8832-d31608f3c7d3-istio-podinfo\") pod \"a9dcab5f-6209-4edd-8832-d31608f3c7d3\" (UID: \"a9dcab5f-6209-4edd-8832-d31608f3c7d3\") "
Jul 06 15:01:21.285451 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:21.285202    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a9dcab5f-6209-4edd-8832-d31608f3c7d3-credential-socket\") pod \"a9dcab5f-6209-4edd-8832-d31608f3c7d3\" (UID: \"a9dcab5f-6209-4edd-8832-d31608f3c7d3\") "
Jul 06 15:01:21.285451 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:21.285402    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a9dcab5f-6209-4edd-8832-d31608f3c7d3-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a9dcab5f-6209-4edd-8832-d31608f3c7d3" (UID: "a9dcab5f-6209-4edd-8832-d31608f3c7d3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 15:01:21.285655 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:21.285632    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a9dcab5f-6209-4edd-8832-d31608f3c7d3-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a9dcab5f-6209-4edd-8832-d31608f3c7d3" (UID: "a9dcab5f-6209-4edd-8832-d31608f3c7d3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 15:01:21.285732 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:21.285714    2559 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a9dcab5f-6209-4edd-8832-d31608f3c7d3-workload-socket\") on node \"ip-10-0-133-112.ec2.internal\" DevicePath \"\""
Jul 06 15:01:21.285732 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:21.285728    2559 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a9dcab5f-6209-4edd-8832-d31608f3c7d3-workload-certs\") on node \"ip-10-0-133-112.ec2.internal\" DevicePath \"\""
Jul 06 15:01:21.285846 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:21.285739    2559 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a9dcab5f-6209-4edd-8832-d31608f3c7d3-istiod-ca-cert\") on node \"ip-10-0-133-112.ec2.internal\" DevicePath \"\""
Jul 06 15:01:21.286119 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:21.286088    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a9dcab5f-6209-4edd-8832-d31608f3c7d3-istio-data" (OuterVolumeSpecName: "istio-data") pod "a9dcab5f-6209-4edd-8832-d31608f3c7d3" (UID: "a9dcab5f-6209-4edd-8832-d31608f3c7d3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 15:01:21.287362 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:21.287326    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a9dcab5f-6209-4edd-8832-d31608f3c7d3-kube-api-access-g48nr" (OuterVolumeSpecName: "kube-api-access-g48nr") pod "a9dcab5f-6209-4edd-8832-d31608f3c7d3" (UID: "a9dcab5f-6209-4edd-8832-d31608f3c7d3"). InnerVolumeSpecName "kube-api-access-g48nr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 15:01:21.287481 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:21.287374    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a9dcab5f-6209-4edd-8832-d31608f3c7d3-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a9dcab5f-6209-4edd-8832-d31608f3c7d3" (UID: "a9dcab5f-6209-4edd-8832-d31608f3c7d3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 06 15:01:21.287599 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:21.287565    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a9dcab5f-6209-4edd-8832-d31608f3c7d3-istio-token" (OuterVolumeSpecName: "istio-token") pod "a9dcab5f-6209-4edd-8832-d31608f3c7d3" (UID: "a9dcab5f-6209-4edd-8832-d31608f3c7d3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 15:01:21.287666 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:21.287592    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a9dcab5f-6209-4edd-8832-d31608f3c7d3-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a9dcab5f-6209-4edd-8832-d31608f3c7d3" (UID: "a9dcab5f-6209-4edd-8832-d31608f3c7d3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 15:01:21.358936 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:21.358893    2559 generic.go:356] "Generic (PLEG): container finished" podID="a9dcab5f-6209-4edd-8832-d31608f3c7d3" containerID="f72cf7a3a7dea6534cb4de212b96dfbf801a15a4849eba159c3a5be9153a931a" exitCode=0
Jul 06 15:01:21.358936 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:21.358934    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-aa7c3f-openshift-default-85c8575b7f-blhgz"
Jul 06 15:01:21.359205 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:21.358983    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-aa7c3f-openshift-default-85c8575b7f-blhgz" event={"ID":"a9dcab5f-6209-4edd-8832-d31608f3c7d3","Type":"ContainerDied","Data":"f72cf7a3a7dea6534cb4de212b96dfbf801a15a4849eba159c3a5be9153a931a"}
Jul 06 15:01:21.359205 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:21.359034    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-aa7c3f-openshift-default-85c8575b7f-blhgz" event={"ID":"a9dcab5f-6209-4edd-8832-d31608f3c7d3","Type":"ContainerDied","Data":"9c19490f836cbf4af74d8ae9397e5f3c1034d46d5a4feee526486c253e81afd4"}
Jul 06 15:01:21.359205 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:21.359055    2559 scope.go:117] "RemoveContainer" containerID="f72cf7a3a7dea6534cb4de212b96dfbf801a15a4849eba159c3a5be9153a931a"
Jul 06 15:01:21.371326 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:21.370783    2559 scope.go:117] "RemoveContainer" containerID="fec53383d2a08745763e163fbbf3efa194634a1b04935068c2fa60bdef007f62"
Jul 06 15:01:21.373876 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:21.373849    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-cdhfk_58a6adba-bfd3-41bd-b3f1-ea4b6c13daea/manager/0.log"
Jul 06 15:01:21.382171 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:21.382148    2559 scope.go:117] "RemoveContainer" containerID="f72cf7a3a7dea6534cb4de212b96dfbf801a15a4849eba159c3a5be9153a931a"
Jul 06 15:01:21.382360 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:21.382330    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-aa7c3f-openshift-default-85c8575b7f-blhgz"]
Jul 06 15:01:21.382523 ip-10-0-133-112 kubenswrapper[2559]: E0706 15:01:21.382504    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f72cf7a3a7dea6534cb4de212b96dfbf801a15a4849eba159c3a5be9153a931a\": container with ID starting with f72cf7a3a7dea6534cb4de212b96dfbf801a15a4849eba159c3a5be9153a931a not found: ID does not exist" containerID="f72cf7a3a7dea6534cb4de212b96dfbf801a15a4849eba159c3a5be9153a931a"
Jul 06 15:01:21.382615 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:21.382533    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f72cf7a3a7dea6534cb4de212b96dfbf801a15a4849eba159c3a5be9153a931a"} err="failed to get container status \"f72cf7a3a7dea6534cb4de212b96dfbf801a15a4849eba159c3a5be9153a931a\": rpc error: code = NotFound desc = could not find container \"f72cf7a3a7dea6534cb4de212b96dfbf801a15a4849eba159c3a5be9153a931a\": container with ID starting with f72cf7a3a7dea6534cb4de212b96dfbf801a15a4849eba159c3a5be9153a931a not found: ID does not exist"
Jul 06 15:01:21.382615 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:21.382554    2559 scope.go:117] "RemoveContainer" containerID="fec53383d2a08745763e163fbbf3efa194634a1b04935068c2fa60bdef007f62"
Jul 06 15:01:21.382851 ip-10-0-133-112 kubenswrapper[2559]: E0706 15:01:21.382832    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fec53383d2a08745763e163fbbf3efa194634a1b04935068c2fa60bdef007f62\": container with ID starting with fec53383d2a08745763e163fbbf3efa194634a1b04935068c2fa60bdef007f62 not found: ID does not exist" containerID="fec53383d2a08745763e163fbbf3efa194634a1b04935068c2fa60bdef007f62"
Jul 06 15:01:21.382916 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:21.382856    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fec53383d2a08745763e163fbbf3efa194634a1b04935068c2fa60bdef007f62"} err="failed to get container status \"fec53383d2a08745763e163fbbf3efa194634a1b04935068c2fa60bdef007f62\": rpc error: code = NotFound desc = could not find container \"fec53383d2a08745763e163fbbf3efa194634a1b04935068c2fa60bdef007f62\": container with ID starting with fec53383d2a08745763e163fbbf3efa194634a1b04935068c2fa60bdef007f62 not found: ID does not exist"
Jul 06 15:01:21.386622 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:21.386593    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-aa7c3f-openshift-default-85c8575b7f-blhgz"]
Jul 06 15:01:21.386860 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:21.386834    2559 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a9dcab5f-6209-4edd-8832-d31608f3c7d3-istio-envoy\") on node \"ip-10-0-133-112.ec2.internal\" DevicePath \"\""
Jul 06 15:01:21.386912 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:21.386869    2559 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a9dcab5f-6209-4edd-8832-d31608f3c7d3-istio-data\") on node \"ip-10-0-133-112.ec2.internal\" DevicePath \"\""
Jul 06 15:01:21.386912 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:21.386884    2559 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a9dcab5f-6209-4edd-8832-d31608f3c7d3-istio-podinfo\") on node \"ip-10-0-133-112.ec2.internal\" DevicePath \"\""
Jul 06 15:01:21.386912 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:21.386899    2559 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a9dcab5f-6209-4edd-8832-d31608f3c7d3-credential-socket\") on node \"ip-10-0-133-112.ec2.internal\" DevicePath \"\""
Jul 06 15:01:21.387017 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:21.386915    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g48nr\" (UniqueName: \"kubernetes.io/projected/a9dcab5f-6209-4edd-8832-d31608f3c7d3-kube-api-access-g48nr\") on node \"ip-10-0-133-112.ec2.internal\" DevicePath \"\""
Jul 06 15:01:21.387017 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:21.386932    2559 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a9dcab5f-6209-4edd-8832-d31608f3c7d3-istio-token\") on node \"ip-10-0-133-112.ec2.internal\" DevicePath \"\""
Jul 06 15:01:21.495148 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:21.495057    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-rbh57_4c04aa96-f5dc-4734-9d06-e53704e040f1/manager/0.log"
Jul 06 15:01:21.620677 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:21.620647    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-c7fb74df8-wgl2x_8f1e0af1-095f-4f50-ab37-d5d06814aa49/maas-api/0.log"
Jul 06 15:01:21.741242 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:21.741214    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29722470-zdppp_daa61e06-6298-40c6-8e9d-ca593a1638b8/cleanup/0.log"
Jul 06 15:01:21.853360 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:21.853274    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29722485-97sgd_97633ee4-b353-43f6-ad58-8b726df28551/cleanup/0.log"
Jul 06 15:01:22.083387 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:22.083350    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-58b9bc5c65-m78gp_ba19bb7d-8ef5-4e75-884d-cb4a9a25d7e7/manager/0.log"
Jul 06 15:01:22.195448 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:22.195421    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-662tf_3697cccd-51bf-47c0-8db1-7237afd847a1/server/0.log"
Jul 06 15:01:22.228005 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:22.227971    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a9dcab5f-6209-4edd-8832-d31608f3c7d3" path="/var/lib/kubelet/pods/a9dcab5f-6209-4edd-8832-d31608f3c7d3/volumes"
Jul 06 15:01:22.312683 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:22.312645    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-hdzl7_4729c197-d9fa-4278-a2b0-79812874376a/manager/0.log"
Jul 06 15:01:22.542807 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:22.542718    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6ccf4f7c96-998d9_0143a9db-b269-4df3-9bd5-d489eb70716c/copy-manifests/0.log"
Jul 06 15:01:22.563258 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:22.563221    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6ccf4f7c96-998d9_0143a9db-b269-4df3-9bd5-d489eb70716c/manager/0.log"
Jul 06 15:01:22.799407 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:22.799322    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-69fb4594cf-6rvg5_6f446a12-03fd-44e7-8fe4-2a6346653d8c/postgres/0.log"
Jul 06 15:01:25.409770 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:25.409730    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-w5jmh_a0a508b6-5385-43de-8245-76384185d86c/discovery/0.log"
Jul 06 15:01:25.635625 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:25.635561    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-64c7864974-tk5t2_53e0f0e5-2817-4b79-b14e-967b32b18de6/kube-auth-proxy/0.log"
Jul 06 15:01:25.761739 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:25.761702    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-vszcz_065f6787-1fa0-40e0-ae75-6f87a4df4d10/istio-proxy/1.log"
Jul 06 15:01:25.887359 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:25.887307    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-clksh_10c09d8b-de7d-4798-a71c-971997b4fbe0/payload-pre-processing/0.log"
Jul 06 15:01:26.008344 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:26.008305    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-56f8d_f7bed500-2a8f-4fab-aaa4-1122ffdc412b/payload-processing/0.log"
Jul 06 15:01:26.472168 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:26.472126    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-zpmrz_0b873ff1-9a28-4917-97af-09a20796835f/main/0.log"
Jul 06 15:01:26.821359 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:26.821279    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfp7wv_b7f32cfe-8038-4da0-948e-95ecda736905/main/0.log"
Jul 06 15:01:34.804763 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:34.804727    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-qjsqm_ab514f6f-ea23-486a-b065-aac1b24a5035/global-pull-secret-syncer/0.log"
Jul 06 15:01:34.972056 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:34.972023    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-qr4tr_6a513fd4-439d-4806-afbc-6462321a6d29/konnectivity-agent/0.log"
Jul 06 15:01:34.999519 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:34.999488    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-133-112.ec2.internal_112772d63ce9100ee913123b6eaccc48/haproxy/0.log"
Jul 06 15:01:43.766107 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:43.766068    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-tq6gr_0383f486-87cb-45e9-aa3d-99bf4066cd9d/node-exporter/0.log"
Jul 06 15:01:43.810632 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:43.810599    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-tq6gr_0383f486-87cb-45e9-aa3d-99bf4066cd9d/kube-rbac-proxy/0.log"
Jul 06 15:01:43.855616 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:43.855563    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-tq6gr_0383f486-87cb-45e9-aa3d-99bf4066cd9d/init-textfile/0.log"
Jul 06 15:01:44.493209 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:44.493180    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-d559d66d5-z4962_9019944c-f0a1-4309-af53-817d7f767b10/prometheus-operator-admission-webhook/0.log"
Jul 06 15:01:45.726203 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:45.726161    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-25f49/perf-node-gather-daemonset-8ldc9"]
Jul 06 15:01:45.726786 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:45.726768    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a9dcab5f-6209-4edd-8832-d31608f3c7d3" containerName="istio-proxy"
Jul 06 15:01:45.726873 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:45.726789    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="a9dcab5f-6209-4edd-8832-d31608f3c7d3" containerName="istio-proxy"
Jul 06 15:01:45.726873 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:45.726808    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9f4c88f0-8bd9-43a1-a648-a312759fe26d" containerName="maas-api"
Jul 06 15:01:45.726873 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:45.726817    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="9f4c88f0-8bd9-43a1-a648-a312759fe26d" containerName="maas-api"
Jul 06 15:01:45.726873 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:45.726831    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8b9bfd85-5b75-4113-b0f6-656bd17770d4" containerName="istio-proxy"
Jul 06 15:01:45.726873 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:45.726839    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="8b9bfd85-5b75-4113-b0f6-656bd17770d4" containerName="istio-proxy"
Jul 06 15:01:45.726873 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:45.726850    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a9dcab5f-6209-4edd-8832-d31608f3c7d3" containerName="istio-proxy"
Jul 06 15:01:45.726873 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:45.726858    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="a9dcab5f-6209-4edd-8832-d31608f3c7d3" containerName="istio-proxy"
Jul 06 15:01:45.727231 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:45.726887    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8b9bfd85-5b75-4113-b0f6-656bd17770d4" containerName="istio-proxy"
Jul 06 15:01:45.727231 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:45.726896    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="8b9bfd85-5b75-4113-b0f6-656bd17770d4" containerName="istio-proxy"
Jul 06 15:01:45.727231 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:45.726912    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d0b274dc-94ae-4af7-b562-8929db8b7ac3" containerName="maas-api"
Jul 06 15:01:45.727231 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:45.726921    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="d0b274dc-94ae-4af7-b562-8929db8b7ac3" containerName="maas-api"
Jul 06 15:01:45.727231 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:45.727014    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="a9dcab5f-6209-4edd-8832-d31608f3c7d3" containerName="istio-proxy"
Jul 06 15:01:45.727231 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:45.727028    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="8b9bfd85-5b75-4113-b0f6-656bd17770d4" containerName="istio-proxy"
Jul 06 15:01:45.727231 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:45.727039    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="d0b274dc-94ae-4af7-b562-8929db8b7ac3" containerName="maas-api"
Jul 06 15:01:45.727231 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:45.727049    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="9f4c88f0-8bd9-43a1-a648-a312759fe26d" containerName="maas-api"
Jul 06 15:01:45.727231 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:45.727058    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="8b9bfd85-5b75-4113-b0f6-656bd17770d4" containerName="istio-proxy"
Jul 06 15:01:45.730205 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:45.730185    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-25f49/perf-node-gather-daemonset-8ldc9"
Jul 06 15:01:45.733405 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:45.733363    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-25f49\"/\"kube-root-ca.crt\""
Jul 06 15:01:45.734227 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:45.734210    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-25f49\"/\"default-dockercfg-c76v2\""
Jul 06 15:01:45.734288 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:45.734256    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-25f49\"/\"openshift-service-ca.crt\""
Jul 06 15:01:45.748236 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:45.748207    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-25f49/perf-node-gather-daemonset-8ldc9"]
Jul 06 15:01:45.816729 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:45.816680    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/861e9059-0d51-4278-b860-f6497b123597-proc\") pod \"perf-node-gather-daemonset-8ldc9\" (UID: \"861e9059-0d51-4278-b860-f6497b123597\") " pod="openshift-must-gather-25f49/perf-node-gather-daemonset-8ldc9"
Jul 06 15:01:45.816729 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:45.816731    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/861e9059-0d51-4278-b860-f6497b123597-sys\") pod \"perf-node-gather-daemonset-8ldc9\" (UID: \"861e9059-0d51-4278-b860-f6497b123597\") " pod="openshift-must-gather-25f49/perf-node-gather-daemonset-8ldc9"
Jul 06 15:01:45.816986 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:45.816761    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/861e9059-0d51-4278-b860-f6497b123597-podres\") pod \"perf-node-gather-daemonset-8ldc9\" (UID: \"861e9059-0d51-4278-b860-f6497b123597\") " pod="openshift-must-gather-25f49/perf-node-gather-daemonset-8ldc9"
Jul 06 15:01:45.816986 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:45.816816    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/861e9059-0d51-4278-b860-f6497b123597-lib-modules\") pod \"perf-node-gather-daemonset-8ldc9\" (UID: \"861e9059-0d51-4278-b860-f6497b123597\") " pod="openshift-must-gather-25f49/perf-node-gather-daemonset-8ldc9"
Jul 06 15:01:45.816986 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:45.816842    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pn7l9\" (UniqueName: \"kubernetes.io/projected/861e9059-0d51-4278-b860-f6497b123597-kube-api-access-pn7l9\") pod \"perf-node-gather-daemonset-8ldc9\" (UID: \"861e9059-0d51-4278-b860-f6497b123597\") " pod="openshift-must-gather-25f49/perf-node-gather-daemonset-8ldc9"
Jul 06 15:01:45.917879 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:45.917840    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/861e9059-0d51-4278-b860-f6497b123597-proc\") pod \"perf-node-gather-daemonset-8ldc9\" (UID: \"861e9059-0d51-4278-b860-f6497b123597\") " pod="openshift-must-gather-25f49/perf-node-gather-daemonset-8ldc9"
Jul 06 15:01:45.917879 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:45.917882    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/861e9059-0d51-4278-b860-f6497b123597-sys\") pod \"perf-node-gather-daemonset-8ldc9\" (UID: \"861e9059-0d51-4278-b860-f6497b123597\") " pod="openshift-must-gather-25f49/perf-node-gather-daemonset-8ldc9"
Jul 06 15:01:45.918145 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:45.917915    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/861e9059-0d51-4278-b860-f6497b123597-podres\") pod \"perf-node-gather-daemonset-8ldc9\" (UID: \"861e9059-0d51-4278-b860-f6497b123597\") " pod="openshift-must-gather-25f49/perf-node-gather-daemonset-8ldc9"
Jul 06 15:01:45.918145 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:45.917953    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/861e9059-0d51-4278-b860-f6497b123597-lib-modules\") pod \"perf-node-gather-daemonset-8ldc9\" (UID: \"861e9059-0d51-4278-b860-f6497b123597\") " pod="openshift-must-gather-25f49/perf-node-gather-daemonset-8ldc9"
Jul 06 15:01:45.918145 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:45.917974    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/861e9059-0d51-4278-b860-f6497b123597-proc\") pod \"perf-node-gather-daemonset-8ldc9\" (UID: \"861e9059-0d51-4278-b860-f6497b123597\") " pod="openshift-must-gather-25f49/perf-node-gather-daemonset-8ldc9"
Jul 06 15:01:45.918145 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:45.917995    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/861e9059-0d51-4278-b860-f6497b123597-sys\") pod \"perf-node-gather-daemonset-8ldc9\" (UID: \"861e9059-0d51-4278-b860-f6497b123597\") " pod="openshift-must-gather-25f49/perf-node-gather-daemonset-8ldc9"
Jul 06 15:01:45.918145 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:45.917977    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pn7l9\" (UniqueName: \"kubernetes.io/projected/861e9059-0d51-4278-b860-f6497b123597-kube-api-access-pn7l9\") pod \"perf-node-gather-daemonset-8ldc9\" (UID: \"861e9059-0d51-4278-b860-f6497b123597\") " pod="openshift-must-gather-25f49/perf-node-gather-daemonset-8ldc9"
Jul 06 15:01:45.918145 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:45.918089    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/861e9059-0d51-4278-b860-f6497b123597-podres\") pod \"perf-node-gather-daemonset-8ldc9\" (UID: \"861e9059-0d51-4278-b860-f6497b123597\") " pod="openshift-must-gather-25f49/perf-node-gather-daemonset-8ldc9"
Jul 06 15:01:45.918145 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:45.918096    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/861e9059-0d51-4278-b860-f6497b123597-lib-modules\") pod \"perf-node-gather-daemonset-8ldc9\" (UID: \"861e9059-0d51-4278-b860-f6497b123597\") " pod="openshift-must-gather-25f49/perf-node-gather-daemonset-8ldc9"
Jul 06 15:01:45.927272 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:45.927240    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pn7l9\" (UniqueName: \"kubernetes.io/projected/861e9059-0d51-4278-b860-f6497b123597-kube-api-access-pn7l9\") pod \"perf-node-gather-daemonset-8ldc9\" (UID: \"861e9059-0d51-4278-b860-f6497b123597\") " pod="openshift-must-gather-25f49/perf-node-gather-daemonset-8ldc9"
Jul 06 15:01:46.040506 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:46.040409    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-25f49/perf-node-gather-daemonset-8ldc9"
Jul 06 15:01:46.055894 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:46.055868    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6bffdf9778-k5fj6_73b247d2-a95f-47c5-adc7-e511cbf91a3d/networking-console-plugin/0.log"
Jul 06 15:01:46.185472 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:46.185444    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-25f49/perf-node-gather-daemonset-8ldc9"]
Jul 06 15:01:46.187664 ip-10-0-133-112 kubenswrapper[2559]: W0706 15:01:46.187634    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod861e9059_0d51_4278_b860_f6497b123597.slice/crio-f8f1e8f456d4186658ff5cb36056ba0118d6eef574c3f1b9fcd18be3bc5f940c WatchSource:0}: Error finding container f8f1e8f456d4186658ff5cb36056ba0118d6eef574c3f1b9fcd18be3bc5f940c: Status 404 returned error can't find the container with id f8f1e8f456d4186658ff5cb36056ba0118d6eef574c3f1b9fcd18be3bc5f940c
Jul 06 15:01:46.481875 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:46.481836    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-25f49/perf-node-gather-daemonset-8ldc9" event={"ID":"861e9059-0d51-4278-b860-f6497b123597","Type":"ContainerStarted","Data":"89be77bee6ec8d4fc53926df5d50af16b72774dd5c7dc18cd25432a7f47675ed"}
Jul 06 15:01:46.482074 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:46.481896    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-25f49/perf-node-gather-daemonset-8ldc9" event={"ID":"861e9059-0d51-4278-b860-f6497b123597","Type":"ContainerStarted","Data":"f8f1e8f456d4186658ff5cb36056ba0118d6eef574c3f1b9fcd18be3bc5f940c"}
Jul 06 15:01:46.482074 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:46.481955    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-25f49/perf-node-gather-daemonset-8ldc9"
Jul 06 15:01:46.500536 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:46.500483    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-25f49/perf-node-gather-daemonset-8ldc9" podStartSLOduration=1.5004692849999999 podStartE2EDuration="1.500469285s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 15:01:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 15:01:46.498299195 +0000 UTC m=+2528.805591276" watchObservedRunningTime="2026-07-06 15:01:46.500469285 +0000 UTC m=+2528.807761365"
Jul 06 15:01:46.679241 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:46.679213    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-g5w5m_e35b0247-7c4e-4bd1-bc28-c084dc40ffa1/console-operator/1.log"
Jul 06 15:01:46.687660 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:46.687634    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-g5w5m_e35b0247-7c4e-4bd1-bc28-c084dc40ffa1/console-operator/2.log"
Jul 06 15:01:47.239880 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:47.239848    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-685bfb5f64-psvjq_2855a4b2-bc67-4364-b3f0-40fa0c71c599/download-server/0.log"
Jul 06 15:01:49.019444 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:49.019418    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-kfvv5_e1c99840-57b6-4e0b-8e6d-80bca474168d/dns/0.log"
Jul 06 15:01:49.043249 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:49.043222    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-kfvv5_e1c99840-57b6-4e0b-8e6d-80bca474168d/kube-rbac-proxy/0.log"
Jul 06 15:01:49.189728 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:49.189699    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-9tndp_f07d461c-9de3-43c8-b501-f50c0cd463eb/dns-node-resolver/0.log"
Jul 06 15:01:49.861584 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:49.861550    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-564974887b-25vkr_1ef4105d-8624-4b20-b82c-9130feb99981/registry/0.log"
Jul 06 15:01:49.891005 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:49.890976    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-pr7f9_ef2ced35-fb83-46cb-9e31-2c0814bd5db2/node-ca/0.log"
Jul 06 15:01:51.291784 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:51.291744    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-w5jmh_a0a508b6-5385-43de-8245-76384185d86c/discovery/0.log"
Jul 06 15:01:51.342599 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:51.342555    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-64c7864974-tk5t2_53e0f0e5-2817-4b79-b14e-967b32b18de6/kube-auth-proxy/0.log"
Jul 06 15:01:51.378289 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:51.378254    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-vszcz_065f6787-1fa0-40e0-ae75-6f87a4df4d10/istio-proxy/0.log"
Jul 06 15:01:51.433818 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:51.433783    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-vszcz_065f6787-1fa0-40e0-ae75-6f87a4df4d10/istio-proxy/1.log"
Jul 06 15:01:51.475397 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:51.475361    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-clksh_10c09d8b-de7d-4798-a71c-971997b4fbe0/payload-pre-processing/0.log"
Jul 06 15:01:51.548497 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:51.548408    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-56f8d_f7bed500-2a8f-4fab-aaa4-1122ffdc412b/payload-processing/0.log"
Jul 06 15:01:52.357440 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:52.357410    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-7r8pk_e19cda1c-b504-43fa-8dfb-8f53092dd8d8/serve-healthcheck-canary/0.log"
Jul 06 15:01:52.496584 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:52.496544    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-25f49/perf-node-gather-daemonset-8ldc9"
Jul 06 15:01:52.978853 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:52.978810    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5c847dcfc4-2s94h_de6193cf-468b-4733-934b-71d909d33445/insights-operator/0.log"
Jul 06 15:01:52.979041 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:52.978927    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5c847dcfc4-2s94h_de6193cf-468b-4733-934b-71d909d33445/insights-operator/1.log"
Jul 06 15:01:53.006030 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:53.005999    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-lghzh_38ad328a-62dd-4a21-aaf9-3141c532fafb/kube-rbac-proxy/0.log"
Jul 06 15:01:53.029852 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:53.029826    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-lghzh_38ad328a-62dd-4a21-aaf9-3141c532fafb/exporter/0.log"
Jul 06 15:01:53.053415 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:53.053382    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-lghzh_38ad328a-62dd-4a21-aaf9-3141c532fafb/extractor/0.log"
Jul 06 15:01:55.819664 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:55.819636    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-cdhfk_58a6adba-bfd3-41bd-b3f1-ea4b6c13daea/manager/0.log"
Jul 06 15:01:56.036946 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:56.036917    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-rbh57_4c04aa96-f5dc-4734-9d06-e53704e040f1/manager/0.log"
Jul 06 15:01:56.080865 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:56.080835    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-c7fb74df8-wgl2x_8f1e0af1-095f-4f50-ab37-d5d06814aa49/maas-api/0.log"
Jul 06 15:01:56.124602 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:56.124558    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29722470-zdppp_daa61e06-6298-40c6-8e9d-ca593a1638b8/cleanup/0.log"
Jul 06 15:01:56.154130 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:56.154096    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29722485-97sgd_97633ee4-b353-43f6-ad58-8b726df28551/cleanup/0.log"
Jul 06 15:01:56.345439 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:56.345381    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-58b9bc5c65-m78gp_ba19bb7d-8ef5-4e75-884d-cb4a9a25d7e7/manager/0.log"
Jul 06 15:01:56.370388 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:56.370356    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-662tf_3697cccd-51bf-47c0-8db1-7237afd847a1/server/0.log"
Jul 06 15:01:56.476379 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:56.476353    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-hdzl7_4729c197-d9fa-4278-a2b0-79812874376a/manager/0.log"
Jul 06 15:01:56.649621 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:56.649502    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6ccf4f7c96-998d9_0143a9db-b269-4df3-9bd5-d489eb70716c/manager/0.log"
Jul 06 15:01:56.667942 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:56.667915    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6ccf4f7c96-998d9_0143a9db-b269-4df3-9bd5-d489eb70716c/copy-manifests/0.log"
Jul 06 15:01:56.756686 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:56.756657    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-69fb4594cf-6rvg5_6f446a12-03fd-44e7-8fe4-2a6346653d8c/postgres/0.log"
Jul 06 15:01:56.863319 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:56.863291    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-w5jmh_a0a508b6-5385-43de-8245-76384185d86c/discovery/0.log"
Jul 06 15:01:56.896275 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:56.896244    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-64c7864974-tk5t2_53e0f0e5-2817-4b79-b14e-967b32b18de6/kube-auth-proxy/0.log"
Jul 06 15:01:56.924131 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:56.924106    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-vszcz_065f6787-1fa0-40e0-ae75-6f87a4df4d10/istio-proxy/0.log"
Jul 06 15:01:56.976702 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:56.976667    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-vszcz_065f6787-1fa0-40e0-ae75-6f87a4df4d10/istio-proxy/1.log"
Jul 06 15:01:57.006800 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:57.006756    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-clksh_10c09d8b-de7d-4798-a71c-971997b4fbe0/payload-pre-processing/0.log"
Jul 06 15:01:57.071382 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:57.071345    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-56f8d_f7bed500-2a8f-4fab-aaa4-1122ffdc412b/payload-processing/0.log"
Jul 06 15:01:58.785156 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:58.785073    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-w5jmh_a0a508b6-5385-43de-8245-76384185d86c/discovery/0.log"
Jul 06 15:01:58.819776 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:58.819750    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-64c7864974-tk5t2_53e0f0e5-2817-4b79-b14e-967b32b18de6/kube-auth-proxy/0.log"
Jul 06 15:01:58.845056 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:58.845023    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-vszcz_065f6787-1fa0-40e0-ae75-6f87a4df4d10/istio-proxy/0.log"
Jul 06 15:01:58.901518 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:58.901491    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-vszcz_065f6787-1fa0-40e0-ae75-6f87a4df4d10/istio-proxy/1.log"
Jul 06 15:01:58.940178 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:58.940142    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-clksh_10c09d8b-de7d-4798-a71c-971997b4fbe0/payload-pre-processing/0.log"
Jul 06 15:01:59.012046 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:01:59.012007    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-56f8d_f7bed500-2a8f-4fab-aaa4-1122ffdc412b/payload-processing/0.log"
Jul 06 15:02:04.208875 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:02:04.208840    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-pfrz4_8a987d28-7a8b-4a1c-a9a8-551bf382b523/migrator/0.log"
Jul 06 15:02:04.241301 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:02:04.241270    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-pfrz4_8a987d28-7a8b-4a1c-a9a8-551bf382b523/graceful-termination/0.log"
Jul 06 15:02:04.716765 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:02:04.716727    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-846bf668f7-q7t4k_aecb0b51-89db-4843-9c3c-960ac76a1697/kube-storage-version-migrator-operator/1.log"
Jul 06 15:02:04.722399 ip-10-0-133-112 kubenswrapper[2559]: I0706 15:02:04.722369    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-846bf668f7-q7t4k_aecb0b51-89db-4843-9c3c-960ac76a1697/kube-storage-version-migrator-operator/0.log"